Updated 2026-11-11. Two Amazon engineers replaced Claude Sonnet 4.6 with a decision model on an internal ticketing system and reported about 44 times cheaper, a 98% reduction at published list prices [1]. The arithmetic behind that claim is reproducible, and that is what this guide is for.
# Copy-paste starter: what your current model costs per task, from your own log. # Feed it the token counts you already have and the prices you already pay. python3 cost_per_task.py --in-tokens 6554 --out-tokens 170 \ --in-price 3.00 --out-price 15.00
It answers a closed question about a piece of text and nothing else: which of these twenty labels fits, is this urgent yes or no, how frustrated is this writer on a three-level scale. You give it a state (the ticket) and one or more typed questions, and it returns the chosen option with a probability for each option and a confidence. It cannot write a summary, plan a task or explain itself. That is the point: it is a classifier with a serving API, not a small language model, and the documentation is explicit that it is not a drop-in replacement for the model behind your coding agent [2][3].
The trade is worth stating plainly before the numbers. You give up the model's ability to handle an open-ended task, and in exchange the per-task cost stops being a function of how many tokens you paste in.
Three things about that table matter more than the ratio. The engineers who wrote it work on the system they measured, so treat it as a first-party claim rather than an independent benchmark [1]. The prices are published list prices for us-east-1 on-demand and they exclude the Lambda that now runs for less time per ticket [1]. And their own summary of the win is "performed just as well at 98% less cost", which is an outcome claim resting on their comparison procedure rather than a published accuracy figure. An independent measurement of the same model family, published the next day and quoted below, puts it behind the best general models on its own test. The Hacker News thread for the case study went from 154 points and 45 comments to 710 and 247 in the 26 hours after it was posted [5], so this is live rather than settled.
The comparison procedure is the part worth copying, and it is the reason this is not just a cheaper-model story.
A second measurement was published the day after the case study, by an AWS solutions architect, on frames from security cameras rather than tickets. It is the honest half of this story: "On a hand-labeled set of camera stills it scored 85.2% with no fine-tuning, against 88.0% for the best vision LLMs on Amazon Bedrock" [6]. Two things follow from that sentence.
Quality parity is not established. 85.2% against 88.0% is close enough that a cost-per-task decision can still go the decider's way, and far enough that "just as well" is a property of their task on their tickets rather than of the approach. The same article records the model at its 0.5 default scoring 74.6%, which is the second thing and the more dangerous one.
The threshold is a decision you own. From the article's rules: "Do not use 0.5 as the cutoff. At 0.5 the decider leaned hard toward "no" and missed 43% of actual events. On my data the useful range was 0.2 to 0.3" [6]. A decision model returns a probability, and a probability is not a decision. Take the default threshold and you can land on a confidently wrong operating point, then blame the model for it.
One more line from it belongs beside the cost table, because it is the plainest statement of the trade: "Strands Decider is not currently available on Amazon Bedrock (as of November 2026), so for now you host it yourself and pay for the instance by the hour, whether frames arrive or not" [6]. That is the utilisation warning from the other side.
The article carries its author's own disclaimer, "Any opinions in this article are those of the individual author and may not reflect the opinions of AWS", and records that while text fine-tunes of this model show large gains, one went from 67.7% to 96.7%, no image fine-tune had been published at the time of writing [6]. Do not assume your domain is covered by somebody else's fine-tune.
Rather than flipping a switch and hoping, the labelling Lambda reads a three-way setting from AWS AppConfig on every message [1]:
BEDROCK
DUAL
DECIDER
Their verdict on that pattern: "The part we'd copy regardless of model is the three-way switch. Running both on real traffic, then judging every disagreement, told us more than any offline eval would have. It also meant going back was a 45-second config change" [1].
Here is that switch as something you can run, with the setting outside the code so the rollback really is a config change:
{ "labelling": { "mode": "DUAL", "decider": { "endpoint": "decider-e4b-prod", "window_tokens": 4096, "sort_keys": true } } }
#!/usr/bin/env python3 """switch.py - the three-way rollout switch, read per message.""" import json, sys VALID = {"BEDROCK", "DUAL", "DECIDER"} def load(path="config.json"): with open(path) as fh: cfg = json.load(fh) mode = cfg["labelling"]["mode"].upper() if mode not in VALID: raise ValueError(f"unknown labelling mode {mode!r}; expected one of {sorted(VALID)}") return mode, cfg["labelling"]["decider"] def plan(mode): return { "BEDROCK": ["call_general_model"], "DUAL": ["call_general_model", "call_decider", "apply_general_label", "log_disagreement"], "DECIDER": ["call_decider"], }[mode] if __name__ == "__main__": mode, decider = load(sys.argv[1] if len(sys.argv) > 1 else "config.json") print(f"mode={mode} endpoint={decider['endpoint']} window={decider['window_tokens']} sort_keys={decider['sort_keys']}") for step in plan(mode): print(f" -> {step}")
Run it in DUAL, then change one key and run it again:
mode=DUAL endpoint=decider-e4b-prod window=4096 sort_keys=True -> call_general_model -> call_decider -> apply_general_label -> log_disagreement mode=DECIDER endpoint=decider-e4b-prod window=4096 sort_keys=True -> call_decider
Their exact configuration is not public: the case study gives the three values and the 45-second rollback, and stops there. The AWS surface underneath it is documented, and it explains the number. AppConfig ships four predefined deployment strategies, AppConfig.AllAtOnce, AppConfig.Canary10Percent20Minutes, AppConfig.Linear20PercentEvery6Minutes and AppConfig.Linear50PercentEvery30Seconds [4]. The agent that reads the configuration has its own knobs: AWS_APPCONFIG_EXTENSION_POLL_INTERVAL_SECONDS defaults to 45, AWS_APPCONFIG_EXTENSION_POLL_TIMEOUT_MILLIS defaults to 3000, and AWS_APPCONFIG_EXTENSION_HTTP_PORT defaults to 2772 [7].
AppConfig.AllAtOnce
AppConfig.Canary10Percent20Minutes
AppConfig.Linear20PercentEvery6Minutes
AppConfig.Linear50PercentEvery30Seconds
AWS_APPCONFIG_EXTENSION_POLL_INTERVAL_SECONDS
AWS_APPCONFIG_EXTENSION_POLL_TIMEOUT_MILLIS
AWS_APPCONFIG_EXTENSION_HTTP_PORT
That default poll interval is very likely why their rollback is 45 seconds: the agent re-reads the configuration on that cadence, so a change takes effect on the next poll. The case study does not say so, and I am not putting their number on my explanation. Read it as the reason the pattern is worth copying, not as a claim about their code.
The case study's headline number is a ratio. Your version of it depends on three inputs you can measure this week: your token counts per task, your prices, and how busy the decider instance would be. The script below takes those and prints the comparison.
#!/usr/bin/env python3 """cost_per_task.py - is a decision model cheaper than the general model you run now?""" import argparse, json def per_million(inp, out, in_price, out_price): return (inp / 1_000_000) * in_price + (out / 1_000_000) * out_price def main(): p = argparse.ArgumentParser() p.add_argument("--in-tokens", type=float, required=True) p.add_argument("--out-tokens", type=float, required=True) p.add_argument("--in-price", type=float, required=True, help="USD per 1M input tokens") p.add_argument("--out-price", type=float, required=True, help="USD per 1M output tokens") p.add_argument("--instance-hour", type=float, default=None) p.add_argument("--seconds-per-call", type=float, default=None) p.add_argument("--utilisation", type=float, default=1.0) a = p.parse_args() general = per_million(a.in_tokens, a.out_tokens, a.in_price, a.out_price) out = {"general_model_usd_per_task": round(general, 6), "general_model_usd_per_million": round(general * 1_000_000, 2)} if a.instance_hour and a.seconds_per_call: capacity = 3600 / a.seconds_per_call * a.utilisation decider = a.instance_hour / capacity out.update({ "decider_capacity_tasks_per_hour": round(capacity, 1), "decider_usd_per_task": round(decider, 6), "decider_usd_per_million": round(decider * 1_000_000, 2), "ratio_general_to_decider": round(general / decider, 1) if decider else None, "saving_pct": round((1 - decider / general) * 100, 1) if general else None, }) print(json.dumps(out, indent=2)) if __name__ == "__main__": main()
Run it with their published inputs and the general-model side reproduces their figure independently:
$ python3 cost_per_task.py --in-tokens 6554 --out-tokens 170 --in-price 3.00 --out-price 15.00 per ticket: $0.02221 per million: $22,212
Their table says $0.0222 and about $22,200. Then add the decider and the interesting part appears, because the answer is dominated by how busy the instance is rather than by anything clever:
$ ... --instance-hour 1.408 --seconds-per-call 0.640 --utilisation 1.0 capacity: 5625.0/hour | per million: $250 | ratio: 88.7x | saving: 98.9% $ ... --instance-hour 1.408 --seconds-per-call 0.640 --utilisation 0.5 per million: $501 | ratio: 44.4x | saving: 97.7%
At full use, one ml.g5.xlarge at $1.408 per hour handles about 5,625 requests an hour at 640 ms each, which is about $250 per million tickets and an 88x improvement. Their production figure is about $500 per million, "at full use ... we keep headroom for bursts" [1], which lands on 44x, the number in the headline. Both figures are theirs; the point is that halving utilisation halves the advantage, and that is the variable a small business typically gets wrong, because a ticketing system with quiet nights is not busy 100% of the time.
ml.g5.xlarge
If your volume is low, that changes the answer again: an instance is billed by the hour whether or not tickets arrive, so at a few hundred tickets a day a per-token API can stay cheaper than a per-hour instance. The independent measurement puts the same point in one sentence: you "host it yourself and pay for the instance by the hour, whether frames arrive or not" [6]. The script prints the ratio precisely so you can see which side of that line you are on.
The reference implementation is open and small enough to run locally [2]. Its serving documentation gives the exact shapes, so there is no need to guess [3]:
python3.12 -m venv .venv && source .venv/bin/activate pip install -e ".[dev]" strands-decider serve checkpoints/hobson-2b-recipe --port 8000
The HTTP request takes one state and a set of typed questions. This is the documented example, verbatim [3]:
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{ "state": "Help! My payouts have been failing for 3 days.", "model": "hobson-latest", "questions": { "is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"}, "department": {"type": "choice", "instructions": "Route it.", "criteria": {"billing": "money", "technical": "bugs", "sales": "pricing"}}, "frustration":{"type": "score", "instructions": "How frustrated?", "criteria": ["Calm", "Frustrated", "Very angry"]} } }'
And this is the output the same documentation records from a v19 checkpoint, verbatim, not a mock-up [3]:
noul_0 noul = 0.801 choice_0 -> technical (confidence 0.622) technical 0.748 billing 0.234 sales 0.018 score_0 score = 1.24 (confidence 0.578) 0: calm 0.093 1: frustrated 0.578 2: very angry 0.330
The confidence column is the one to build on. 0.622 for the routing decision is a lean, not a verdict, and the project documents a routing convention that treats bands of confidence differently, including a threshold above which the convention acts without a human check [3]. A plausible-looking label at 0.55 is exactly the failure a general model hides behind fluent prose and a classifier exposes as a number.
0.622
The documentation is also candid about the serving mode it ships with, and this is worth reading as a warning rather than a footnote: the server binds 127.0.0.1, has "no authentication" by default, its "behaviour under concurrent requests is not verified", and it is intended "for local experiments only" [3]. Put it behind your own gateway before it touches customer text. The same script exposes GET /health for the model name and calibration temperature, plus the /ping and /invocations paths that SageMaker and Bedrock AgentCore call, if you deploy it as an endpoint [3].
127.0.0.1
GET /health
/ping
/invocations
The finding that surprised the engineers is the most transferable sentence in the piece: "Sending the same content with keys in a different order moved the output probabilities by up to 0.19. We now send every request with sorted keys, so if you serve a decision model behind an API, check this before you compare runs" [1].
That is testable in an afternoon, and it matters for every comparison you will ever run, including the DUAL-mode one:
#!/usr/bin/env python3 """key_order.py - does your decision endpoint depend on JSON key order? Replace call_endpoint() with your real request. It must return {"choice": str, "score": float}. """ import json, random, sys PAYLOAD = {"ticket_id": "T-10231", "priority": "high", "product": "billing", "customer_tier": "enterprise", "language": "en", "opened_hours_ago": 6} def call_endpoint(body: dict) -> dict: # stub: swap for your endpoint score = abs(hash(json.dumps(body))) % 1000 / 1000 return {"choice": "billing_question" if score > 0.4 else "account_access", "score": round(score, 3)} def variants(body): keys = list(body) yield "sorted", {k: body[k] for k in sorted(keys)} yield "reversed", {k: body[k] for k in reversed(keys)} shuffled = keys[:]; random.Random(7).shuffle(shuffled) yield "shuffled(seed=7)", {k: body[k] for k in shuffled} results = [] for name, body in variants(PAYLOAD): out = call_endpoint(body); results.append((name, out)) print(f" {name:<16} choice={out['choice']:<18} score={out['score']}") spread = round(max(r[1]["score"] for r in results) - min(r[1]["score"] for r in results), 3) print(f"\n score spread across key orders: {spread} -> " f"{'ORDER-SENSITIVE' if spread else 'stable'}") sys.exit(0 if spread == 0 else 1)
The stub is order-sensitive on purpose, which is what the real model turned out to be. Running it here:
sorted choice=billing_question score=0.453 reversed choice=billing_question score=0.85 shuffled(seed=7) choice=account_access score=0.262 score spread across key orders: 0.588 -> ORDER-SENSITIVE
Your endpoint replaces the stub, and the exit code can gate a deployment. Then judge the DUAL-mode traffic, which is where the real signal lives:
#!/usr/bin/env python3 """disagreements.py - judge every disagreement instead of sampling a few.""" import csv, sys from collections import Counter def main(path="dual-mode-log.csv"): path = sys.argv[1] if len(sys.argv) > 1 else path rows = list(csv.DictReader(open(path))) if not rows: raise SystemExit(f"{path}: no rows; nothing to judge") disagree = [r for r in rows if r["general_label"] != r["decider_label"]] with open("disagreements.jsonl", "w") as fh: for r in disagree: fh.write('{"ticket_id": "%s", "general": "%s", "decider": "%s"}\n' % (r["ticket_id"], r["general_label"], r["decider_label"])) print(f" rows: {len(rows)} disagreements: {len(disagree)} rate: {len(disagree)/len(rows):.1%}") for pair, n in Counter((r["general_label"], r["decider_label"]) for r in disagree).most_common(3): print(f" {pair[0]:<20} -> {pair[1]:<20} {n}") print(" written: disagreements.jsonl (every one, not a sample)") if __name__ == "__main__": main()
On a six-row log it prints the counts and writes every disagreement out to a file a person can read:
rows: 6 disagreements: 2 rate: 33.3% billing_question -> refund_request 1 outage_report -> account_access 1 written: disagreements.jsonl (every one, not a sample)
A 33% disagreement rate on a small sample is not a verdict on the models, it is a count of what to read. The engineering habit is the point: the ratio only means something once you have looked at the rows.
The documented failure for an over-long prompt is specific and worth designing around: a prompt longer than the checkpoint's window "is refused with HTTP 422 and a message that names the context window" [3]. The case study hit the same wall from the other side, because production's label descriptions alone were longer than the decider's 4,096-token window, so they rewrote twenty labels as "a one- or two-sentence criterion" and sent them as the options of a single choice question plus a none option [1]. A 422 with a named window is a design constraint, not a bug: your prompt has to fit, and long tickets lose their oldest comments [1].
none
Three more, captured from the scripts above rather than invented:
$ python3 cost_per_task.py --out-tokens 1 --in-price 3 --out-price 15 cost_per_task.py: error: the following arguments are required: --in-tokens
$ python3 switch.py config-bad.json ValueError: unknown labelling mode 'DECIDER_MAYBE'; expected one of ['BEDROCK', 'DECIDER', 'DUAL']
$ python3 disagreements.py missing.csv FileNotFoundError: [Errno 2] No such file or directory: 'missing.csv'
And the malformed-config error, which is the one that shows up when a hand-edited config gets a trailing comma:
SyntaxError: Expected double-quoted property name in JSON at position 44 (line 2 column 43)
The case study says it more directly than a critic would, and I would rather quote it than paraphrase: "If your labels need explanation text, if the catalog changes weekly, or if you can't fit the useful context into a few thousand tokens, a decision model probably can't replace the LLM's job" [1].
Read that as three disqualifiers. You need the answer to be a choice from a known, stable set. You need the useful context to fit in a few thousand tokens. And you need the task to be classification rather than judgement, because a decision model returns a label and a confidence, and nothing else: no explanation to show a customer, no summary to paste into a ticket, no fallback when the request is ambiguous.
Two more limits from the project's own documentation. The reference server is not production-grade out of the box (no authentication, untested concurrency) [3]. And the JevBench result needs reading carefully: 231 of 231 tasks were attempted with schema validity 1.000 through a compatibility adapter, and the documentation states that "compatibility with the Jev API itself is not verified" [3]. Schema validity is not accuracy. If you need to know whether the labels are right, that is the DUAL-mode review, not a benchmark number.
The 44x figure is real and it is conditional, and the condition is volume: an instance is billed by the hour, so the saving arrives at high, steady throughput and evaporates at a few hundred tickets a day. That is the same utilisation trap our earlier piece on what an agent costs when nobody reviews it describes from the other direction.
The durable lesson is not the model, it is the shape of the change: a task narrow enough to state as a closed question, a switch that runs both versions on live traffic, every disagreement judged by a person, and a rollback measured in seconds. That pattern is worth more than the discount, and it is the same discipline behind deciding what an agent may approve and keeping credentials out of the files agents write.
For a small team the practical question is narrower than the case study, and it is the one we ask first: which of your recurring AI tasks is actually a closed choice with a stable catalog? If you are still choosing tools rather than models, our comparison of what small-business AI automation actually does is the better starting point. We audit that before writing anything, and a written intake describing the task and its volume is enough to start.
[1] Strands Agents, "Amazon saves millions by replacing Sonnet 4.6 with Strands Decider", 2026-11-09, by the two Amazon engineers who work on the system. Source of the latency, token and price table, the three-way AppConfig switch, the 45-second rollback, the JSON key-order finding, the twenty rewritten labels, and the three disqualifiers. First-party claim about its own product, on its own internal system: treat the ratio as their measurement.
[2] strands-labs/strands-decider, the reference implementation, fetched 2026-11-11.
[3] strands-decider, docs/inference.md, fetched 2026-11-11. Source of the request payload, the recorded v19 output, the three question types, the confidence definition, the HTTP 422 window refusal, the local-serving warnings, the health and invocation paths, and the JevBench result with its stated limit.
[4] AWS AppConfig, predefined deployment strategies, fetched 2026-11-11. Source of the four AppConfig.* strategy names.
AppConfig.*
[5] Hacker News thread for the case study, submitted 2026-11-10T06:01:08Z, read through the item API on 2026-11-11: 710 points and 247 comments, against 154 and 45 on 2026-11-10.
[6] Nick Ihm, "Classifying images with Strands Decider 2B decision model", AWS Builder Center, published 2026-11-10, fetched 2026-11-11. Source of the 85.2% against 88.0% comparison, the 74.6% at the 0.5 default, the 43% miss, the 0.2 to 0.3 range, the Bedrock availability statement and the hardware requirements. Carries the author's own disclaimer and is not an AWS product commitment.
[7] AWS AppConfig Agent Lambda extension configuration, fetched 2026-11-11. Source of the environment variable names and their defaults.
Maxpertise is an AI-native engineering company. We embed native AI engineers inside your team, live in about 10 days. Please enable JavaScript to view the site, or email contact@maxpertise.net.