Amazon Bedrock Prompt Caching for Open-Weight Models
Bedrock can cache a stable prompt prefix. Implicit is automatic. Explicit needs a mode flag and a breakpoint. The model card sets the minimum length.
By Muhammad Saif and Aashan Khan · Techtribe co-founders
LLM prompt caching is the difference between paying to reread the same manual on every question and paying to reread only the question. On November 16, 2026, Dustin Liu, Aman Arora, Melanie Li, and Saurabh Trikande published a Bedrock guide showing that open-weight models on Amazon Bedrock can now do this explicitly. Implicit caching, where Bedrock notices a repeated prefix on its own, was already the fit for conversations. Explicit caching is the fit when you know the shared part and you do not want the unique part written into the cache at a higher rate.
The mechanism is not magic and it is not a discount you turn on in a console and forget. Prompt caching reuses the computed state of a matching prefix. Bedrock holds that cache. Your application still sends the full prompt. If the beginning matches a previous request, Bedrock reads that beginning from cache and processes the rest. Cache reads cost less than ordinary input tokens. Cache writes, on some models, cost more than ordinary input. The savings show up when the same prefix is read many times. They disappear when every request is unique, and they can reverse when you pay a write premium for text nobody asks for again.
This is the implementation companion to what agent loops do to the token bill. That post is why the bill grows. This one is one lever on Bedrock, for the models and regions whose cards say the lever exists. It is not a number I can promise you. AWS tells you to read the price page per model. Do that. A blog that invents a percentage is how teams turn the feature on, see a higher bill, and conclude caching is a scam. The higher bill is usually a write of content that never hit again.
Implicit and explicit are different products
AWS draws the line cleanly in the November 16 post.
Implicit caching needs no flag. Bedrock reuses eligible prefixes on a best-effort basis. In a chat, each request resends the whole history, so yesterday's request is the prefix of today's. That works when the start of the prompt stays put. AWS's own figure shows a conversation where an early edit on a later turn rebuilds the cache from the edit. Best-effort means you do not get a contract that a particular prefix will hit. You get a chance, and you should still structure the prompt so the chance is real.
Explicit caching is a request change. On the Chat Completions API you set prompt cache options so the mode is explicit, and you put one or more prompt cache breakpoint markers on the text parts you want to freeze. Content up to and including each breakpoint is written once and read later. Content after the last breakpoint is processed on every call and billed as normal input. You can stack breakpoints, up to the maximum the model allows, so a tool list that changes rarely and a document that changes daily are different layers. The model card states the minimum tokens before a breakpoint, the maximum number of breakpoints, and the time to live. The time to live resets when the entry is hit. If you wait too long between calls, you pay to write it again.
The guide's warning about implicit mode is the one to memorize. Implicit caching may write text that later requests never reuse, such as a question that is different every time. If the tail changes on every call, switch to explicit and put the breakpoint at the end of the shared prefix. Then you do not pay a premium write for the tail. On models that charge more for writes, the tail stays at the normal input rate. That is the whole design. Mark the stable part. Leave the question outside.
AWS also says prompts and outputs on Bedrock are not shared with the model provider or used to train models, and points at the Bedrock data-protection page. Caching is not a backdoor into another customer's prompt. It is still your data in your account's cache. Treat the cache as a copy of the prefix with a TTL, not as a place secrets go to die. A system prompt that contains a live key is a bad system prompt with or without caching. Take the key out. The cache will faithfully reuse whatever you put at the front, including the mistake.
What the guide actually demonstrates
The post's example is a support bot for a fictional water filter. The system prompt holds a short instruction and an owner's manual of about four thousand tokens. Each request asks a different customer question. That is the canonical shape: a long stable document, a short changing question. They send it through the OpenAI Python SDK against the Bedrock Chat Completions endpoint, with a short-lived Bedrock token minted from AWS credentials rather than a long-term key stored on disk. The token helper they name is valid for up to twelve hours. You do not need a permanent secret in the repo to try the sample. Do not create one anyway because a sample repo did.
They then show the same API under OpenCode, a coding agent, using implicit caching, because an agent session already resends a growing transcript that shares a prefix. The version pins in the post are specific: OpenAI SDK 3.26.0 or later, the token generator package 1.1.0 or later, OpenCode 1.18.35, Python 3.12. Those pins will age. The durable part is the shape of the prompt, not the package number. If you copy the sample, pin what you ran, and re-read the model card when you change models. Support for the mode, the minimum length, and the region are properties of the model, not of Bedrock as a blob.
Usage comes back on the response. The guide tells you to read the cache fields so you can see reads and writes instead of guessing from the invoice. Do that in your own client. Log cache read tokens, cache write tokens, and ordinary input tokens next to the model id. A week of those three numbers will tell you whether the breakpoint is in the right place. A week of "it feels faster" will not. If reads are near zero, the prefix is not stable or it is under the minimum. If writes are high and reads are low, you are paying to cache the question. Move the breakpoint.
Minimums are why a short system prompt does not cache. AWS's general prompt-caching documentation says checkpoints currently want on the order of hundreds to thousands of tokens before they stick, and that a checkpoint placed too early still lets the request succeed. The prefix just is not cached. Success with no savings is the confusing outcome. Check the card. A 200-token policy pasted in front of a customer message will not meet a 1,024-token minimum no matter how correctly you set the flag. Add the real stable context, or do not bother with the flag.
Time to live is why a nightly batch that runs once does not benefit. If the TTL is five minutes on one model and thirty on another, a user who asks a follow-up after lunch misses the cache. The guide says the TTL resets on a hit, so an active conversation keeps the entry warm and a cold start pays the write. Design the product for that. Do not promise a discount on the first message of the day.
Where to put the breakpoint in an agent
Agent loops are the expensive case, because they resend the system prompt, the tool definitions, and the whole thread on every step. That prefix is exactly what implicit caching is for, if it stays byte-for-byte stable. A tool list that gets sorted differently each call, a timestamp in the system prompt, a user name injected at the top: each of those is an early edit, and AWS says the cache rebuilds from the edit. Put the clock and the user at the end. Put the tools in a stable order. Serialize JSON the same way every time. A cosmetic reorder is a cache miss you will not see in the answer, only in the bill.
Use explicit breakpoints when the agent has layers that change at different speeds. The tool schema changes when you deploy. The retrieved document changes per ticket. The user's latest sentence changes per call. Three layers, three breakpoints, if the model allows three and each prefix meets the minimum. If the model allows one, put it after the tool schema and accept that the document is reprocessed. That is still a win when the schema is large. It is not a win when the schema is small and the document is the cost. Measure the tokens, then place the mark. Do not place it by superstition at the end of the whole prompt, or you will cache the question too.
LLM development that hides the prompt builder in twelve call sites will not keep the prefix stable. One function should assemble the cached head. Tests should assert the head is identical across two calls that differ only in the question. That test is cheaper than a month of cache misses. If an agent or a teammate "improves" the system prompt by adding a date, the test fails. Good.
Open-weight models on Bedrock are still someone else's weights behind your account. Read the license on the model card before you send customer text, which the guide also says. Caching does not change the license. It does not make a model private in the sense of running on your laptop. Running a model locally is the other cost lever, for models that fit. Bedrock caching is the lever when you want their API and a repeated prefix. Use the one you actually operate. Do not turn on caching for a local server that does not implement it and then wonder why the usage fields are empty.
Failures that look like the feature is on
The request succeeds, the answer looks right, and the invoice does not move. Usually the prefix is under the minimum, the mode flag is missing, the breakpoint is on a part the API ignores, or the region you called does not enable the feature for that model. The guide is explicit that availability varies by model and region. A deploy that works in one region and silently does not cache in another is a config bug. Log the region next to the usage fields.
The invoice goes up. You are writing more than you read. Move the breakpoint earlier, or stop caching a per-request document. Implicit mode on a workload of unique documents is the classic own goal. Switch that workload to explicit, or turn the hope off.
Latency does not drop on the first call. The first call pays the write. The second call is the one that should be cheaper and often faster, if the entry is still alive. Benchmark two calls, not one. A load test that uses a fresh prefix every time is a test of the feature being disabled. Use one shared prefix and many questions.
A coding agent that edits the system prompt as part of its scratchpad will invalidate the cache on every step. Keep scratch space after the breakpoint. The agent's notes are the question, not the manual.
Secrets in the cached prefix live for the TTL in a place you might not include in your deletion story. Do not put customer records you promised to delete into a prefix with a long TTL. Put a retrieval step after the breakpoint if the record must be fresh and forgettable. Caching and retention are the same design.
A support bot is the easy demo because the manual does not change. An agent is harder because the prefix includes tool results from the previous step, and those results are supposed to change. Cache the instructions and the tool schemas. Do not try to cache the observation stream. If you do, the second step misses, you pay a write, and you have added complexity for nothing. Draw the line in the prompt builder with a comment that says which half is the breakpoint. The next person who adds a "current date" to the system prompt will otherwise do it on the cached side and quietly double the bill. The test that hashes the head is how you catch them.
What to change in the client
- Read the model card for your exact model: modes, minimum tokens, breakpoint cap, TTL, regions, and whether writes cost more than input.
- Put static text first and the new question last. Stabilize tool order and JSON encoding.
- For a shared document plus a changing question, set explicit mode and one breakpoint at the end of the document. Confirm the prefix meets the minimum.
- Log cache reads, cache writes, and normal input. Alert if writes dominate for a day.
- Use a short-lived credential the way the guide does. Do not commit a long-term Bedrock key because a sample felt easier.
- Add a test that two requests with the same head and different tails produce the same cached prefix.
- Re-check the card when you change models. A flag that worked on one open-weight model is not a promise about the next.
The November 16 guide is a how-to, not a coupon. Structure the prompt, mark the stable part, and read the usage fields. The discount is what those fields say, not what the launch post hopes.
If this is your app, send the repo.
A 30-minute scope call. We read what is live, name what is dangerous, and quote a fixed scope.
How Techtribe puts a cache breakpoint where the tokens actually repeat
We read the prompt your app sends, split the stable prefix from the per-request tail, and wire the Bedrock flag only if the model card says the prefix is long enough to count. You get the usage log and a test that the head does not change when the question does. We do not turn caching on for a unique prompt and call it a saving.
A single chat or agent loop can be a fixed pass. A product with many call sites and no shared prompt builder is a Production MVP if the rest of the path needs that level of care. The scope call is 30 minutes with a founder, and it is where we say which you are in.
Bring a sample request and last month's token bill if you have it. If every call builds the prompt in a different file, say that first.
The fixed scope for taking a generated app to production is a page per tool: Lovable, Bolt, v0, and Replit.
Questions
What did AWS announce for Bedrock prompt caching?
On November 16, 2026 AWS published a guide showing explicit prompt caching for open-weight models on Amazon Bedrock, through the Chat Completions API. Implicit caching can already reuse a stable prefix automatically. Explicit caching is how you mark the prefix yourself.
What has to be set for explicit caching?
AWS says you set prompt cache options to explicit mode and place one or more prompt cache breakpoint markers on text parts. Content up to and including each breakpoint is what gets cached. Content after the last breakpoint is processed every time.
Does caching always save money?
Cache reads are billed below the normal input rate. On some models, cache writes cost more than normal input. If you cache a prefix you never reuse, you can pay more. Check the model card for the minimum prefix length, the write price, and how long the entry lives.
Where should the stable text go?
Put the system prompt, tool definitions, and reference documents first. Put the user's new question last. If an early part changes, AWS says the cache rebuilds from that point.