How to Save AI Tokens

In this post, I want to talk about how to save AI tokens.

In the early days, I focused on results and process rather than performance and cost. AI-generated output had plenty of gaps, so it needed verification, and the pressure to produce results quickly led many people, myself included, to pay for more tokens or move to a higher subscription tier whenever they ran short. I did the same. (For the first few months, I did not even pay attention to how much tokens were costing me.)

After a while, however, I became increasingly conscious of token usage. Individuals felt the burden of monthly subscriptions, while companies grew more concerned about labor and operating costs. As I wrote in The AI Agent Tool Landscape, my other articles have focused less on what AI is or how it works and more on how to use it well, what help it can provide, which tools exist, what is currently popular, and why those trends emerged. I still think those topics matter, but as time passes, cost will ultimately become the question people care about most.

I covered the mechanics of tokens—what exactly they are, how BPE creates them, and what happens inside a transformer when prompt caching lowers the unit price—in a separate article, How Tokens Work. Building on that foundation, this article first examines how costs are billed and where inefficiencies arise, then organizes proven cost-saving patterns, and finishes with a small POC that measures token usage by running the same task under several strategies.

---

Where Do Token Costs Come From?

Let us take a quick, focused look at how token costs arise and how providers bill them.

Every piece of text we send—the system prompt, tool definitions, conversation history, and user messages—becomes input tokens, while the model's response becomes output tokens. To the model, every call is a new input it has never seen before. Whether the conversation happened yesterday or the previous call was one minute ago, a new call sends the same content again in full as input. (This simple fact is the heart of token costs: the model has no memory, so we tell it everything again each time.)

One more variable enters the picture. Resending the static portion of a repeated call from scratch is expensive, so major LLM providers introduced prompt caching. Static input is stored in a cache once, and on subsequent calls, tokens read from that cache are billed at a much lower rate. (I covered the underlying mechanics—how BPE creates tokens and how the cache reuses the transformer's KV cache to lower the unit price—in How Tokens Work. Here, the focus is on how those mechanics translate into cost.)

Input Tokens, Output Tokens, and Cache Tokens

Let us look at four fields in the usage object returned by the Anthropic SDK.

• input_tokens the portion of the submitted input excluding cache reads
• output_tokens the response generated by the model
• cache_creation_input_tokens tokens stored in the cache for the first time on this call
• cache_read_input_tokens tokens read again from an existing cache

Each of these four fields is multiplied by a different rate. Output tokens are the most expensive, while tokens read from cache are the cheapest. According to Anthropic's official documentation, cache reads cost 0.1 times the base input rate—exactly 10%. Cache writes cost 1.25 times the base rate with a five-minute TTL (Time To Live, or cache validity period) and 2 times the base rate with a one-hour TTL. In other words, you pay slightly more for the first call, then save 90% from the second call onward.

Caching also has several less widely known constraints. The minimum cacheable token count varies by model and even by version within the same family. According to Anthropic's official documentation, Sonnet 4.6 and Opus 4.8 require 1,024 tokens, Opus 4.7 requires 2,048, and Haiku 4.5 plus the older Opus 4.5/4.6 require 4,096. A shorter prompt quietly goes uncached even if you add cache_control. A request can contain at most four cache_control breakpoints, and the cache is read hierarchically in the order tools → system → messages. As a result, changing even one tool definition near the front invalidates the entire cache that follows it.

The 90% discount comes from reusing a transformer's KV cache across calls, a mechanism covered in detail in How Tokens Work. From a cost perspective, there is one key point to remember: a cache hit occurs only when the prefix matches exactly. Therefore, put static content first and dynamic content that changes on every call last so the cache stays intact. Even a single timestamp character near the beginning of a prompt invalidates the entire cache after it.

This naturally raises a question: “But the input changes on every turn in a multi-turn conversation. Wouldn't that break the cache almost every time?” The short answer is no. A conversation's input is not rewritten as a fresh block on each turn; it uses an append structure that preserves everything accumulated at the front and adds only the new utterance at the end. Once the system prompt, tool definitions, and previous questions and answers are fixed, they remain unchanged, and only the latest question is appended. Compare the current turn's input with the previous turn from the beginning, and the first difference will always be the new question at the very end. The full input changes every time, but most of the prefix stays the same, so the cache survives. (The cache breaks on every turn only in a flawed design that inserts dynamic values near the front of the prompt. Tools such as Claude Code and opencode are designed to keep the front fixed and append only at the end, so users do not need to configure anything separately for caching. Typos and spelling in the user's input do not affect the cache for the same reason: that variation always occurs at the very end, outside the cacheable prefix.)

Comparing AI Providers

Putting the official Anthropic, OpenAI, and Google price lists side by side as of June 2026 makes the pattern immediately clear. (Prices are in USD per million tokens, focusing on models commonly used in everyday coding workflows.)

Provider	Model	input	cached input	output
Anthropic	Claude Opus 4.8	$5.00	$0.50	$25.00
Anthropic	Claude Sonnet 4.6	$3.00	$0.30	$15.00
Anthropic	Claude Haiku 4.5	$1.00	$0.10	$5.00
OpenAI	GPT-5.5	$5.00	$0.50	$30.00
OpenAI	GPT-5.5 Pro	$30.00	$3.00	$180.00
Google	Gemini 3.1 Pro	$2.00	$0.20	$12.00
Google	Gemini 3.5 Flash	$1.50	$0.15	$9.00

For all three providers, output tokens cost five to six times as much as input tokens. That means longer output quickly drives up the cost, even when the answer is substantively the same. Within the same tier, model prices also differ by a factor of three to six. The arithmetic alone points to three major principles for reducing token costs: request shorter output, use a cheaper model when it can produce the same answer, and cache static input. (One more calculation is revealing. Suppose you reuse a static context of 10,000 tokens 100 times. At GPT-5.5's base rate, that costs $5; as cache-read tokens, it costs $0.50. The cache therefore saves $4.50 across 100 reuses of the same context, and because the first call has almost no cache-write surcharge, the investment pays off from the second call.)

How Cache Mechanisms Differ

Even when the discount rates are similar, the internal design differs by provider. Understanding why those differences exist makes policy decisions easier.

Item	Anthropic	OpenAI	Google Gemini
Trigger	Explicit cache_control breakpoints	Automatic (no code changes needed)	Supports both automatic (implicit) and explicit caching
Minimum cache size	1,024–4,096 tokens (varies by model)	1,024+ (in increments of 128)	2,048–4,096 (varies by model)
Cache-write cost	1.25x (5 min) / 2x (1h) input rate	Free	Free (but hourly storage fees apply)
Cache-read cost	0.1x input rate	About 0.1x input rate	0.1x input rate
TTL	5 minutes or 1 hour (user-selected)	5–10 minutes of inactivity by default, up to 1 hour (24 hours when extended)	User-defined (hourly storage fees apply)
Additional cost	None	None	Storage: Flash $1/M-hour, Pro $4.50/M-hour

The design philosophy of each provider is easy to see. Anthropic asks users to mark cacheable sections explicitly, charges a modest premium on the first call (1.25 times), and then applies a deep discount. Because the prefix position is easy to control, the cache-read rate is predictable. OpenAI takes the opposite approach and automates everything. Inputs of at least 1,024 tokens are cached automatically at no additional cost, but users have less control over cache behavior. Google offers both approaches and charges a separate storage fee when users explicitly manage the cache. Implicit caching is advantageous for short, frequent reuse, while explicit caching plus storage fees suits large contexts that must remain available for an hour or more.

Tool Definitions and Tokenizers

Two more variables have a surprisingly large impact.

The first is tool definitions. In an environment with several MCP servers attached, every call includes every tool name and parameter schema in the input—we will measure how much this inflates costs later. It is also worth noting that even with the exact same tools, changing only the model changes the cost. Anthropic's official pricing documentation shows that the tool system prompt itself differs in length by model. At tool_choice: auto, Sonnet 4.6 and Haiku 4.5 use about 497 tokens, Opus 4.7 uses 675 in the same position, and Opus 4.8 drops back to 290. Before model routing, checking how verbosely the current model expands tool definitions can reveal a surprisingly large difference.

The second is the efficiency of the tokenizer, the tool that breaks human-written text into token units the model can process. As discussed in How Tokens Work, the same text can produce different token counts depending on the tokenizer. OpenAI's o200k_base uses far fewer tokens than cl100k_base for non-English text, and Anthropic states that after switching to a new tokenizer beginning with Opus 4.7, the same text may be billed as up to 35% more tokens than with previous models. Choosing a model based only on price can allow costs to leak back in through tokenizer inefficiency. An honest comparison multiplies the unit price by the expected token count.

The difference is especially pronounced in Korean. When I tokenized the same sentences with two OpenAI tokenizers, the old and new versions produced identical token counts for English technical sentences, but the older cl100k_base used 31–43% more tokens for Korean than the newer o200k_base. One Korean paragraph was especially striking: 167 characters became 169 tokens with the older tokenizer, producing more tokens than characters. On average, each Korean character consumed more than one token.

The older tokenizer splits Korean more aggressively at the byte level, while the newer one treats frequently used chunks such as “개발” and “입니다” as complete tokens. That means even at the same unit price, a Korean-heavy workload can cost more than 1.5 times as much solely because of tokenizer efficiency.

At this point, another question arises naturally: “Where exactly are we creating all this inefficiency?”

Common Ways Tokens Get Wasted

Token waste is surprisingly common even when we believe we are using AI well. Patterns I have observed in my own work and among other developers fall into several groups.

Too Many MCP Servers and Tool Definitions

I mentioned this in the previous section, but it bears repeating. Once people attach MCP servers such as Linear, GitHub, Notion, Figma, Slack, and Sentry, they rarely remove them. Unused tool schemas inflate the input token count on every call. To address this, Claude Code enables MCP Tool Search by default: only tool names and server descriptions enter the context at the beginning of a session, and the full schema is loaded only when the model actually calls that tool.

I measured how large the difference could be. Using 27 MCP tools from a Claude Code session I was working in—10 from Serena, eight across four claude.ai OAuth integrations, two from Figma, and seven from agentmemory—I sent the same user message under two configurations. One had no MCP servers installed. The other had all 27 attached but unavailable for the model to call. In both cases, the number of tool calls was zero.

With the same question, the same model, and an answer with the same meaning, input tokens alone rose from 41 → 10,335 (+10,294) on Opus 4.7. The cost of a single call increased from $0.0048 → $0.0563, about 12x. The 250-fold increase in input also raised prefill load, adding +783ms to response time. More striking than the dollar amount was that this was a cost paid on every call even though the user did not invoke a single MCP tool on that turn. This is the overhead that Tool Search prevents. (Since taking this measurement, I have continuously pruned MCP servers I do not use regularly.)

Context Accumulation and Lost in the Middle

Carrying a long conversation forward does more than increase input tokens per call; it also lowers the model's accuracy. The “Lost in the Middle” paper by Liu's Stanford research team quantified a U-shaped curve: models retrieve key information best when it appears at the beginning or end of the context, and perform noticeably worse when it is buried in the middle. It is the worst possible combination—spending more tokens for a poorer answer. Because a transformer's self-attention computation grows with the square of the token count, the absolute share of attention available to each token becomes more diluted as context length increases. The middle weakens first because training distributions tend to place important information near the beginning and end.

Looking one level deeper, the model's treatment of token “position” contains two tendencies that bury the middle. They may sound technical, but the intuition is straightforward.

First, the model listens more closely to nearby tokens. RoPE (Rotary Position Embedding), the positional representation used by most current open models, is designed so that two tokens connect more weakly as the distance between them increases—a decay effect in which the signal fades with distance. Distant tokens therefore receive less attention naturally.

Second, the model sends an unnecessary amount of attention to the very first token. At every moment, the model must distribute a full 100% of its attention—the softmax forces attention weights to sum to 1. When there is nothing in particular to focus on, the remaining attention must be discarded somewhere, and that destination is usually the first token in the sequence. This phenomenon, in which the first token absorbs surplus attention, is called an attention sink. (Xiao's MIT research team first quantified this structure in the StreamingLLM paper. It is less a sign that the first token contains important information than a byproduct of using it as a drain for spare attention.)

Together, these two tendencies concentrate attention at both ends—the nearby recent tokens and the first token—while information buried in the middle receives the weakest treatment. Importantly, this is not a bug in one particular model. Most open models, including LLaMA, Mistral, and Qwen, use RoPE-family mechanisms, and proprietary models such as Claude and GPT are believed to use similar approaches. Lost in the middle is therefore closer to a shared bias in modern transformer architecture.

More recently, this phenomenon has been called context rot. An analysis by the Chroma research team put 18 frontier models—including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3—through the same NIAH (needle in a haystack) task. It quantified accuracy declines of 20–50%, depending on the model, when the input grew from 10k to more than 100k tokens. All 18 models performed worse as context length increased, with the Claude family declining most gradually. Anthropic also explains this as an “attention budget” problem arising from the transformer's n² attention, with the budget being depleted across tokens. Keeping context light therefore reduces costs and protects accuracy at the same time.

Calling Subagents Indiscriminately

Delegating every task just because subagents are useful is another trap. A subagent starts in a separate context from its parent, so it pays the fixed cost of loading the system prompt and tool definitions from scratch. If you delegate a lightweight task such as a short shell command or a simple git lookup, that startup cost can exceed any savings from keeping the main context clean. According to Anthropic's published report on its multi-agent research system, a single agent uses roughly four times as many tokens as ordinary chat, while a multi-agent system uses roughly 15 times as many. Delegation makes sense only when the accuracy gain justifies that four- to fifteen-fold overhead.

The same trap applies to attaching MCP tools. Every enabled tool adds its schema to the system prompt on every turn, and the cost is billed even on turns when the model does not use the tool. “Enable everything just in case” may feel intuitively safer, but actual measurements tell a different story.

I tested both accuracy and cost on the same batch of questions about the reconciler source in facebook/react v19, divided into three setups:

• No tools
• One tool (CodeGraph, Serena, ripgrep, or bare grep)
• All four tools attached at once

Three findings stood out. (Here, recall means how completely the correct answers were found.)

• No tools achieved an average recall of only 0.31, confirming that tools themselves clearly provide value.
• Serena (LSP) alone achieved 1.00 recall at a cost of $0.38, making it the most efficient of all single-tool strategies.
• All four tools attached reduced recall to 0.89 while raising the cost to $0.47. On multi-hop questions in particular, the all-tools score matched CodeGraph alone (0.78 / 0.88) to the second decimal place. The model gravitated toward one of the four tools and inherited that tool's weaknesses.

One principle applies equally to subagents and tool attachment: the extra cost matters only when an accuracy improvement justifies it. Choosing the one tool that fits the task domain is cheaper and more accurate than enabling everything just in case.

So what are the proven ways to save tokens? Do effective methods really exist?

Proven Ways to Save Tokens

Each savings pattern attacks a different cost axis. Some reduce input, some lower the unit price of the same input, and some assign the same work to a cheaper model. Let us examine them one by one.

Prompt Caching

This produces the most immediate effect. As discussed above, cache reads cost 0.1 times the base input rate. After paying a small write premium of 1.25 times on the first call, you can reuse the same static section from the second call onward at one-tenth the price.

From a cost-saving perspective, one point deserves emphasis: under any caching approach, the parts that remain nearly identical across calls—tool definitions, code snippets, and RAG context—are exactly what the cache can capture. The benefit remains intact as long as those static blocks are not mixed with dynamic content such as the current turn's question or a tool result that just arrived. If you call the API directly, grouping static blocks at the front and moving dynamic parts behind them delivers most of the savings. If you use a finished tool such as Claude Code or opencode, the tool handles that ordering for you.

Batch Asynchronous Work with the Batch API

When a call does not need to finish immediately, the Batch API is the simplest way to lower the rate itself. Anthropic, OpenAI, and Google all apply the same 50% discount to both input and output tokens in exchange for returning results within 24 hours. With Anthropic, most batches finish within an hour in practice, but the SLA itself is 24 hours. The calls remain unchanged while the unit price is cut in half, so the implementation cost is minimal.

What makes that 50% especially interesting is that it compounds multiplicatively with other savings. Anthropic's official documentation explicitly states that caching and batch discounts stack. For static input, the calculation is standard rate × 0.5 (batch) × 0.1 (cache read) = 0.05 times, meaning the static portion can fall to 5% of the standard rate. The key is that the discounts multiply rather than add.

I calculated the difference directly. Suppose 100 jobs each contain 10,000 tokens of static context, 500 tokens of dynamic input, and 1,000 output tokens. Using Opus 4.8 pricing, I compared the cost of four strategies.

Caching alone cuts the cost by 57%, batching alone by 50%, and using both by as much as 79%. The discounts multiply over the static-input portion rather than simply adding together. (Output tokens are not cacheable and receive only the 50% batch discount, so workloads with a high share of output see a total reduction smaller than 79%. The larger the static-input share, the stronger the compounding effect.) There are more areas than one might expect where nobody needs to wait in front of a screen: overnight codebase indexing, pre-publication article evaluation, data extraction, and recurring reports.

Not every task can be deferred to asynchronous processing, however. Batch does not fit work where response time itself creates value, such as the main session of a coding agent, where the user reads an answer before choosing the next action, or an interactive chat UI. Simply dividing work into two lanes—“results needed immediately” and “results needed by the next workday”—can cut the bill in half.

Isolate Verbose Work with a Subagent

To use the wording from Claude Code's official documentation, a subagent “operates in its own isolated context window, with intermediate tool calls and results remaining inside the subagent and only the final message returning to the parent.” Delegating an entire verbose task therefore leaves only a clean summary in the parent context. According to Anthropic's article on context engineering, even when a subagent spends tens of thousands of tokens exploring, it typically returns a compressed summary of only 1,000–2,000 tokens to the parent. The benefit of keeping the parent context light is clear.

A verbose task is one that produces a vast number of tokens in the process of finding a one-line answer. Running tests, searching documentation, and analyzing logs are typical examples.

One misconception needs clarification: a subagent does not automatically reduce total cost. Anthropic's own report says agents use about four times as many tokens as ordinary chat and multi-agent systems about 15 times as many. A subagent protects accuracy and the cost of an expensive long-running parent context by removing verbose output from it; it is not magic that always reduces total token expenditure. That is why the earlier principle still applies: delegate only when the cleanup savings in the main context exceed the startup cost. For short tasks, it is cheaper for the parent to handle them directly.

There is also a domain consideration. Anthropic's multi-agent research system report says that in its internal evaluation, an Opus 4 leader with Sonnet 4 subagents improved performance by 90.2% over a single Opus 4 agent. That improvement, however, is not uniform across domains. In the same report, Anthropic explicitly says that domains where agents must share the same context or have many interdependencies are poor fits for multi-agent systems—and identifies coding as exactly such a case. Research can explore independent directions in parallel, but code operates on a dependency graph where changing one function affects another. (If you use a multi-agent setup for coding, it is worth asking whether the workflow truly resembles parallel exploration. A single agent plus one isolated exploration subagent may be a safer default for the coding domain.)

compact and Progressive Disclosure

Claude Code's /compact command compresses the entire conversation so far into a summary and restarts with a fresh context. According to Anthropic's official explanation, compaction is not simple truncation but semantic summarization: it retains the context of ongoing work and recent changes while discarding material unlikely to be referenced again, such as repetitive tool output. Unlike /clear, which removes everything, /compact preserves a summary. When the context window approaches 95% full, auto-compact performs the same operation automatically. Tidying a long session at an appropriate point stops input tokens from accumulating.

Looking further inside, /compact is only the final stage a user can invoke explicitly; before it, a four-stage context-compression pipeline runs automatically. According to an external study of Claude Code's internals, “Dive into Claude Code,” query.ts checks the following five stages in order before every call.

• Budget Reduction truncates portions of individual tool outputs that exceed their size limit.
• Snip cuts off older history along the time axis.
• Microcompact performs fine-grained compression while preserving cache awareness.
• Context Collapse projects extremely long history again at read time to reduce its dimensionality.
• Auto-Compact triggers semantic compression at the 95% mark as a last resort.

The higher stages are lighter and cheaper; the lower ones are heavier but more effective. This five-stage design reflects the recognition that no single compression strategy can resolve every kind of context pressure. Explicitly invoking /compact is roughly equivalent to triggering the final stage of this automatic pipeline early.

The same way of thinking appears in Claude Code's Skills architecture. Where /compact reduces context after it has accumulated, the three-stage loading used by /skills prevents that context from accumulating in the first place.

According to Anthropic's documentation, a skill loads in three stages. Only its name and one-line description—about 100 tokens—enter the context at session start. Its body (SKILL.md, fewer than 5,000 tokens) loads only when the skill is triggered. When bundled scripts or resources are executed through bash, only their output returns; the code itself never enters the context. Installing dozens of skills therefore adds almost nothing to the initial context.

Model Routing: Use a Cheaper Model for the Same Answer

The input price of Opus 4.8 is five times that of Haiku 4.5. Using the largest model for every simple search, exploration, or short summary is a major waste of money. Routing by task difficulty—Haiku → Sonnet → Opus—and calling Opus only for genuinely reasoning-heavy stages is becoming standard practice. LMSYS's RouteLLM research demonstrated a router that preserved 95% of GPT-4 quality while reducing strong-model calls to 14%, though the benchmark measured general reasoning rather than coding specifically.

The tooling landscape has also settled quickly. Commercial gateways include OpenRouter, Martian, and NotDiamond, while the open-source self-hosted ecosystem includes LMSYS's RouteLLM and cost-observability layers such as LiteLLM and Bifrost. Within the Anthropic ecosystem, the Agent SDK's model-selection argument, the Claude Code subagent model field—Explore defaults to Haiku—and the /model slash command all support this kind of routing. One caveat matters: when you call an API directly, it does not inspect input difficulty and choose a model automatically. The Anthropic, OpenAI, and Google APIs all use exactly the model name the user specifies by default. Automatic routing is an optional product-level feature, available in places such as Cursor's Auto, ChatGPT's auto mode, and OpenRouter's openrouter/auto. To save money through routing, you must build it yourself, whether with a gateway or a classifier.

That does not mean adding an external automatic router indiscriminately is the answer. A router such as OpenRouter's Auto Router, which dynamically changes the model on each call, conflicts directly with prompt caching. Anthropic's ephemeral cache hits only when the same model and the same prefix match. If the model key changes on every call, the cache key misses every time. You lose the 90% discount from cache reads, and it is common to save nearly half through routing only to give the money back through cache misses. This is why OpenRouter recognizes the issue and recommends a session_id-based stickiness option.

The pattern that production teams have converged on is surprisingly simple: do not classify dynamically on every call; route statically by task type. The subagent model field discussed earlier follows this pattern. An Explore subagent always uses a Haiku-powered code-exploration lane, while a code-review subagent always uses an Opus-powered review lane. Within each lane, the same model, system prompt, and tool definitions repeat, allowing every model to accumulate and hit its own separate cache. When people say “routing is becoming standard,” they increasingly mean static task-type branching in a “coordinator + executor” structure, not a classifier that runs on every call. That is how routing and caching can coexist without contradiction.

Cursor Composer 2.5

This is a slightly different way to save, but one option is to use an agent such as Cursor. Composer 2.5, Cursor's proprietary model released on May 18, 2026, is based on Moonshot AI's open-source Kimi K2.5 checkpoint and fine-tuned specifically for coding. In Cursor's own benchmarks, the team says it delivers coding performance comparable to Claude Opus 4.7 at roughly one-tenth the price. Published base pricing is $0.50 for input and $2.50 for output, exactly one order of magnitude below Opus 4.8's $5.00 input and $25.00 output rates.

Because Cursor measured the specialized model's benchmark itself, it is safer not to treat the absolute figures as definitive. The broader trend is what matters: a smaller model trained specifically for coding can deliver practically comparable performance to a general-purpose frontier model on the same work while reducing costs by an order of magnitude. This has become one of the clearest cost-saving trends of 2026.

The truly interesting part is that this approach does not merely reduce the unit price: when the model is co-designed with an IDE, it also reduces the input token count itself. Models such as Composer and Cascade are trained to consume codebase context sent by the IDE—current files, adjacent files, and indexed symbols—efficiently. On the same task, that reduces the input-token inflation caused when a general-purpose model repeatedly asks to see relevant files and performs more grep and read operations. The unit price drops by an order of magnitude and the token count falls slightly as well, so real savings can exceed the price difference alone.

Move Context Outside the Context

One of the most interesting trends of 2026 is loading “only what is needed, when it is needed” into context. Anthropic calls this just in time (JIT) context. Instead of placing every resource in the context in advance, the agent carries lightweight references such as file paths or queries and retrieves the actual material with a tool only when necessary. Claude Code follows this pattern by reading files on demand with glob and grep rather than indexing and loading the entire codebase into context at once.

The memory tool and context editing that Anthropic released with Sonnet 4.5 follow the same philosophy. One distinction is important: the “memory” in the memory tool differs from what people commonly imagine. It is neither the conversation history that accumulates within a session—remaining in the context window and disappearing when the session ends—nor a configuration file such as CLAUDE.md that a person writes in advance and the tool loads automatically at startup. The memory tool is a tool through which the model directly writes and reads files.

At a deeper level, the two tools expose surprisingly simple interfaces. The memory tool lets Claude create, read, update, and delete files in a dedicated memory directory hosted in the user's infrastructure. In other words, the model handles notes like an ordinary file system rather than being told to “write this down somewhere in the text.” Keeping storage in user infrastructure is the crucial design choice. It gives Claude persistent memory outside the context window that can carry across sessions, while letting the user control how the data is stored and retained. Session memory is volatile inside the context; this memory remains outside it and survives into the next session. Context editing does the same work in the opposite direction. As the token limit approaches, it automatically clears old tool-call results that are no longer referenced, preserving the conversation flow while allowing the agent to run longer.

In Anthropic's own evaluation, using both tools improved performance by 39% over the baseline, while context editing alone improved it by 29%. In a 100-turn web-search evaluation, token consumption fell by 84%. (These are vendor benchmarks and should be read accordingly, but the direction is clear: designing context to be emptied matters more and more than filling it.) This trend aligns with Claude Code's five-stage /compact pipeline. The difference is that /compact reduces what is inside the context, while the memory tool creates separate storage outside it. The two approaches complement rather than conflict with each other.

Conclusion

Viewed together, token savings ultimately reduce to three axes: send less for the same work, pay less for the same input, and assign the same answer to a cheaper model. Prompt caching, Batch API, subagent isolation, /compact, the memory tool and context editing, model routing, and specialized models differ only in which of these three axes they attack. If the direction of 2026 had to be summarized in one sentence, it would be this: the center of gravity is shifting from the technology of filling context to the technology of emptying and selecting it—context engineering. As context rot demonstrates, a lighter context improves not only cost but accuracy.

There is no single correct answer, of course. Every team has different work patterns, and even one person's cost structure changes between writing prose and writing code. One thing is certain: as the ecosystem moves quickly, assuming that “a savings pattern that worked yesterday will still work today” is becoming increasingly risky. Keeping up with new models, tools, and price lists—and building a habit of validating them through a small POC in your own workflow—may ultimately be the least conspicuous yet most durable way to save. I encourage readers to take out their own token ledger at least once. (If you want to understand why these strategies work at the root level, read the discussion of BPE and KV cache in How Tokens Work as well.)

One question remains. Now that we have techniques for emptying and selecting context, what comes next? Following the available research, the focus is moving one step beyond context to the agent system as a whole: how to operate it (harness design), how to measure whether it operated well (eval), and how to confine it when it goes wrong (containment). Interestingly, when the “measure it yourself with a POC” mindset emphasized throughout this article scales to the system level, it becomes exactly an eval. I plan to explore that next direction in detail in Beyond Context.

참고 자료

Anthropic Prompt Caching

Anthropic Batch Processing

Anthropic Claude API Pricing

Anthropic Claude Code Subagents

Anthropic Claude Code MCP Tool Search

Anthropic Agent Skills

OpenAI Prompt Caching

OpenAI API Pricing

Google Gemini Context Caching

Google Gemini API Pricing

Anthropic, Managing context on the Claude Developer Platform

LLMRouterBench (Findings of ACL 2026)

VILA-Lab, Dive into Claude Code