The backbone is Modularizing React Applications with Established UI Patterns, which Juntao Qiu of Thoughtworks published as a series on martinfowler.com in February 2023. I did not transcribe the original; I retold it in my own words, and every figure in this post comes from that article. On top of it I added the results of cloning the author's example repository and running the type check and tests, along with other material that deals with the same problem.

React is a view library

When you first learn React, the most appealing idea is that UI is a pure function that turns data into DOM. To a degree, that is true. But the moment you send a request to a server or navigate to another page, the component is no longer pure. Add tangled global and local state, and the code gets complicated fast.

The author's answer is to change the point of view. There is no special breed of software called a "React application." React is a library for drawing UI, and where to put calculations or business rules was never its concern to begin with. What sits on top of it is an ordinary JavaScript app. If so, we can reuse the designs we have used since the days of desktop GUIs, in particular the approach of splitting code into three layers: presentation, domain, and data.

A real frontend app has more than views. It has a router, local storage, caches at several levels, network requests, integrations with external services and login, security, logging, and performance tuning. Push all of that into components and hooks, and within one file a line that requests the order status is followed by a line that trims leading whitespace from a string, and then by a line that navigates to another screen. The reader has to keep switching levels of abstraction and jumping back and forth.

The reasons for splitting come down to two. The screen changes more often than the business rules. And once the two are separated, you only have to think about one thing at a time. The second reason is what Martin Fowler names as the biggest benefit of this separation in PresentationDomainDataLayering. You can think about the three topics relatively independently, which narrows what you have to pay attention to.

Five stages of a growing app

Before getting into the refactoring, the author shows in five figures how the structure changes as an app grows. What matters is the trigger for moving to the next stage. It is not a fixed point in time; the pain that appears at one stage calls for the next. The colors mean the same thing in all five figures. Light green is a container component that holds state, dark green is a presentational component that only draws, purple is a hook, blue is a domain object, and orange is infrastructure such as the network.

At first, a single component owns the network request, state, calculation, and render. For a small app or a one-off project this is enough, and because the code looks like HTML it is actually easy to read. Once code that loops over a list to build items gets mixed with code that configures an external component, and it starts taking time to read what is going on, you move to the next stage.

As the screen grows, you split components along the shape of the resulting HTML. Components that only draw break off, but the top component still holds the network request, the code that reshapes the response for the screen, and the code that gathers the data to send to the server. None of the three is UI, yet they live inside a component.

Next, you move state and its changes into a custom hook. Then what is left inside the hook is pure calculation that is neither a side effect nor state.

Move that calculation into an object unrelated to React, and a domain object appears. Data format conversion, null checks, and fallback values go there. As objects multiply, inheritance or polymorphism starts to become necessary.

Finally, when you see objects that belong to no UI and do not care whether data comes from the server or from local storage or a cache, you group them into a separate model layer.

The figures alone are abstract. Let's walk through these five stages once with real code. The example the author chose is the payment screen of an online order.

Starting from a single payment screen

Payment methods come from server configuration and differ by country. If the server returns even one, cash payment is appended at the end and selected by default; if it returns none, nothing is shown.

The initial code has the shape you often see in tutorials. Inside useEffect it fetches, reshapes the response for the screen, appends cash payment, and draws a list of radio buttons. Below is my shortened version.

export const Payment = ({ amount }: { amount: number }) => {
  const [paymentMethods, setPaymentMethods] = useState<LocalPaymentMethod[]>([]);
 
  useEffect(() => {
    fetch(url)
      .then((res) => res.json())
      .then((methods: RemotePaymentMethod[]) => {
        if (methods.length === 0) return setPaymentMethods([]);
        const extended = methods.map((m) => ({ provider: m.name, label: `Pay with ${m.name}` }));
        extended.push({ provider: "cash", label: "Pay in cash" });
        setPaymentMethods(extended);
      });
  }, []);
 
  return (
    <div>
      {paymentMethods.map((method) => (
        <label key={method.provider}>
          <input type="radio" defaultChecked={method.provider === "cash"} />
          {method.label}
        </label>
      ))}
      <button>${amount}</button>
    </div>
  );
};

At this size it is not a problem. But to change this component you have to understand four things together: how the network request starts, how server data is turned into a shape the screen knows, how one payment method is drawn, and how the whole payment section is drawn. As the code grows, the reader has to keep moving between these four.

Hooks and pure components

The first thing to pull out is state and fetch. The signal is simple. When code unrelated to render takes up more than half of the component body, move it into a hook. usePaymentMethods takes care of fetching and conversion, and the component just receives the array from the hook and draws it.

The second is the JSX that draws the list. When you see a block that can be drawn from props alone, split it into a component. PaymentMethods is a pure component that receives only an array of payment methods, so it is easy to test and reuse. Even if an onSelect callback that reports the selection is added later, it does not touch outside state, so it stays pure.

const usePaymentMethods = () => {
  const [paymentMethods, setPaymentMethods] = useState<LocalPaymentMethod[]>([]);
  useEffect(() => { /* 위의 fetch와 변환 */ }, []);
  return { paymentMethods };
};
 
export const Payment = ({ amount }: { amount: number }) => {
  const { paymentMethods } = usePaymentMethods();
  return (
    <div>
      <PaymentMethods paymentMethods={paymentMethods} />
      <button>${amount}</button>
    </div>
  );
};

The author calls both of these moves Extract Function from the refactoring catalog, because in React a component is also a function. The important point is that no new concept was brought in; one function was simply split into two.

This may look finished, but two spots remain. Inside the view there is the rule "select cash by default," and inside the hook there is an anonymous function that reshapes the response for the screen. The author calls this kind of business rule seeping into the view and the hook a logic leak.

Rules gathered in a domain object

The third signal is when decisions about the same data are scattered across the view and the hook. Gather the leaked decisions and conversions into a single PaymentMethod class. As I covered in domain models on this blog, the idea is to put the rules next to the data.

class PaymentMethod {
  constructor(private remote: RemotePaymentMethod) {}
  get provider() { return this.remote.name; }
  get label() { return this.provider === "cash" ? "Pay in cash" : `Pay with ${this.provider}`; }
  get isDefaultMethod() { return this.provider === "cash"; }
}
 
const convertPaymentMethods = (methods: RemotePaymentMethod[]) =>
  methods.length === 0
    ? []
    : [...methods.map((m) => new PaymentMethod(m)), new PaymentMethod({ name: "cash" })];

The radio button in the view now asks the object, with defaultChecked={method.isDefaultMethod}. The cash payment default is also represented as an instance of the same class. Because what the server provided and what the app appended now have the same shape, the view no longer needs to tell them apart.

The author cites easier testing as a benefit of this structure. The original stops at that statement, but seeing how much easier it gets is faster in code. Below is a test I wrote. No React, no renderer, no fetch mock.

test("서버가 준 결제 수단 끝에 현금 결제를 기본값으로 붙인다", () => {
  const methods = convertPaymentMethods([{ name: "apple" }, { name: "google" }]);
 
  expect(methods.map((m) => m.label)).toEqual(["Pay with apple", "Pay with google", "Pay in cash"]);
  expect(methods.filter((m) => m.isDefaultMethod).map((m) => m.provider)).toEqual(["cash"]);
});
 
test("서버가 아무것도 주지 않으면 현금 결제도 보여 주지 않는다", () => {
  expect(convertPaymentMethods([])).toEqual([]);
});

To check the same rules in the initial code, you had to render the component, intercept fetch, and wait until the radio buttons were drawn. The moment the rules came out as an object, the test became a single function call.

Another benefit is that when a new requirement arrives, it already has a place to go. At this point the file structure looks like this.

src
├── components
│   ├── Payment.tsx
│   └── PaymentMethods.tsx
├── hooks
│   └── usePaymentMethods.ts
└── models
    └── PaymentMethod.ts

Whether the structure really holds up is only known when a new requirement arrives. Here the author adds one more feature.

A donation feature and the hook as a state machine

The new requirement is to round up the order amount and let the user donate the difference to charity. For a $19.80 order, it asks about a $0.20 donation, and if the user agrees, the button shows $20.

The author deliberately puts the agreement state and the calculation inside the payment component at first. With the checkbox markup, the message branching, and the total calculation all arriving at once, the component gets heavy again. The cleanup follows the same order as before. State and calculation go to a useRoundUp hook, message assembly goes to a helper function, and the checkbox markup goes to a DonationCheckbox component.

export const useRoundUp = (amount: number) => {
  const [agreeToDonate, setAgreeToDonate] = useState(false);
 
  const { total, tip } = useMemo(() => ({
    total: agreeToDonate ? Math.floor(amount + 1) : amount,
    tip: parseFloat((Math.floor(amount + 1) - amount).toPrecision(10)),
  }), [amount, agreeToDonate]);
 
  const updateAgreeToDonate = () => setAgreeToDonate((v) => !v);
  return { total, tip, agreeToDonate, updateAgreeToDonate };
};

toPrecision(10) is something the original does not explain. When I ran it on Node 24, 20 - 19.8 came out as 0.1999999999999993. It is a device for scrubbing off floating point error.

The author sees this kind of hook as "a state machine behind the view." When an event comes from the UI, it produces new state, and the new state triggers another render. Hooks were originally created so that several components could share logic, but the author's position is that even with a single user, a hook is worth extracting because it lets the component focus on render. I would add one thing. As React's official docs point out in Reusing Logic with Custom Hooks, what a custom hook shares is the logic that handles state, not the state itself. This state machine is created separately for each component that calls the hook.

After the cleanup, the payment component calls two hooks and lays out two child components and a button. A feature was added, yet the payment component's role stayed the same.

Per-country rules and shotgun surgery

The next requirement is to use a different rounding unit per country. Japan rounds to 100 yen, Denmark to 10 kroner. Saying "it seems like an easy fix," the author deliberately shows first the path of taking the country code as a prop and adding branches.

// useRoundUp 안
total: agreeToDonate
  ? countryCode === "JP" ? Math.floor(amount / 100 + 1) * 100 : Math.floor(amount + 1)
  : amount,
 
// 체크박스 문구 helper 안
const currencySign = countryCode === "JP" ? "¥" : "$";
 
// 버튼 JSX 안
<button>{countryCode === "JP" ? "¥" : "$"}{total}</button>

The same countryCode === "JP" check has landed in three places, one each in the hook, the helper, and the button. Adding Denmark means editing all three places again, and once ternaries become unmanageable, a table that looks up currency signs by country code appears. That table, too, ends up referenced separately from many places. This smell, where one change forces you to edit several modules at once, is called shotgun surgery.

What makes this stage different from the previous three is that the signal comes not from the code but from a change request. Looking only at the current code, three branches are not hard to read. The problem only shows when you count how many places you have to edit to add one more country. So where should the scattered branches be gathered?

Per-country differences gathered in a Strategy

The author's answer is to turn the per-country difference itself into an object. First, using Extract Class and Replace Conditional with Polymorphism, he creates one interface and a per-country implementation.

interface PaymentStrategy {
  getRoundUpAmount(amount: number): number;
  getTip(amount: number): number;
}
 
class PaymentStrategyAU implements PaymentStrategy { /* 1달러 단위 */ }
class PaymentStrategyJP implements PaymentStrategy { /* 100엔 단위 */ }

Once they are built, it turns out the only things that differ between implementations are the currency sign and the rounding function, and the rest, like getTip, is identical. So the author uses Inline Class to merge the implementations into a single class that takes only the differing parts through its constructor. The reason you can take the rounding algorithm as a function instead of creating a subclass per country is that functions are values in JavaScript. Below is my shortened version of the result.

class CountryPayment {
  constructor(readonly currencySign: string, private roundUp: (n: number) => number) {}
  getRoundUpAmount(n: number) { return this.roundUp(n); }
  getTip(n: number) { return parseFloat((this.getRoundUpAmount(n) - n).toPrecision(10)); }
}
 
const japan = new CountryPayment("¥", (n) => Math.floor(n / 100 + 1) * 100);
japan.getRoundUpAmount(3312); // 3400
japan.getTip(3312);           // 88

The hook and the component no longer know about countries. They receive this one object and simply use strategy.getRoundUpAmount(amount) and strategy.currencySign. Adding a country means creating one more object, and there are no files to edit.

Going through an interface and then merging back may look like wasted motion. I think this is the most instructive part of the original. Polymorphism was not the goal but a tool for exposing what is common and what differs, and once it became clear that the difference had shrunk to a single function, the class hierarchy was removed. Knowing when to remove a pattern again is a harder judgment than bringing it in.

Moving fetch out of the hook

The last thing to pull out is the fetch and conversion inside usePaymentMethods. A hook is a React concept, so it stays on the view side. Add error handling and retries and the hook quickly bloats, and if you move to another view library, the hook cannot be used. A plain function, on the other hand, can be used anywhere.

const fetchPaymentMethods = async () => {
  const response = await fetch(`${API}/payment-methods?countryCode=AU`);
  return convertPaymentMethods(await response.json());
};
 
export const usePaymentMethods = () => {
  const [paymentMethods, setPaymentMethods] = useState<PaymentMethod[]>([]);
  useEffect(() => {
    fetchPaymentMethods().then(setPaymentMethods);
  }, []);
  return { paymentMethods };
};

The author writes that this function acts like an Anti-Corruption Layer or a Gateway. That means even if the structure of the server response changes, the place to fix stays within this one function. The two names are not the same thing, though. Fowler's Gateway is an object that wraps access to an external system in one place, while Anti-Corruption Layer is a term from Eric Evans's DDD for a boundary that translates an external model with a different semantic system so it does not corrupt your own model. The server response in this example is just { name }, so there is almost no difference in meaning to translate, and I think Gateway is the more accurate name. The same spot becomes an Anti-Corruption Layer when the server is a legacy system owned by another team and the very meaning of a payment method differs.

The final structure looks like this. Render is in components, state in hooks, rules in domain objects, and network requests in functions.

The author lists five benefits of splitting things this way. It becomes easier to find where a defect is. Code becomes easier to reuse and compose. It becomes easier to read. Adding features does not shake the whole. And because domain logic does not know about the view, you can swap out just the view layer. For the last one, the author himself adds the caveat that it is very rare in most projects. It is also worth remembering that none of the five comes with measurements.

That is the storyline of the original. But when you actually run this code, things show up that were not visible in the article.

What running the example reveals

I cloned the author's example repository (16 commits, last commit de186c5) and ran tsc --noEmit and the 7 tests. Both passed, and the helpers whose definitions are missing from the article were in the repository's src/utils.ts. But looking into the structure, I found three things you should know before carrying it over.

A Strategy created anew on every render

The payment component receives a Strategy as a prop and, if none is passed, creates a default with new. And useRoundUp puts that Strategy in the dependencies of useMemo.

export const Payment = ({
  amount,
  strategy = new PaymentStrategy("$", roundUpToNearestInteger),
}: { amount: number; strategy?: PaymentStrategy }) => {
  const { total, tip } = useRoundUp(amount, strategy); // 안에서 [agreeToDonate, amount, strategy]
  // ...
};

JavaScript default parameters are evaluated at every call. And useMemo compares dependencies with Object.is. Put the two together, and every render creates a new instance and the memo recomputes every time. When I rendered the same structure four times in React 18, the calculation ran 4 times with the default and once when the same instance was passed. The repository's App.tsx passes no Strategy with <Payment amount={19.9} />, so the app's execution path is exactly the default case.

The calculation in this example is a few additions, so there is no cost. The problem is the structure. Neither the article nor the repository settles who creates the Strategy and where it is passed from; only the tests create Japan and Denmark instances. If it were me, I would keep the instances as module-level constants and look them up by country code to pass them in.

const strategies = {
  AU: new CountryPayment("$", (n) => Math.floor(n + 1)),
  JP: new CountryPayment("¥", (n) => Math.floor(n / 100 + 1) * 100),
} as const;
 
<Payment amount={amount} strategy={strategies[countryCode]} />

The repository's fetch has no dependency array

The useEffect in the article's code has [], but usePaymentMethods in the repository has no dependency array from the first commit to the last. An effect without an array runs again on every render. When the response arrives, it replaces state with a new array, that triggers a render, and the effect fetches again. Because convertPaymentMethods returns a new array every time, it never stops.

When I replaced fetch with a mock that responds immediately and counted calls over 300ms in Jest, four measurements gave 152, 225, 231, and 233 calls. With [] it is 1. The repository's tests only check whether text appears on screen, so they do not catch this repetition and all pass. Readers who typed along with the article do not have this problem; it only hits when you take the repository as is.

The race that appears the moment the country code becomes a prop

The fetch effect in the article's code has no cleanup either. React's official docs, in Synchronizing with Effects, say that when you fetch inside an effect, an earlier response that arrives late can overwrite the latest one, so you should discard earlier responses with an ignore flag. The article fetches only once on mount, so there is no problem for now. But the moment you take the countryCode=AU in the URL as a prop, a dependency appears and the race opens up. When carrying this structure over, I would write it like this from the start.

useEffect(() => {
  let ignore = false;
  fetchPaymentMethods(countryCode).then((methods) => {
    if (!ignore) setPaymentMethods(methods);
  });
  return () => { ignore = true; };
}, [countryCode]);

One more thing. The rounding in the article is Math.floor(amount + 1), so when the amount is already an integer it rounds up by another 1. A $20 order becomes $21. The article only gives $19.80 and 3312 yen as examples, so there is no telling whether this is intended. If you carry it over into a real service, you first need to decide how it should differ from Math.ceil.

When to use this structure

The author himself says that splitting is not a fixed rule. A small, cohesive component is easier to understand when kept in one file, and what to watch out for is a file growing too large to understand. He notes that a Todo app or a single-form app is fine with everything in the component, and that the example was made complex on purpose to show many patterns. Other sources add caveats in the same direction.

In the same layering article, Fowler says to apply this separation only to relatively small units. Having view, model, and data as top-level folders is fine in a small system, but if any one of them gets too large, split the top level by domain and then split each of those into layers. Carry the components/, hooks/, and models/ folders we saw earlier over as the top-level structure of a large app, and you end up with exactly the shape Fowler says to avoid.

Dan Abramov has a similar history. In 2015 he proposed splitting into presentational and container components, and in 2019 he added an update at the top saying he no longer recommends splitting that way. The reason was that Hooks do the same job without an arbitrary division, and he added that he had seen it enforced far too often, dogmatically and without need. The hook extraction in the original is close to what this update describes. There is no separate component named container; Payment, which calls the hooks, plays that role as well.

On fetch, React's official docs go one step further. You Might Not Need an Effect says that extracting fetch logic into a custom hook makes it easier to move to a better strategy later, yet as the first choice it recommends a framework's built-in fetching or a client cache like TanStack Query. If you extract fetch into a plain function as in the original's last stage, you can pass that function straight to TanStack Query's queryFn, and the library takes over the race and repeated calls from the previous section. I see this as the most practical gain, one the original mentions in just a single sentence. How keys and functions are bound together I covered separately in queryKey.

To sum up, here is my judgment. The original's order (hook, pure component, domain object, Strategy, network client) is worth following as is, because at each stage the pain of that stage becomes the reason for the next. But you do not have to go all the way. Look at which stage the pain you feel right now belongs to, and stop there. If there is a lot of code unrelated to render, go as far as hooks; if the same decision is scattered around, as far as domain objects; if one change has you editing several files, as far as Strategy. And if you use Strategy, decide first where the instances are created, and if you keep fetch in an effect, write the cleanup and dependencies first. These two things the original left out were the first places to break when I actually carried it over.

There is no single right answer, but I hope you, the reader, will take a moment to check which stage's pain your own code is going through right now.

The first use that caught my eye was browser automation. Browserbase opened a PR that wires Jev into Stagehand's act(). The page's accessibility tree goes in as state, and the question "which element should be clicked next" is asked as a set of options. Across 40 tasks the median act() latency dropped from 1.97 seconds to 0.46, and of 147 actions only 4 fell back to an LLM. The spot where a Playwright script breaks the moment one selector changes was filled with a call that returns a single probability. As I write this, the PR has not been merged.

The line that PR drew is the question of this post. It accepts Jev's answer at a probability of 0.7 and hands anything below that to an LLM. Can you trust the probabilities that model returns and draw a line in your code?

Someone else's benchmark cannot settle that, so I want to verify it on my own data. This blog has a gate that blocks transliterated loanwords. The rule says to write 피커 as picker, but the regular expressions only catch the half that is certain, and words that sit inside a compound or are already settled in Korean, like 멀티스레드 or 콜 스택, I judge by hand every time. There are only two answers, revert or keep, anyone who knows decides in a second, and the pass or fail has to be decided inside a build script with no human checking. It has the same shape as Stagehand's selector decision.

So the order is this. What Jev returns, what kind of training produces that probability, what numbers come out when I hand it my gate, and where those numbers can be used.

A model that produces no text

The release date was September 15, 2026. The company does not call it an LLM; it calls it a System One Model, a new category. In this post I call them decision models. Founder Diogo Almeida was the fourth author of the InstructGPT paper at OpenAI. He was on the team that built the method that became ChatGPT's direct ancestor, and he now stands on the side that points out that method's limits.

Jev does not generate sentences. You fix the shape of the answer in advance, and it picks from that shape and returns probabilities along with it. There are only three question types.

There is only one endpoint, too. Send a POST /v1/systemone with state, the thing to evaluate, model, and questions, a map of questions, and the answer comes back. You can pack several questions about the same state into one request, and adding questions barely increases the response time.

Why it is fast is explained by the asymmetry between prefill and decode that I laid out in how tokens work. An LLM is slow because it squeezes its output out one token at a time, sequentially, and Jev has no decode stage. It reads the input once in parallel and reads the probabilities straight off. So there is no output token charge at all, and input alone costs $0.042 per million tokens.

A developer named Archer Hume called the API roughly ten thousand times and reconstructed its behavior from the outside in an analysis. The median was 57.5ms for 360 input tokens, 218ms for 29,835 tokens, and 610ms when he packed in 1,500 questions. The most direct piece of evidence there is his observation that a response with 200 choices came back as fast as one with 2. It means writing the answer takes no time.

Decisions nobody watches

The speed and the price stand out, but the question Almeida raised in the launch post pointed elsewhere. Models have been superhuman at conversation for years now, so where did all the automation go?

His answer is that the two kinds of work are different. Chatbots, copilots, and coding agents have the goal of satisfying a person watching from beside you. A judgment that runs quietly on a server with nobody watching does not. If you call the former assistant and the latter automation, every model released so far was built for the former.

I drew a similar distinction while writing up harness design. An agent system contains countless judgments nobody looks at. Should this request be handed to a human? Is this command safe to run? Right now we put all of them to an expensive LLM. The other half that my regular expression gate cannot cover is that kind of judgment too.

What RLHF left behind

But why did this call for a new training method? Could you not simply make an existing model answer yes or no?

The clue lies in calibration. If accuracy is what percentage you get right, calibration is whether you know what percentage you will get right, and the metric for that gap is ECE (expected calibration error). It averages the difference between the stated probability and the actual hit rate in each probability bin, weighted by sample share, so 0 is perfect. Yet there is a case where this scale drifted after RLHF (reinforcement learning from human feedback), which optimizes human preference. In Figure 8 of the GPT-4 technical report, the pre-trained model's ECE was 0.007, while the post-trained model's was 0.074, more than ten times worse. The report does not state the cause, and I have not found a primary source that establishes one either. I wrote up the definitions and how much of the cause is known separately in LLM Calibration and Overconfidence.

ECE alone is not enough, though. A constant predictor that stamps 0.6 on every input also has an ECE of 0 as long as its real hit rate is 60%. If the probabilities are merely honest and do not separate from case to case, there is nowhere to draw a line. So alongside calibration you have to watch whether the probabilities actually separate, and the "share that can be automated inside an error budget" I use later on is the metric that folds those two into one number.

From a software point of view, this is the part that matters. If the probabilities are honest and they separate, you can draw a line. You get a structure where anything above 0.95 is handled automatically and anything below goes to a person.

TypeSafe says it opened a third road aimed at that spot: RLHF, which optimizes human preference; RLVR (reinforcement learning with verifiable rewards), which optimizes answers a machine can grade; and RLCD, which it says optimizes calibrated decisions.

What has actually been published as RLCD, though, is three lines of output contract. As of September 23, 2026, when I am writing this, there is no paper or technical report anywhere in TypeSafe's documentation or launch post. There is a separate ICLR 2024 paper using the same acronym, but that one is reinforcement learning from contrastive distillation, a different method. Take care not to mix them up when searching.

What confidence really is

So, are the numbers Jev returns honest? Before answering, there is something to look at first. It does not return one number.

Choice and Score responses carry both probabilities and confidence. The first is a probability distribution across all the options, the second a single number between 0 and 1. When you set a threshold in code, the one your hand reaches for first is confidence.

This value is not something the model produced. _utils/confidence_metrics.py in system-one-adapter-python, published by the TypeSafe organization, is 32 lines in total, and the Choice part looks like this.

def choice_confidence(probs: list[float]) -> float:
    """Scale peak choice probability from uniform to certainty."""
    if len(probs) == 1:
        return 1.0
 
    normalized_probs = _normalize(probs)
    uniform_probability = 1.0 / len(normalized_probs)
    return (max(normalized_probs) - uniform_probability) / (1.0 - uniform_probability)

There is no model call and no learned parameter. One probabilities vector goes in and one real number comes out. Written as a formula it is c = (p_max − 1/K) / (1 − 1/K), and with three options and a peak probability of 0.8 it comes to 0.7.

This repository is not the production Jev server but a substitute implementation that imitates the same API with an LLM. So this alone cannot settle it. There are, however, two further pieces of evidence pointing at the same formula. The confidence page of TypeSafe's official documentation describes this value as "a statistic computed from the probability distribution" and, in its demo code, gives (3 × largest probability − 1) / 2 as an approximation for three options. With K at 3, that is the formula above. And a claims audit ledger recorded that on the real API, when the winner of a response given only two options that were both wrong was 0.52, confidence came back as 0.04. With a K of 2, (0.52 − 0.5) / (1 − 0.5) = 0.04. It matches the formula.

In the same repository, Score uses an entirely different formula. It divides the mean absolute distance from the modal level by the same quantity for a uniform distribution and then subtracts the result from 1. The documentation does not publish the formula for Score. So there are two formulas under the single name confidence, and with a different type the same number means something different. Noul having no confidence is for the same reason. There is no distribution to fold.

The documentation does not hide this either. It goes as far as telling you that if you want a different calculation it will hand you the whole probabilities and you can do as you like. It is written down, and the problem is on the side that uses it without reading.

The comparison table in the launch post, however, puts the two right next to each other. It says "Calibrated: higher confidence means higher accuracy." The reader takes that to mean confidence is the calibrated value, whereas the side the calibration claim actually attaches to is probabilities.

There is a measurement of how far this distinction opens up in numbers. An independent calibration measurement published by a developer called scienthoon separately measured the ECE you get when you read confidence as "the probability of being right." It is 0.035 on OpenBookQA, 0.078 on HellaSwag, and 0.18 on a synthetic set. The raw probabilities on the same sets were 0.024, 0.029, and 0.107 respectively. On all three sets the confidence side is worse.

Why it gets worse is in the formula. With K fixed, c is a monotonically increasing transform of p_max, so the ordering is preserved. Discriminative power stays the same, and you only rewrite the threshold. What breaks is not the information but the scale. If K differs from question to question, one more problem appears. The stretching factor differs too. The same p_max of 0.8 becomes 0.6 with two options and 0.75 with five. The moment you use a single threshold on a workload with mixed option counts, that becomes a problem.

Calibration that shifts with the distribution

So can the probabilities side be trusted?

There is a published record of someone actually calling Jev and measuring. The repository for Kev, an open source reimplementation built by Jared Palmer, has a runs/jev-* directory, and usage.json records with "model": "typesafe-ai/jev" that these were real calls made through the Vercel AI Gateway. Adding up the six jev-* directories' usage.json files, across September 18 and 20, 2026, it made 5,364 calls on 2,418,260 input tokens. Of the seven sets below, scienthoon was imported by converting that developer's result files, so it is not in this total.

That the pricing really charges for input only is confirmed elsewhere. The author of the phishing benchmark looked at the TypeSafe dashboard and wrote it down. Of the 4.56 million tokens across two runs, 3.66 million were input and 900,000 were output, and the bill was $0.15, which matched the figure you get by counting input alone.

Pulling the metrics for seven evaluation sets out of report.json in the same directory gives this. Below, peak probability is not the confidence field but the maximum of probabilities. Overconfidence is 평균 확신도 − 정확도, mean peak probability minus accuracy, over 5,743 items in total. The values come from the clean block, which excludes from accuracy scoring the "unknowable" items whose grounds for judgment were erased.

Two things read out of it.

First, the ECE moves by nearly ten times. From 0.011 to 0.105. Same scorer, same model. "Calibrated" is not a property of the model but a relation between the model and the distribution. That calibration collapses when the distribution shifts is itself a known result. What is new here is that the same shape appears even in a model that puts calibration forward as its training objective.

The 0.105 of the set with the largest ECE is bigger even than the 0.074 of the post-RLHF GPT-4 seen earlier. But you must not read these two as a ranking. The tasks differ, and ECE is an estimator whose value moves with how many bins you split into and which bins the samples pile up in, so placing numbers from different scorers side by side does not make a valid comparison.

Second, the measured ECE is almost entirely explained by overconfidence alone. The difference between the two columns runs from 0.002 to 0.011 for all seven. ECE is a weighted average of per-bin gaps, so it cannot be smaller than the overall overconfidence, and that difference being nearly 0 means the sign of the error in each bin leans one way. It is not noise that crosses over and cancels bin by bin.

Reading down the columns shows why. Leave out semif and six of the mean peak probabilities sit packed between 0.851 and 0.906. Over the same stretch accuracy moves twice as wide, from 0.753 to 0.857. Peak probability barely moves when the distribution changes, and only accuracy moves. The difference left over is the ECE.

Another benchmark shows how this property surfaces in practice. In a measurement on 2,000 phishing emails with Claude Haiku 4.5 as the control, Jev's accuracy was 62.6% and Haiku's was 81.3%. More striking, though, is the ECE. The repository reports an ECE of 0.154 for Jev and 0.097 for Haiku, and when the claims audit ledger re-bins Jev's P(phishing) values into 10 bins from 0 to 1, it comes to 0.170. Read either way, a model that puts calibration forward as its training objective lost on calibration to an LLM built with RLHF. The same author got 91.6% from the link host list rule alone.

Looking at the same metric across several sets makes the spread plainer. The share that can be handled automatically at a 5% error budget is 0.486 on the scienthoon set, 0.695 on transfer-v9, and 1.000 on semif. Kev's scorer notes that this value is the maximum with the threshold chosen inside the sample, not an error guarantee after deployment. You must not cite any one of them as if it were the model's spec.

The transliteration gate

Reading someone else's benchmark and measuring on my own data are different things. So I put the gate from the opening through it for real.

I do not have a Jev API key yet. Instead I ran Kev-9B locally, the reimplementation mentioned above. It puts a rank-16 LoRA and a pointer head on Qwen3.5-9B-Base, and it is Apache-2.0. The numbers below are not Jev's numbers. Kev's training data is English decision tasks, and in the same author's measurements Kev-9B trails Jev even on English tasks. Its automation share at a 5% error budget is 0.45 to 0.57 against Jev's 0.70, and on MMLU-Pro it is 0.52 against 0.84. The TypeSafe documentation also states, about Jev, that English is the primary training language and CJK is not equivalent.

The evaluation sets were drawn from 18 Korean posts in this repository. The ground truth for a label is which spelling this repository consistently uses for that word. That means it is this blog's convention rather than a consensus of the Korean technical writing community, and on words where the two diverge the model can be right by the community standard and wrong by this one.

• The gate set, 102 sentences: These are words the gate already knows. The positives are 23 counterfactual sentences in which words the body writes in English, such as calendar, picker, and adapter, were turned back into Korean transliterations; the negatives are 79 sentences that actually use the exceptions write-post.md spells out, such as 리렌더링 and 콜 스택. On this set the current regular expression gate is right 100% of the time by definition.
• The held-out set, 60 sentences: These are words the gate has never seen. This is what machine learning calls a held-out set. The positives are 30 sentences that turn loader, mutation, and prefill, which the body writes only in English, back into transliterations; the negatives are 30 sentences with 리듀서, 스냅샷, and 런타임, which the body writes only in Korean. Here the regular expressions catch none of the positives.

Let me disclose up front the asymmetry that the positives are counterfactual sentences I built while the negatives are real sentences. And the effective sample is the number of words, not the number of sentences. Labels are decided per word, so sentences containing the same word are not independent observations. The gate set has 24 distinct words and the held-out set 13.

The table below tallies only the one noul question asking "should this word be turned back into English." I put four questions in the same request, but the other three were a negated form, a Choice variant, and a compound-word judgment whose labels I could not set properly, so they are not mixed in here. The 5% budget automation share is the largest share you get by cutting down from the highest peak probability such that the error above the cut does not exceed 5%.

The majority class baseline is the score you get by never reading the sentence and stamping the more common label. The model sits below it.

The last row is this experiment's answer. On the held-out set, no matter how high I set the threshold, there was not a single decision that could be handled automatically within a 5% error.

The prediction distribution shows what happened. The model answered "it should be turned back" on all 162 sentences. All 37 distinct words. Every positive right and every negative wrong. So the 0.500 on the held-out set is not skill; it is a number that comes from the set being balanced.

And while doing that, its mean peak probability was 0.876. On the gate set, 49% came in above a peak probability of 0.9, and the actual hit rate in that band is 0.240.

The structural identity did not hold either. Adding the probabilities for "should it be turned back" and "should it be left as is" gave a mean of 1.655 on the gate set, and all 102 sentences deviated from 1 by more than 0.1.

That is by design. The two questions are separate evaluations that cannot read each other, and TypeSafe itself put an example on its jaggedness page where refund at 0.72 and not_refund at 0.47 add up to 1.19.

Here is where it bites in practice. You must not move a threshold tuned for Noul over to Choice. For the same judgment, 14% to 17% of the sentences came out with different conclusions.

If the model says yes to whatever you ask, this result is meaningless. That is what I checked first.

It reads Korean, it says "no," and it separates 0.02 from 0.98. This is not a global acquiescence bias. That is as far as this rules anything out. The possibility that the wording of the gate question is itself bad remains.

Putting the rules into state

Would it change, then, if I handed it the knowledge the judgment needs? This is the domain adaptation method the TypeSafe documentation recommends: leave the weights alone and ship the reference material inside state.

I took the rules paragraph already written in write-post.md, put it into state alongside the sentence, and ran it again. The input grew from 100 tokens per sentence to 450. The gate set is leakage. That reference material lists the gate words and the exception words by name. So that side is a control for "does it read the reference material at all," and the real test is the held-out set.

The latency is local inference run on an M2 Max, so it is not on the same axis as the Jev API latencies seen earlier. What to look at here is not the absolute value but the 2.4x when the reference material is added.

On the balanced held-out set, sentence-level accuracy went from 0.500 to 0.500, not moving at all. At the word level it actually went down, from 7/13 to 5/13.

The 0.225 to 0.676 on the gate set is not an improvement in skill. Before, it said "turn it back" on all 102 sentences; once given the reference material it answered that way on only 12. Negatives climbed from 0/79 to 68/79, but positives collapsed from 23/23 to 1/23. That set has more negatives, 79 against 23, so a constant that says "mostly leave it" automatically scores higher. It did not learn the distinction; it changed which way it leans.

That does not mean the model is not reading the sentence. For the same sentences containing the same word, the predictions scattered sentence by sentence once the reference material was given. The standard deviation for 로더 grew from 0.096 to 0.217, and 뮤테이션 from 0.042 to 0.236. It reads, but it cannot distinguish.

And this result also shakes the previous section's reading. If a single piece of state text flips the output wholesale, then the "turn everything back" of the first run may be a shortcoming of the question's wording rather than a limit of the model. This post has not ruled that possibility out.

One thing improved noticeably. The sum of the probabilities for "turn it back" and "leave it" closed from 1.508 to 0.963 on the held-out set, and the sentences deviating far from 1 fell from 59 to 18. Yet the accuracy did not move. Two questions' probabilities agreeing with each other and those probabilities being right are different things.

Where decision models belong

Reading this far, it is easy to lean toward the conclusion that this whole model category is useless, but lining up the published benchmarks says otherwise.

Only the rerank row is a search ranking quality metric, so it sits on a different axis from the accuracies in the other rows.

Inside the distribution, a regular expression or a fitted classifier wins or ties. It is the same place where the regular expressions beat the model 1.000 to 0.225 on my gate set. If you can collect labels, training a small model is cheaper, faster, and more accurate. That is something we have been doing for a long time.

The gap opens when the distribution is new and there are no labels. On out-of-distribution spam it is 98.6% against 73.0%. The fitted classifier collapses in front of a shape it has never seen, while this side holds. The place for decision models is judgments you cannot collect data for, label, and train on. It is where every situation is new, so labels cannot be gathered, yet the judgment has to be made within seconds.

This frame also explains why my transliteration judgment failed. What that judgment needs is not general reasoning but the specific domain knowledge of which words have settled in Korean technical writing, and for a 9B reimplementation trained on English decision tasks, that is not out of distribution so much as knowledge it simply does not have. That is why giving it the rules in prose did not work either.

In a TechCrunch article, Armin Ronacher, the CTO of Earendil, which builds the Pi harness, put it this way.

At the end of the day, it delegates the hallucination problem a little bit to the user.

It has not removed hallucination; it has handed the judgment to the developer. The TypeSafe documentation also writes that calibration is a property that holds over a batch of predictions, not a guarantee that an individual answer is right, and under the 0% hallucination bar in the launch post, the Nuance entry reads "Our number is not empirical." The form is guaranteed and the content is not.

The 194 projects on jevable

Since it failed on my data, I looked at where other people are using it. jevable.com is an independent curation by a developer named Nikunj, who reviews Jev projects posted on X and collects them. As of September 23, 2026 it lists 194, and the registration dates cluster between September 16 and 20. 152 of them arrived on September 18 alone, three days after the release.

Regrouping by "what is being asked" rather than by category gives three shapes. Every figure below is what the posters wrote in their own posts.

First, classification and routing. Sorting 1,500 emails, classifying tax documents, classifying 1,891 competitor ads in 19 seconds for $0.12, 14 typed checks on a single PR diff, spotting ads among Android notifications, classifying 26 construction drawings in 2.9 seconds. This is the same shape as the loanword gate at the start of this post. As the table in the previous section said, these are places where labels pile up every day, so given time a small classifier trained on your own labels wins or ties. Jev's advantage is on day one, when there are no labels.

Second, action selection loops. A browser agent that hooked into Browser Use and finished a flight search in 7 seconds for $0.0039, computer use that drives a Mac by voice, a game that forks the VM four ways every time Mario dies and picks the branch that survives, an expression engine that decides ten things per message for a 3D character, including its mouth, eyebrows, and gaze. The action space changes at every step, so you cannot collect labels, and the judgment has to come in under a second. This is the same place where the gap opened up on out-of-distribution spam. I think it is also why Games, at 39, is the largest category. A game is an action selection loop where being wrong can be undone.

Third, UI that shows the probability to the user. Ask Jev, which returns only a verdict instead of an answer; JevForm, a branching form that picks the next question by probability; Upweight, which re-sorts the Hacker News front page with six sliders such as technical depth and drama. Since no threshold is hard-coded and a person reads the probability, these have the lowest calibration requirement of anything this post has taken issue with.

One thing stands out. Of the 194 blurbs, 30 mention cost and 53 mention speed, but only 7 mention accuracy or a baseline. This is a lower bound because the blurbs only carry the start of each post, but the direction is clear. Fast and cheap you can know on day one; whether it is right you can only know by measuring, and the measuring side is rare. One of those 7 is jevcal. It says everyone picks thresholds by feel, and given your data and a target accuracy it returns a threshold and an automation rate. It is the procedure the next section recommends, turned into a tool.

How to draw the line

This post's conclusion is this. Do not read the probabilities a model returns as a spec; measure on your own data and draw the line yourself.

The measuring procedure goes like this. Gather 100 or so labeled items, build a table of the actual hit rate per probability bin, and lower the threshold from the top while watching the error rate above it. Once you have set an error budget, the maximum coverage that keeps that budget is the share you can automate. In my experiment, the fact that the actual hit rate above a peak probability of 0.9 was 0.240 was exposed by 102 sentences.

A scale that is off can usually be fixed. If the overconfidence is spread evenly across all the bins, a single temperature straightens out most of it. Kev did exactly that and brought its ECE down from 0.106 to 0.042. The catch is that fitting that temperature requires labels from that distribution, and labels are precisely what you do not have in production. On top of that, temperature cannot change the ordering of the probabilities, so the automatable share does not rise by the same amount. Which is why ordering matters more than scale, and error budget coverage is a more practical metric than ECE.

Had I hardcoded the 0.9 the documentation uses in its example, this gate would have quietly mis-corrected 38 items. No error is raised, so it is a failure that is hard to notice. The TypeSafe documentation does say, right below that example, to test it on your own data. What ends up hardcoded is usually the number above, not the sentence below.

Even without a key, the measuring part you can do today. Kev is published under Apache-2.0 and runs on a laptop.

I would like you to count how many calls in the service you run ask a big model nothing but yes or no. Before you move those calls to a decision model, I would suggest measuring where the line has to go after the move. I kept that order, and that is why I could decide to leave the gate alone. When I get a Jev key I plan to run the same sets again, and if the conclusion changes then, I will write that down too.

I like declarative code. When it comes to opening an overlay, closing it, and doing asynchronous work inside it, I think the overlay itself should manage that sequence. When the calling side holds isOpen and flips it on and off, the code that handles the state ends up in a different place from the component that renders it.

The modals on the screens I owned at work were exactly the opposite. Whether a modal was open lived in a global store, the call site opened it, and the call site closed it. The call site was also writing the code that closed the modal and the callbacks that received its result. So I spent a long time working out how to move this in a declarative direction, and in the end I rewrote it.

While doing that work I looked at overlay-kit. It was close to the interface I was aiming for. This post is a record of overlay-kit's declarative interface and of how I designed our internal modal system.

The modal I was holding in state

In my post on state management, I used a modal's open flag as the simplest example of local state. I wrote that it is state used inside a single component, state that the outside has no need and no right to know about. I still think that is true. But once you have dozens of modal types, they stack on top of one another, and they have to return a value to the call site, that state does not stay inside the component.

The code I inherited kept the list of open modals in a global store. Since it is company code, the example below keeps the structure and changes the identifiers.

const [openPopup, closePopup] = useModalStore(
  useShallow((state) => [state.openPopup, state.closePopup]),
);
 
const handleAddNote = useCallback(() => {
  openPopup(PopupType.ADD_NOTE, {
    itemId,
    onClose: () => closePopup(PopupType.ADD_NOTE),
    onSubmit: (note) => {
      closePopup(PopupType.ADD_NOTE);
      saveNote(note);
    },
  });
}, [closePopup, openPopup, itemId]);

Three things about this code bothered me.

First, the opening function has to come out of a hook. openPopup and closePopup are values you select from the store, so you can only obtain them inside a component. That means the code that opens a modal cannot leave the component. To open a modal somewhere you cannot use hooks, such as an API response interceptor or a router guard, you have to pass that function down as an argument.

Second, the result comes back through callbacks. Whether the note was saved or the user simply closed the dialog arrives through two separate branches, onSubmit and onClose. Anything you want to do after opening the modal goes inside a callback. If you have to open two modals in a row, you get a callback inside a callback.

Third, closing is the call site's responsibility. The call site is the one that calls closePopup(PopupType.ADD_NOTE). The modal does not close itself; the side that opened it remembers which kind of modal it was and closes it. That is why the same constant appears three times in one function.

What the three have in common is that opening a single modal requires the call site to write three kinds of code: the code that opens it, the callback that receives the result, and the code that closes it.

If deciding what to show is the call site's job, then here the call site was carrying everything after that as well. In particular, closePopup(PopupType.ADD_NOTE) names its target with a string constant. Writing another modal's constant still passes type checking because it belongs to the same enum, and the wrong modal closes.

An interface that waits for a result

So which of those three can be removed? The answer I arrived at was to make opening a modal a function call, and to have that function return a Promise carrying the result.

const result = await openModalAsync('ADD_NOTE', { itemId });
if (result.reason !== 'confirmed') return;
 
await saveNote(result.value);

Three things changed. The opening function became a module function rather than a hook, so it can be called outside a component. The result became a return value rather than a callback argument, so the follow-up work stayed inside the flow of the calling function. And closing became the modal component's own job.

Because the return type is Promise<ModalResult<V>>, the end of a modal collapsed into a single value.

A Promise settles only once. Resolving an already settled Promise does nothing, so every reason a modal can end has to fit into that one value. That is why I split reason into five cases: the user confirmed, the user explicitly cancelled, it closed without a value, it was replaced by a new modal of the same kind, or it was force-closed by a full teardown. With a single boolean, "cancelled" and "the whole screen was torn down" become the same value, and flows that need cleanup have to tell those two apart.

In the other direction, if you never settle it, the call site waits forever. Nothing happens when a callback is not called, but an await simply never returns. So every time a new path for removing a modal from the screen appeared, I had to check whether that path also settled the Promise. Replacement and force-close each got a reason of their own for that reason.

overlay-kit's call site

The call site in toss's overlay-kit looks like this.

const confirmed = await overlay.openAsync<boolean>(({ isOpen, close }) => (
  <ConfirmDialog
    open={isOpen}
    onConfirm={() => close(true)}
    onCancel={() => close(false)}
  />
));

Three points struck me as a good interface.

What opens is visible at the call site. My version passes a string key, 'ADD_NOTE'. Which component that key refers to is known to a registry file: a file containing one object that pairs each string key with the import() function that loads that modal.

export const modalImporters = {
  ADD_NOTE: () => import('.../add-note-modal'),
  CONFIRM: () => import('.../confirm-modal'),
  // 모달 종류만큼 이어진다
} as const;

overlay-kit writes the JSX right there. Anyone reading the code knows what will appear without navigating to that file.

The result type is decided at the call site. The type argument of openAsync<boolean> is both the argument type of close and the result type of the await. You do not have to declare a result type per modal and register it in a table.

Closing and removing are separate. The controller receives four props: overlayId, isOpen, close, and unmount. close sets isOpen to false and leaves the component in place. unmount is what actually removes it. The official guide to opening and closing overlays explains it this way.

The reason for this difference is that when you use close, the overlay is kept in memory so that its closing animation can play.

My version has no such distinction. Closing removes it from the list immediately. That means I was leaving closing animations to the component library, and that I was only dealing with screens where that was possible.

From overlay.open to render

Once you like an interface, the next question is what is inside it. For a single module function to render a component inside the React tree, something has to bridge the two.

Events and the reducer

overlay.open does not change state. It emits one event.

// packages/src/event.ts
const open = (controller: OverlayControllerComponent, options?: OpenOverlayOptions) => {
  const overlayId = options?.overlayId ?? randomId();
  const componentKey = randomId();
  const dispatchOpenEvent = createEvent('open');
 
  dispatchOpenEvent({ controller, overlayId, componentKey });
  return overlayId;
};

The receiving side is OverlayProvider. The provider holds a useReducer, subscribes to the event, and feeds ADD into the reducer. The thing to watch here is that isOpen goes in as false.

// packages/src/context/provider/index.tsx
const [overlayState, overlayDispatch] = useReducer(overlayReducer, {
  current: null,
  overlayOrderList: [],
  overlayData: {},
});
 
const open: OverlayEvent['open'] = useCallback(({ controller, overlayId, componentKey }) => {
  overlayDispatch({
    type: 'ADD',
    overlay: { id: overlayId, componentKey, isOpen: false, isMounted: false, controller },
  });
}, []);

Then, on the frame after the controller mounts, it dispatches OPEN to turn isOpen into true.

// packages/src/context/provider/content-overlay-controller.tsx
useEffect(() => {
  requestAnimationFrame(() => {
    overlayDispatch({ type: 'OPEN', overlayId });
  });
}, [overlayDispatch, overlayId]);

One part here was not what I expected.

The state does not live outside React. Because the API opens overlays through a module function, I assumed the store would sit at module level too, but it is actually a useReducer inside the provider. The only thing outside React is the event bus, and that holds no state. Overlays render as siblings next to the provider's children. It does not use createPortal, and it does not import react-dom.

One more thing stands out. From the call site it is one line, overlay.open(), but inside it is a fixed procedure: emit an event, feed ADD into the reducer, wait a frame, and feed OPEN in again. A declarative interface does not remove imperative code; it moves it to the other side of a boundary.

The opening side and the closing side

The requestAnimationFrame above and the split between close and unmount from the previous section look unrelated at first. In fact they are two devices blocking the two ends of the same constraint.

That constraint is that a CSS transition only runs when both the start state and the end state are actually rendered. MDN's CSS transitions guide warns about elements just added to the DOM.

This is treated as if the initial state had never occurred and the element was always in its final state.

The failure shows up in two shapes. On opening, there is no start state, so nothing runs. On closing, the element disappears before the end state is painted, so nothing runs.

The last row explains why the closing side is left to the user. On the opening side, the library knows the constant: one frame is enough. On the closing side, the library cannot know whether the animation is 200 or 400 milliseconds.

Closing handed over as unmount

The reducer is what makes the closing side work. CLOSE only flips, inside overlayData, the isOpen value, and leaves overlayOrderList alone. The provider renders by walking that list, so the component stays alive as long as it remains in the list. It only unmounts once REMOVE takes it out.

That said, rAF is not always necessary. react-transition-group does not run the enter transition on first mount by default. It skips it even when in is already true, and you have to turn on appear. overlay-kit's rAF changes isOpen from false to true after mount, which covers that case.

Twenty lines wrapped in a Promise

openAsync is not a separate implementation. It is a thin layer around open.

// packages/src/event.ts
const openAsync = async <T>(controller: OverlayAsyncControllerComponent<T>, options?: OpenOverlayOptions) => {
  return new Promise<T>((_resolve, _reject) => {
    open((overlayProps, ...deprecatedLegacyContext) => {
      const close = (param: T) => {
        _resolve(param);
        overlayProps.close();
      };
      const reject = (reason?: unknown) => {
        _reject(reason);
        overlayProps.close();
      };
      const props: OverlayAsyncControllerProps<T> = { ...overlayProps, close, reject };
      return controller(props, ...deprecatedLegacyContext);
    }, options);
  });
};

All it does is swap the close that goes to the controller. The original close takes no arguments; this one takes an argument, resolves the Promise with it, and then calls the original close. This is also why the result type is decided at the call site. T is both the argument type of close and the type of the Promise.

There is one hole in this structure, though. Resolution only happens when close is called. So if you remove an overlay with unmount alone, the Promise stays pending forever. It is filed as issue #169, and I think it needs fixing, but the issue is still open.

The fact that resolution happens at close carries another implication. It is the moment the closing starts, not the moment the closing animation ends. That is fine for a flow that takes a confirmation and then calls an API, but in a flow that takes a confirmation and then opens the next modal, the second modal rises while the first is still disappearing.

Where the design came full circle

I followed the commit history to explore how the design got here, and found that the structure was not like this from the start. 1.0 used the same event and reducer combination as today. 1.2.0 moved the state into a module-level external store read through useSyncExternalStore, and then 1.8.0 moved it back to a reducer.

The reason for reverting is written in issue #148.

useSyncExternalStore resolves tearing issues, but it doesn't work well with suspense.

When fetching data inside an overlay caused it to suspend, React warned that a component had suspended while responding to synchronous input. The original goal of moving to an external store was to let overlay state be read without a hook, and the PR at the time says outright that the goal was not achieved the intended way. In the end the goal was solved with a hook, and only the storage mechanism returned to where it started.

What I took from this record is not a conclusion but a criterion. Putting state outside React frees up where you can call things, at the cost of opening room to fall out of step with React's scheduling. overlay-kit put only the call site outside and kept the state inside.

Reopening with the same id

Something caught my eye while reading the reducer. At the very top of ADD there is this branch.

// packages/src/context/reducer.ts
case 'ADD': {
  if (state.overlayData[action.overlay.id] != null && state.overlayData[action.overlay.id].isOpen === false) {
    const overlay = state.overlayData[action.overlay.id];
    if (overlay == null || overlay.isOpen) {
      return state;
    }
    return {
      ...state,
      current: action.overlay.id,
      overlayData: { ...state.overlayData, [action.overlay.id]: { ...overlay, isOpen: true } },
    };
  }
  // ...

If you open an overlay with an explicit overlayId, close it, and then open it again with the same id, you land in this branch. But the return value reuses the existing overlay object.

Nothing ever uses the incoming action.overlay and its controller.

Reading alone did not convince me, so I cloned the library's repository and attached a test myself. I prepared two components that render different content and opened them in order under the same id.

const ControllerA = ({ isOpen }: { isOpen: boolean }) => (isOpen ? <div data-testid="b">AAA</div> : null);
const ControllerB = ({ isOpen }: { isOpen: boolean }) => (isOpen ? <div data-testid="b">BBB</div> : null);
 
overlay.open(ControllerA, { overlayId: 'id-close' });
overlay.close('id-close');
overlay.open(ControllerB, { overlayId: 'id-close' });

The results were as follows, run with vitest on the main branch at overlay-kit 1.9.0.

The third row is behavior the docs describe as a feature, so it was as expected. The problem is the first row.

The new JSX is ignored

I checked one more thing here. Further down in the same reducer there is this comment.

/**
 * @description Brings the overlay to the front when reopened after closing without unmounting.
 */
overlayOrderList: [...state.overlayOrderList.filter((item) => item !== action.overlay.id), action.overlay.id],

It says that reopening after closing brings the overlay to the front. But because the earlier branch hits return first, this line does not run in that situation. I stacked two overlays, ran close on the one underneath, reopened it, and the DOM order did not change.

Behavior that contradicts the comment

I did not confirm whether this shows up as a visible problem on screen, though. The test only looked at DOM order, and since most dialog libraries use their own z-index or portals, DOM order is not the same as visual order.

My version exposes the same situation as an option. When the same kind of modal is opened again, the call site chooses whether to keep the existing one or replace it with new props. Choosing replacement settles the existing Promise as replaced. This is one of the reasons I split the result type into five branches earlier.

This is where the two designs diverge. My approach treats modals as data. Because the string key and the props live in a store, they can be replaced or updated later. overlay-kit treats modals as functions. The controller is a closure, so once it is stored there is no way to change its contents from outside.

The price of JSX at the call site

So is writing JSX at the call site always better than using a string key? I do not think so. There are three prices to pay.

Code splitting drops out of the default. Writing JSX at the call site means the call site imports that component. If the modal is a heavy editor screen and only a few users press that button, its weight goes into the bundle of the list screen.

React.lazy does work inside a controller. I measured it the same way as the earlier test: the module was not loaded before open, and when opened, the Suspense fallback appeared first and then the body rendered.

const Heavy = React.lazy(() => import('./HeavyModal'));
 
overlay.open(({ isOpen, close }) => isOpen ? (
  <Suspense fallback={<Spinner />}><Heavy onClose={close} /></Suspense>
) : null);

So the difference is not whether it is possible but what the default is.

In my version every value in the registry file is an import() function, so code splitting is structurally enforced. With overlay-kit you have to write it by hand at each call site.

The user has to decide where to call unmount. If you only call close and stop there, the overlay's information stays in memory. The official docs warn about this and give separate guidance for where to hook unmount in each design system. MUI and Mantine have a callback for when the transition ends, so you can hook it there, but for libraries without such a callback the docs tell you to set a timer matching the animation length. The fact that the guidance is split per library is itself the cost of this decision.

Overlays render where the provider is. Because it does not use portals, the same code runs on React Native, which is an advantage. Conversely, context provided below OverlayProvider is not visible inside an overlay. When opening a modal that relies on a router or form context, check this point first.

How far the word declarative reaches

Having written this far, one thing nags at me. I started this post calling this interface declarative, yet overlay.openAsync(...) is a function call by any measure.

Let us start from the definition. The React documentation draws the line this way.

In React, you don't directly manipulate the UI--meaning you don't enable, disable, show, or hide components directly. Instead, you declare what you want to show, and React figures out how to update the UI.

The analogy in the same document is sharper. Imperative is calling out every turn to the person driving the car; declarative is getting into a taxi and naming your destination.

A declarative interface does not remove the imperative procedure. It moves it to the other side of a boundary and gives that boundary a name. The inside of overlay-kit we saw earlier is exactly that. The call site is one line, while inside, a procedure emits an event and dispatches twice.

So let us look at where each project applies the word. overlay-kit's docs call the previous approach imperative. What they point to is code that holds the open flag with useState and flips it in an event handler. What they call declarative is the side that treats an overlay as an action rather than as state. The toss tech blog's article on declarative code defines the word more broadly: declarative code is code at a higher level of abstraction, and a hook that abstracts the act of showing an overlay is one example.

So I decided to narrow the word this way. The call site does not write the closing code. When to close, and what remains on screen while it closes, are decided by the overlay; the call site writes only what to show and what it expects to receive back.

By this criterion, code written with useState that holds isOpen, and code that keeps a list of open modals in a global store while the call site closes them, are on the same side. Only the storage location differs; in both, the call site writes the closing statement.

Closing thoughts

What I fixed in our company's code was not, in the end, where the state was stored. It was who writes the closing code.

Even when we used a global store, the state was already outside the component, but the statements that opened and closed it were still at the call site. Only after switching to a single await did those statements move inside the modal.

overlay-kit reached the same place and then went one step further: it also puts what to show at the call site. I think that interface reads well. And while writing this, I learned what that one step costs.

It was the difference between treating a modal as a function and treating it as data. Writing JSX at the call site makes the controller a closure, and a closure's contents cannot be changed from outside once it is stored. That structure is where the ignored JSX on reopening with the same id comes from. Treating it with a string key makes the modal a value in a store that you can replace, update, and code split, but the call site only ever learns a name.

I think the feeling that you like a library's interface is a fairly reliable signal. Still, you should put into words at least once what exactly that feeling points at. Without doing that, you cannot tell whether what you liked was the interface or the reputation of the people using the library. I would encourage readers to do that exercise once for a library they are using right now.

I had to sort out these concepts first while testing the claims of Jev, a decision model released by TypeSafe AI. Instead of sentences, Jev returns a probability for each option, and it puts calibration forward as its training objective. To weigh that claim, I first needed to know what calibration is and why it drifts in existing models.

Accuracy and calibration

First you have to separate accuracy from calibration. Accuracy is what percentage you get right; calibration is whether you know what percentage you will get right. If you gather only the days a forecast said a 70% chance of rain and it actually rained seven times out of ten, that forecast is well calibrated. That does not mean it is accurate. It means it knows its own limits. A model that gets only 60% right scores full marks on calibration if it says 60% about itself.

The scale can drift in two directions. When the probability a model states is higher than its actual hit rate, that is overconfidence; when it is lower, underconfidence. Put the stated probability on the horizontal axis and the actual hit rate of each probability bin on the vertical axis, and you can see at a glance where the two split. A plot drawn this way is called a reliability diagram.

A curve on the diagonal means the model got right as often as it said. The further it sags below the diagonal, the less it gets right relative to the probability it states.

The metric for that gap is ECE (expected calibration error). Following the definition in Guo et al. 2017, for each probability bin it takes the difference between the "stated probability" and the "actual hit rate," weights it by that bin's share of the samples, and averages; 0 is perfect. The code below transcribes that formula. It compares a model that gets 6 of 10 questions right when it says 0.6 every time and when it says 0.95 every time.

// Guo et al. 2017 의 식 (3). 구간은 등간격 10개(논문 실험은 15개), 경계는 왼쪽 닫힘
function ece(preds, M = 10) {
  const bins = Array.from({ length: M }, () => ({ n: 0, hit: 0, conf: 0 }))
  for (const { p, correct } of preds) {
    const b = bins[Math.min(M - 1, Math.floor(p * M))]
    b.n++
    b.hit += correct ? 1 : 0
    b.conf += p
  }
  return bins.reduce(
    (sum, b) => (b.n ? sum + (b.n / preds.length) * Math.abs(b.hit / b.n - b.conf / b.n) : sum),
    0,
  )
}
 
// 10문제 중 6문제를 맞힌다
const answers = Array.from({ length: 10 }, (_, i) => ({ correct: i < 6 }))
console.log(ece(answers.map((a) => ({ ...a, p: 0.6 }))).toFixed(3)) // 매번 0.6 이라고 말할 때
console.log(ece(answers.map((a) => ({ ...a, p: 0.95 }))).toFixed(3)) // 매번 0.95 라고 말할 때

This is the result of running it with Node v24.16.0 on 2026-10-08.

0.000
0.350

Accuracy is 0.6 in both cases. Only the stated probability changed, yet ECE rises from 0 to 0.35. But because the formula takes an absolute value, ECE measures only the size of the gap, not its direction. A model that says 0.25 every time and still gets 6 questions right also gets an ECE of exactly 0.35. Whether it is overconfidence or underconfidence, you have to look at the reliability diagram or at average confidence minus accuracy.

The RLHF objective

Then why does a model polished with human feedback drift off this scale? First we need to see what RLHF (reinforcement learning from human feedback) optimizes. The skeleton of building a reward model out of human preference comparisons came from Christiano et al. 2017, an early carry-over to language models was Ziegler et al. 2019, and InstructGPT applied it to instruction following. The value pushed up is the score of a reward model that imitates human preference. The language-model work adds to the reward a KL penalty that keeps the model from straying too far from where it stood before reinforcement learning began, and InstructGPT also mixed in gradients from the pretraining data (PPO-ptx), but what it pushes up is still that score.

For a chatbot, human preference is the right objective. The trouble is that this preference leans away from wording that reveals uncertainty. Zhou et al. 2024 showed, on four public preference datasets, that annotators chose answers containing weakeners such as "I'm not sure, maybe" less often. The difference was small but significant. They did not, however, choose answers with strengthened, emphatic certainty more often. Leng et al. 2025 showed that reward models give higher scores to answers that state a high confidence score regardless of their actual quality. Both results concern expressions of confidence in text (verbalized confidence), not token probabilities. TypeSafe, which offers a training method meant to replace RLHF, also notes in its own primer that RLHF can reward confident-sounding hallucinations. That is the position of a vendor selling the alternative.

GPT-4's calibration curves

A drift in the same direction has been observed in token probabilities too. Figure 8 in the main body of the GPT-4 technical report places the reliability diagrams of the pre-trained model and the post-trained model side by side, and the caption reads:

Right: Calibration plot of the post-trained GPT-4 model on the same subset of MMLU. The post-training hurts calibration significantly.

The horizontal axis is the probability (logprob) the model assigned to each of the choices A, B, C, and D on MMLU multiple-choice questions, and the vertical axis is the actual hit rate in that bin. By the numbers printed on Figure 8, the pre-trained model's ECE is 0.007 and that of the model tuned with PPO (the reinforcement learning algorithm used in RLHF to update the model toward higher reward-model scores) is 0.074. More than ten times worse.

In the right-hand (PPO) panel of Figure 8, bars at 0.4 and above sit below the diagonal, but the bins between 0.1 and 0.3 actually sit above it. The bars between 0.1 and 0.9 are bunched flat at an actual hit rate of roughly 0.3 to 0.5, so as the probability the model states goes up, the hit rate barely follows. Those underconfident bins are mixed into the 0.074 as well. And as Guo et al. point out, a reliability diagram does not show how many samples fall in each bin, so the distance of a bar from the diagonal cannot be assumed proportional to the share that bin contributes to ECE.

This panel differs from the concept diagram above in two ways. First, it does not just sag below the diagonal like the curve in the concept diagram; it crosses the diagonal once. Second, it puts the probabilities of all four choices into the bins, so it also differs from the definition in the code above, which measures the probability of the single chosen answer. So I do not compare 0.074 with the 0.35 from the code above on the same scale.

So why did it get worse? The report's main text says only that the pre-trained model was well calibrated and that calibration was reduced after post-training; it does not state a cause. A clue lies in Anthropic's Kadavath et al. 2022. The RLHF policies they trained from their own language models looked very poorly calibrated at face value. The paper called this unsurprising, since RL fine-tuning tends to collapse predictions toward the behaviors that receive the most reward. It was a quick side experiment, not the paper's main subject. Yet applying a single temperature, T=2.5, to every evaluation largely fixed the calibration problems on three evaluations. Temperature is a value that divides the logits; when it is greater than 1, it leaves the order of answers intact and only flattens the probability distribution. I read the fact that one value largely fixed it as meaning the drift was not concentrated in a few bins but was the whole distribution narrowed to one side. The paper also added the caveat that more intensive RL training might distort calibration in ways that cannot be fixed like this.

This is a result on Anthropic's models, so it cannot be carried over directly as the cause of Figure 8. The studies in the previous section also dealt with expressions of confidence in text. I could not find a primary source that directly links the finding that people avoid uncertain wording to the drift in token probabilities.

The probability your code receives

The probability in Figure 8 is a token log probability. To receive the same kind of value, following the OpenAI API specification, you set logprobs to true in a Chat Completions request and use top_logprobs to choose, between 0 and 20, how many candidates to receive at each token position. A value you get by having the model state a confidence number along with its answer is confidence stated in words, and it is a different value from the token probability.

Studies of the two values compare different pairs. Tian et al. 2023 reported that inside RLHF models such as ChatGPT, GPT-4, and Claude, confidence stated in words was typically better calibrated than conditional probabilities, often reducing ECE by a relative 50% or so across three benchmarks. In this comparison, though, the conditional probability of models whose weights are not public was not the API's token probability but an estimate: the share of 10 samples of the same question that produced the answer. Leng et al., seen above, reported that compared with pre-RLHF models, RLHF models are more overconfident in the confidence they state in words. The two results can both hold. And 0.074 is the value the 2023 GPT-4 post-trained model produced on a subset of MMLU, so it cannot be carried over as is to the models you call through the API today.

Wrapping up

To sum up, calibration is not about what percentage you get right but whether you know what percentage you will get right, and ECE is the number that measures the size of the gap. For the direction, you have to look at the reliability diagram or at average confidence minus accuracy. RLHF pushes up the score of a reward model that imitates human preference. Kadavath et al. see that optimization as tending to narrow predictions toward what earns the most reward. There is research showing that people choose answers that reveal uncertainty less often, and that reward models give higher scores to answers stating a high confidence score regardless of quality. In GPT-4, ECE went from 0.007 to 0.074 after post-training, but OpenAI did not go as far as stating the cause. Calibration also comes out differently depending on whether the probability your code receives is a token probability or confidence stated in words, so whichever it is, you need to measure it again on your own data.

Whether the probabilities of Jev, a decision model that puts calibration forward as its training objective, are actually honest is covered in Decision Models, Jev and Kev.

Before you hard-code a model's probability as a threshold, I hope you will check at least once what that model was trained to aim for, and measure whether its scale holds on your own data.

As a developer, I spend as much time working out what to build as I spend writing code. I exchange requirements with product managers, align on screens with designers, and ask people who know the domain what a term actually means. And I have experienced several times that the cost of getting this process wrong is far greater than the cost of writing the wrong code.

I recently read Engineering Leaders' Day-to-Day Activities by James Samuel. The author divides a leader's work into six categories, and the first one he covers is gathering information. His reason is that every decision, direction, and action depends on an accurate understanding of what is happening right now.

I do hope to take on a leadership role someday, but what struck me after reading that passage was slightly different. If the first thing a leader handles is gathering information, then the individual contributor's counterpart to that is understanding requirements. The way I handle requirements now will become the way I handle an organization's information later.

I am not trying to write about leadership here. This is about what I have been doing wrong as an individual contributor communicating across functions, and what I can use as tools. To state the conclusion first, I came to believe this is not purely a matter of attitude. There is an alternative framing available, called ubiquitous language, and it is worth looking at what that means.

The same word, different meanings

In his article on Bounded Context, Martin Fowler gives the example of an electricity company. A large share of the misalignments I have run into took this shape.

Here the word “meter” meant subtly different things to different parts of the organization: was it the connection between the grid and a location, the grid and a customer, or the physical meter itself?

here the word 'meter' meant subtly different things to different parts of the organization: was it the connection between the grid and a location, the grid and a customer, the physical meter itself

The same word, “meter,” meant subtly different things in different parts of the organization. And Fowler adds that this confusion is not an accident particular to that case. He says he has repeatedly seen the same confusion around polysemous words like Customer and Product.

This did not happen because the meeting participants were bad at meetings. When an organization contains different contexts, it is natural for the same word to diverge, and if that divergence is left alone it eventually splits the code as well.

I covered the technical background of this story once in Domain Model. That post was about how to express a model inside the code. This post is about the stage before that model exists, when people are still talking to each other.

A word with two meanings

But why does this repeat structurally? There are two lines of explanation.

One comes from organizational structure. In How Do Committees Invent?, the paper Mel Conway published in Datamation in 1968, he says that to the degree an organization is unable to flexibly change its communication structure, it stamps its own image onto every design it produces. The single sentence usually quoted as Conway's Law is in fact a formulation the author himself produced later, and I find this 1968 passage more direct.

What I took from that sentence in practical terms is the word “stamps.” It means that diverging terminology does not end with the conversation but remains in the artifacts. The “meter” above is exactly that. If people build separately while holding different understandings of the scope of “meter,” that difference persists not in the meeting notes but in table names, API response fields, and the wording on screens. And from that point on, the cost of fixing it is nowhere near the cost of having one more conversation.

This is where what an individual contributor can and cannot do separates. Changing an organization's communication structure is beyond an individual contributor's reach. But making the boundaries of terminology visible is something you can do from where you sit right now. It means writing down the scope a word has in this document, and noting explicitly when it differs from the word another team uses. Every tool we look at later is a device that makes that job easier.

The other explanation is a bit more fundamental. Seen from cognitive science, collaboration only holds together on top of a shared background in the first place. Grounding in Communication, written by Herbert Clark and Susan Brennan in 1991, opens with the example of a piano duet. The two performers cannot even begin to coordinate without presupposing a vast amount of shared information, that is, common ground. Common ground here means mutual knowledge, mutual beliefs, and mutual assumptions. And the authors state firmly that all collective action is built on common ground and on its accumulation.

Here is the perspective I arrived at after placing these two lines side by side. Diverging terminology is not an accident; it is the default state. Being aligned is the exception, the state that has to be maintained at a cost. Which makes the practical question this: who pays that cost, and how?

Active watching

In Domain-Driven Design, published in 2003, Evans presents a pattern called ubiquitous language, or shared language. It is often summarized as little more than “developers and domain experts should use the same terms,” but reading the original text, the prescription is far more specific. He asks the team to use the model as the backbone of the language and to commit to using that language relentlessly in all communication within the team and in the code. And the sentence that follows was the most important part for me.

Domain experts raise objections to terms or structures that are awkward or inadequate for conveying an understanding of the domain, while developers stay alert to any ambiguity or inconsistency that will trip up the design.

Domain experts object to terms or structures that are awkward or inadequate to convey domain understanding, while developers watch for ambiguity or inconsistency that will trip up design.

Domain experts object to terms and structures that are awkward or inadequate for conveying domain understanding, and developers watch for the ambiguity or inconsistency that will trip up the design.

Reading that sentence, I corrected my understanding of my own role. Until then I had regarded myself as a receiver in requirements meetings. I thought my job was to take whatever the product side decided and implement it accurately. But the developer's share as Evans defines it was not reception. It is finding ambiguity and handing it back. That is not passive cooperation; it is active watching.

(For reference, Fowler rendered this sentence in his own article as “Domain experts should object ... developers should watch.” The original book has no modal verb. It is a small difference, but the original reads as more definitive. The tone is not that doing so would be good advice, but that doing so is the definition of the pattern.)

So how do you find ambiguity? Evans answers that too. Using the terms repeatedly in conversation is what exposes differences in how they are interpreted. This is how I read that sentence: ambiguity is not something you find by reading documents closely. No matter how carefully you read a spec, the fact that “meter” means several different things will not surface. That word has to be used repeatedly against concrete cases before it splits apart. The moment someone asks, “What does everyone else understand meter to mean?” is that point.

The principle of least collaborative effort

We may know this method and still fail to practice it. So why do we not ask repeatedly?

Let us go back to Clark and Brennan's paper. The authors give the name grounding to the process of turning what has been said into part of the common ground. And they offer one principle about how people behave during that process: the principle of least collaborative effort. It comes from the observation that people do not like to work harder than they have to.

Here comes the important passage.

Under the principle of least collaborative effort, people should aim to reach grounding with no more combined effort than is required. Yet which actions cost effort shifts dramatically from one communication medium to another.

By the principle of least collaborative effort, people should try to ground with as little combined effort as needed. But what takes effort changes dramatically with the communication medium.

What takes effort changes dramatically depending on the medium. A confirmation technique available in one medium may be outright impossible in another, or possible only at a much higher cost. The authors point out in particular that in media where the other party does not receive your words immediately, the cost of relying on someone else to correct a misunderstanding becomes very high, so speakers try to avoid it.

There is one thing to note here. This paper was written in 1991, and the media the authors compared were things like face-to-face conversation, telephone, letters, and answering machines. It does not deal with Slack or Notion. The framework that grounding cost varies by medium is theirs; applying it to today's asynchronous work channels is my interpretation.

Applying it that way explains why I did not ask repeatedly. In an environment where you receive a spec as text and confirm it as text, sending a confirmation message at every ambiguous point is expensive. You do not know when the answer will come, and you worry that asking several times will make you look like you did not understand. So people naturally slide into their own interpretation.

It is worth separating two forces here. One is that the medium raises the cost of confirmation, and that is Clark and Brennan's point. The other is the worry that asking several times will make you look incompetent, and that is not what their principle predicts but belongs to the psychological safety discussion later on. To be precise, what the principle of least collaborative effort predicts is that you will choose a cheaper way to confirm, not that you will stop confirming. Abandoning confirmation and moving on to a guess is not the principle operating; it is grounding failing. And the fact that the interpretation was wrong surfaces only after implementation is done.

Reaching a sufficient level

So should you ask about every ambiguity? That is not realistic, and I have never seen anyone do it. The paper offers a standard for this question too: the grounding criterion. It refers to the state in which both parties believe the listener has understood what the speaker meant to a level sufficient for current purposes. The authors preface it with the observation that perfect understanding is impossible to begin with.

The reason this criterion felt practical to me is that it lowers the target. You do not need to understand requirements perfectly. It is enough to be able to mutually trust that you understand them well enough for what you are about to do. And the authors say that when the purpose changes, the criterion has to change with it.

In practice, this becomes the standard for calibrating how hard to press for confirmation. If you dig into “What does meter mean here?” in a session that is still exploring direction, the conversation stops moving. What is needed at that stage is agreement on what you are trying to do and why, not the settling of boundary values. Conversely, right before implementation begins, that question absolutely has to come up. At that point you need to be able to trust that both sides are picturing the same scope for “meter,” and if you cannot, code will pile up on a false premise.

So when I meet the same ambiguity, I handle it differently depending on which stage I am in. During exploration I write it down on a list and move on; right before implementation I open that list and close the items one by one. Deferring confirmation and abandoning confirmation are different things.

There is one tool that pairs well at this point. Team Topologies, by Matthew Skelton and Manuel Pais, distinguishes three modes of interaction between teams: collaboration, where two teams discover something new together for a defined period; X-as-a-Service, where one side provides and the other consumes; and facilitation, where one side helps and mentors the other.

It is originally a story about organizational design, but I found it practical to shrink it down to the scale of a single meeting. If the participants hold different beliefs about which mode this conversation is in, the meeting goes off the rails. If the product manager thinks this is a session for conveying what has already been decided while the developer thinks it is a session for discovering together, the developer's questions sound like objections rather than collaboration. The reverse happens too. If the developer came to receive a finalized spec while the product manager was still exploring, the developer gets frustrated that there is no spec.

So these days I check this at the start of a meeting: “Is this still open, or is this a session to confirm what has already been decided?” That single question changes the character of everything that follows. It is the same thing as adjusting the grounding criterion to fit the purpose.

When the resolution does not match, the conversation spins in place

Even with the mode aligned, a problem remains. Another misalignment I have run into often is talking at different resolutions.

Requirements handed over as prose from a product manager are usually too abstract. A sentence like “so users can conveniently check their reservation history” does not say how many screens there are or what leads where. Conversely, a designer's mockup is too concrete. Button colors and spacing are already decided, which makes it hard to discuss the very question of whether this flow is right.

Shape Up, written by Ryan Singer at Basecamp, names this problem precisely. Starting from wireframes or concrete visual layouts traps you in unnecessary detail and prevents you from exploring as broadly as you need to. So what Shape Up proposes is a representation in between. It is called breadboarding, a concept borrowed from electrical engineering. A breadboard is a prototype that has all the components and wiring of the real device but none of the industrial design. So there are exactly three things you draw: the places you can navigate to, the affordances the user can act on, and the connection lines that show where those actions take the user.

The reason I liked this technique is that it is an artifact a developer can produce. Instead of asking the product manager to write in more detail or waiting on the designer for a mockup, you can draw the flow as you currently understand it, right there, and hand it back with “this is how I understood it, is that right?” It is a concrete way of executing the watching Evans described, the handing back of ambiguity. And seen through the lens of the previous section, it is a device that lowers grounding cost. One picture stands in for several rounds of confirmation over text.

But this resolution is harder to hold than it sounds. As you draw the flow, you naturally slide down into things like “wouldn't this button be better in the bottom right?” I have done it several times myself. Once you slide down like that, the question that was being discussed, namely whether this flow is right, quietly disappears. If you are deciding button positions while the flow is not settled yet, all that discussion gets thrown away wholesale when the flow changes later.

So at this stage I try to keep the drawing deliberately bad. If it is nothing but boxes, arrows, and names, the other person does not think to point at details and responds only to the flow. Low fidelity is a feature of this tool rather than a defect. (Suppressing the urge to make it pretty is harder than it sounds.)

The idea that the level of abstraction can itself be a tool for collaboration connects to what I wrote in Abstraction. That post was about levels of abstraction in code; conversations have the same thing.

Separating rules, examples, and questions

Another way to align resolution is to structure the discussion itself.

Consider Example Mapping, introduced in 2015 by Matt Wynne, who led the Cucumber project. He diagnoses the reason many teams struggle with requirements discussions as the lack of structure, which makes them long and boring. And so teams end up not doing them regularly or consistently. That was precisely my situation. When a requirements meeting runs long, you try to keep the next one short, and when you keep it short the ambiguities stay in place.

Example Mapping splits the discussion across four colors of cards. Yellow is the story under discussion, blue is a rule or acceptance criterion, green is a concrete example that illustrates that rule, and red is a question nobody knows the answer to.

Of these, I think the red card is the core. The other three are organizing techniques, but the red card is a device that turns “I don't know” into an official output of the meeting. Saying that you do not know stops being an act that delays the meeting and becomes a result the meeting is supposed to produce.

Why that matters becomes clear if you imagine the opposite. In a meeting without red cards, running into an ambiguity leaves two options: dig in now and stretch the meeting out, or move on and interpret it alone later. As we saw in the previous section, people generally choose the latter. But with one card available, a third option appears. You write it down and move on, without letting it disappear. The meeting keeps moving, and the ambiguity remains on the list.

Green cards play a similar role. When only the rule is written, everyone agrees; the moment someone tries to write a concrete example illustrating that rule, interpretations diverge. It is Evans's “using the terms repeatedly in conversation exposes differences” transposed into a meeting format. If you ask people to attach three examples to a single rule, one of them will usually produce a “wait, what happens in this case?” (In my experience, a meeting where that question does not come up is not a meeting where everyone understood, but a meeting where everyone understood differently.)

A similar idea long used in BDD circles is the Three Amigos. The point is that the three perspectives of business, development, and testing each ask a different question, and the Agile Alliance glossary frames those questions as what problem are we trying to solve, how might we build a solution to solve it, and what about this, what could possibly happen. What becomes of a meeting where the third question never gets asked probably does not need spelling out.

Create a process for checking what you do not know

But why do we have such a hard time playing the red card? At this point there is no avoiding a discussion of psychological safety.

The paper that Amy Edmondson published in Administrative Science Quarterly in 1999 is the study that formally established the concept of team psychological safety. (The paper itself credits Schein and Bennis's 1965 work as its root.) It defines team psychological safety as a shared belief that the team is safe for interpersonal risk taking. Saying you do not know, voicing dissent, and admitting a mistake all fall under that. The paper surveyed 51 teams at a manufacturing company and showed that psychological safety is associated with learning behavior, and that learning behavior mediates between psychological safety and team performance.

But the sentence I found most practical in that paper was not the definition; it was the one right after it.

In most cases this belief remains tacit: it is taken for granted, and neither the individual members nor the team as a whole give it direct attention.

For the most part, this belief tends to be tacit—taken for granted and not given direct attention either by individuals or by the team as a whole.

This belief is usually tacit, and neither individuals nor the team give it direct attention.

From here on this is my own conjecture. What the paper says stops at the belief being tacit; the prescription that you should therefore create a form is not in the paper. That said, I have seen several times that a tacit thing does not change through declaration. Saying “our team is a team where it is fine to ask about what you do not know” does not change much on its own. That is why I think the red card from the previous section matters. If you turn “I don't know” into a field inside a form, filling it in becomes procedure rather than courage.

There is an obvious counterargument, of course. On an unsafe team, the red card field will simply stay empty. That is a fair point, and I have no intention of claiming this tool creates safety. But if the field exists, at least the fact that it is empty becomes visible. It creates an opening to ask whether it is empty because nobody has an open question, or empty because it is hard to speak up.

The same idea has made its way into actual document formats. Look at the Bounded Context Canvas created by the DDD community: it is a collaborative tool for designing and documenting a single context, and its arrangement of fields is interesting. Name and purpose, strategic classification, domain roles, inbound and outbound communication, and then Ubiquitous Language, business decisions, Assumptions, verification metrics, and Open Questions.

There is a separate field for writing the shared language, one for writing assumptions, and one for writing open questions. The canvas's own description says that the act of writing the purpose forces fuzzy thinking to be stated clearly and puts the whole team on the same page.

I think this is the most realistic approach to communication problems. Instead of trying to change people's attitudes, you make a field for the things that are hard to say. A good structure does not demand that people perform well; it makes performing well easy.

Why this is not a matter of taste

Reading this far, one reaction is available: nice ideas, but isn't this ultimately just “communicate diligently”? And isn't that a personality trait?

I thought that for a while too. But there is data on this part.

When DORA addresses organizational culture, it borrows the sociologist Ron Westrum's typology: power-oriented pathological, rule-oriented bureaucratic, and performance-oriented generative. And DORA's official documentation summarizes their research findings this way.

An organizational culture built on high trust, one that places weight on the flow of information, predicts how well software delivery performs.

organizational culture that is high-trust and emphasizes information flow is predictive of software delivery performance

An organizational culture that is high-trust and emphasizes information flow is predictive of software delivery performance. The claim is not that teams communicating well feel good, but that it is a variable moving together with performance.

That said, “predictive” must not be read as causal here. DORA's data comes from a survey in which the same respondents answered about both culture and performance, and it is a correlation at the organizational level. It remains possible that people satisfied with their organization rated both generously, and a prescription for how an individual should behave does not follow directly from a relationship at the organizational level. So this is as far as the material takes us: information flow does not live only in the realm of taste, it sits where it moves together with performance.

The same document also lays out Westrum's three properties of good information: it answers the question the receiver needs answered, it is timely, and it is presented in a way the receiver can use effectively. It is worth noticing that all three set the standard at the receiver. Sharing a lot is not what makes information flow good.

Google's Project Aristotle is another frequently cited source. According to what was published on reWork, they studied 180 teams, ran more than 35 statistical models over hundreds of variables, and put forward five factors affecting team effectiveness: psychological safety, dependability, structure and clarity, meaning, and impact. And the conclusion was that how the team worked together mattered more than who was on it.

One thing is worth flagging, though. The official page states that it lists the five factors in order of importance and puts psychological safety first. But magnitude claims like “overwhelmingly number one,” which show up often in articles citing this research, are not on that page. Having an order and dominating the rest are different claims. So this post makes no claim about magnitude. If you want to argue for the importance of psychological safety, leaning on Edmondson's original paper from the previous section is the accurate move.

Back to the leader question

Let us return to the article quoted at the start.

While covering information gathering first, James Samuel says the methods from your individual contributor days stop working. As an individual contributor you hold the whole picture of your own work, but once you become responsible for people you can no longer use the old approach. So he names as a required capability the ability to filter out noise and synthesize information into a coherent picture of reality, because no manager can process everything.

I felt this was the same point as Westrum's three properties of good information above. It is not about collecting a lot but about turning it into a usable form. And that is no different from what I am doing right now while handling requirements. Picking out the ambiguous parts of a spec's sentences, redrawing them as a breadboard to confirm, and leaving what I do not know as a red card is exactly the practice of filtering noise into a coherent picture.

What the author says about decision making stayed with me too: waiting for certainty is itself a decision, and it carries a cost. Waiting until requirements become completely clear is no different. Which is why the grounding criterion, sufficient for current purposes rather than perfectly understood, becomes the practical standard.

Closing thoughts

Here is the summary.

Cross-functional communication failures are usually not purely a matter of attitude. When an organization contains different contexts, the same word diverging is the default state, and being aligned is the exception that has to be maintained at a cost. The way that cost gets paid is shared language, and within it the developer's share is not transcribing requirements but watching for ambiguity and handing it back.

But handing it back takes effort, and people try to minimize that effort. So instead of relying on willpower, I decided to use three things: checking first which mode this conversation is in, drawing the flow at the resolution between prose and mockup and handing it back, and building a field for what you do not know into the meeting format. None of the three demand that you fix your attitude; they make the right thing easier to do.

I am not a leader yet, and I have not verified any of this from that seat. Some of it will not hold at a different team size or in a different organizational culture. But there is one thing I feel confident about now. The way I handle information later will not be newly learned; it will be the way I handle requirements today, grown to scale. So these days I try to be less embarrassed than I used to be about asking “what do you mean by that?” in meetings.

If you can think of one word from a recent meeting that you nodded along to without actually being sure about, I would suggest using that word one more time next time. Use it repeatedly and it will surface.

While wrapping up the previous article on saving tokens, one question kept nagging at me. I believe the center of gravity is shifting away from crafting individual prompts and toward the practice of clearing away and selecting information—context engineering. Once that article was finished, the next question followed naturally: what comes after context?

Writing about “where things are headed” calls for caution. The future contains many possible paths, so the focus here is on a shift in emphasis visible in already-published primary sources. Some inference is inevitably involved, and I hope readers will approach it from multiple perspectives.

From prompts to context

Let us begin by clarifying the terms. For a while, the industry’s central topic was prompt engineering. The problem was how to write a single instruction to a model well: how to design clear directions, useful examples, and an output format.

Then the unit of work grew. As agents running for dozens of turns became common, the important question was no longer one prompt but how to compose the entire context the model sees on every turn (system prompt + tool definitions + conversation history + search results + memory). This is called context engineering. Anthropic organized this frame in its September 2025 article “Effective context engineering for AI agents,” and the context rot research published that same year by the Chroma team (Hong et al.) added quantitative evidence. Across 18 models—including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3—they showed that performance degrades unevenly as inputs grow longer, even on tasks as simple as copying words verbatim. In practice, the common assumption that a model treats the 100th token and the 10,000th token equally does not hold. The conclusion was not “longer context is better,” but “how information is placed matters as much as what it contains.” That finding reinforced the move from filling context to selecting it, and the term quickly took hold.

One common misunderstanding is worth addressing. Prompt engineering was not replaced by context engineering. Writing good prompts is still foundational; context engineering is closer to a higher-level concept built on top of it. (A shift in attention from writing good code to designing good systems does not make coding unnecessary.) The precise description, then, is not that we “switched over,” but that the field “expanded by inclusion.”

So, to ask again: did that expansion stop at context? It does not appear so.

The Anthropic engineering blog

I think the most honest way to gauge direction is to read, in chronological order, what organizations actually driving the field are writing about. Follow the Anthropic engineering blog after “Effective context engineering” (September 2025), and the titles alone begin to outline where the center of gravity moved next.

• October 2025, Equipping agents with Agent Skills
• November 2025, Code execution with MCP: more efficient agents
• November 2025, Effective harnesses for long-running agents
• January 2026, Demystifying evals for AI agents
• January 2026, Designing AI-resistant technical evaluations
• February 2026, Quantifying infrastructure noise in agentic coding evaluations
• March 2026, Harness design for long-running application development
• April 2026, Scaling Managed Agents: separating brains from hands
• May 2026, How to contain Claude across products

Step back from the list and the keywords cluster into three branches: harness, eval, and containment. I read this as a discussion moving one level beyond how to fill and clear context, toward designing, measuring, and controlling the agent system as a whole. (There is, of course, a limitation here: these are one company’s priorities. Yet given that company’s role in the coding-agent ecosystem, it is difficult to dismiss them as the interests of only one vendor.)

Let us take them one at a time.

harness

The word harness may be unfamiliar in this setting. Literally, it is the equipment fitted to a horse to direct its power where we want it to go. For an AI agent, a harness is the complete framework outside the model that surrounds it and puts it to work: which tools it may use and in what order, how it recovers after a failure, how far its permissions extend, and when its loop stops.

If context engineering asks, “What should we show the model?”, harness design asks, “How should the model move within this environment?” It sits one layer farther out. Imagine the model as a talented new hire: context is the work material handed to that person, while the harness is closer to the workplace and operating procedures. The same person’s performance can wobble in a chaotic environment, yet go much farther with the same abilities atop a well-designed process.

Why this matters becomes clear in a failure the Anthropic engineering team encountered firsthand. They put their top coding model, Opus 4.5, on the Claude Agent SDK, supplied only high-level directions such as “build a clone of claude.ai,” and ran it across multiple sessions. Despite the model’s intelligence, it did not produce a production-grade app. The failures repeated in two forms. In one, a session tried to finish everything at once, exhausted its context mid-implementation, and left the next session a half-built feature. In the other, a later session saw that “quite a lot has already been done” and declared completion even though valid work remained. (Imagine a long job passed among people working shifts, with every replacement having no memory whatsoever of the person before them.)

The solution was not to make the model smarter, but to change the framework. The first session received a dedicated prompt for setting up the environment (an initializer agent), expanded more than 200 feature specifications into feature_list.json, and created init.sh to start the development server along with a progress log (claude-progress.txt). Each later session (coding agent) handled exactly one feature, left behind a clean state through a git commit and progress notes, and then ended. The next session first read that progress file and the git log to learn “how far the previous shift got” before continuing. The model was identical, but the result changed when placed in this framework. The harness, not the model, made the difference. (Interestingly, none of Anthropic’s remedies were new. Feature lists, small commits, progress notes, and smoke tests run every time are exactly what experienced developers do each day. A good framework for an agent is ultimately much like embedding good engineering habits into its environment.)

The connection to token efficiency is also clear. Subagent isolation, lean tool definitions, and model routing, which the previous article covered, look like separate cost-saving techniques in isolation. Taken together, however, they are parts of how one harness is designed. Harness design encompasses deciding which model lane receives each task, which tools remain enabled, and where verbose exploration is isolated. The savings are closer to a byproduct of that design.

Harness design has one peculiar trap, however: the better the framework, the more likely it is to become obsolete when the model improves. A harness is fundamentally a collection of assumptions about “what the model cannot do on its own.” The moment the model can do those things, those assumptions become excess baggage. An example from Anthropic illustrates this precisely. Sonnet 4.5 had a habit of rushing to finish as it approached its context limit—a behavior called context anxiety—so the framework included a context-reset mechanism. When the same framework was used with Opus 4.5, that behavior had disappeared, turning the carefully added reset into dead weight. Every time the model becomes one step smarter, another part of the framework can reach its expiration date.

That leads to a further idea: instead of perfecting a particular framework, design interfaces that remain stable even as the framework changes. Anthropic’s Managed Agents follows this direction, and its roots lie, perhaps unexpectedly, in operating systems. The OS endured for decades because it virtualized hardware into abstractions such as processes and files, creating a container in advance for programs that did not yet exist. A single line of read() works the same whether the storage is a 1970s disk or a modern SSD. Applying the same reasoning divides an agent into three pieces: the brain that makes decisions (Claude and the harness), the hands that perform actions (code-execution sandboxes and tools), and the session log that appends everything that occurs. With the three separated, the brain can handle a dead container as a tool-call error, and even if the harness dies, it can wake again from the last point in the session log. Cost and latency also fell as a side effect. By starting containers only when truly needed, Anthropic reports that time to first token (TTFT) fell by about 60% at the median and, at p95, by more than 90%. (This is another point of contact with the token-efficiency article. It recommended placing static material first for prompt caching; arranging context to improve that cache hit rate is exactly the job of the harness beside this brain.)

eval

The second branch was the most interesting to me personally. As if by agreement, Anthropic’s articles from early 2026 converge on evaluation (eval).

The reason is intuitive. How do we verify that context was composed well, the harness was designed well, or costs were truly reduced? The longer and more complex the tasks agents handle autonomously, the harder it becomes for a person to inspect each one and decide, “Did this actually work well?” The basis of trust ultimately shifts to measurement. That is why questions such as “How should agent evaluation be designed?”, “How do we remove noise from the evaluation itself?”, and “How do we handle eval awareness, where a model detects an evaluation and changes its behavior?” have moved to the foreground.

Evaluating agents is difficult because it differs in kind from a one-shot question and answer. An agent calls tools and changes state over multiple turns, so one mistake propagates forward and accumulates. Results also vary from run to run even with identical inputs. Anthropic divides this nondeterminism into two metrics. pass@k is the probability of succeeding at least once across k attempts, so it rises as attempts increase. pass^k is the probability of succeeding on all k attempts, so it falls as attempts increase. pass@1 matters for code generation that only needs to be right once; pass^k is central for a customer-service agent that must work reliably every time. (If the per-trial success rate is 75%, the probability of three consecutive successes is 0.75³, or about 42%. The gap between “usually works” and “works every time” is that large.)

Then what grades a single attempt? The same article divides graders into three types. Code-based graders (test results, static analysis, and tool-call validation) are fast, inexpensive, and objective, but weak on open-ended tasks with multiple valid answers. Model-based graders (LLM-as-judge and rubric scoring) can capture subtle quality differences, but are nondeterministic and must be calibrated periodically against human judgments. Human-based grading is the most accurate but also slow and expensive. In practice, teams combine all three, ideally using deterministic grading as the foundation and model grading as support. There is another distinction. A capability eval asks, “What can this agent accomplish?” It therefore starts at a low score and provides a hill to climb. A regression eval asks, “Can it still do what it used to do?” It must remain near 100%; a lower score signals that something has broken.

One surprisingly common trap is that a low score may be the evaluation’s fault, not the agent’s. Anthropic reports that Opus 4.5 initially scored 42% on CORE-Bench. Investigation revealed causes such as rigid grading that expected “96.124991…” and marked “96.12” wrong, ambiguous task specifications, and irreproducible stochastic tasks. After fixing bugs and rerunning with a scaffold whose constraints had been loosened, the score jumped to 95%. They therefore emphasize one principle: do not take the score at face value; read the transcript yourself. If a frontier model makes 100 attempts and scores 0%, the problem is usually broken rather than the model being incapable.

It is also interesting that as measurement advances, the baseline itself moves quickly. On SWE-bench Verified—the leading coding-agent benchmark, which assigns real GitHub issues and grades whether tests pass—frontier models climbed from the 30% range to above 80% in 1 year. At that point, easy problems are all solved, scores reach a ceiling (saturation), and a paradox appears: large capability gains show up only as small score differences. One code-review startup reportedly dismissed a new model after looking only at one-shot evaluations, then recognized the improvement only after switching to agentic evaluations that measured longer, more complex work. An eval is therefore not something created once and finished; it becomes a living asset that must continually be replaced with harder versions. (Anthropic compares this to the “Swiss cheese model” in safety engineering. One slice full of holes cannot block a failure, but layering automated evaluation, production monitoring, and human transcript review means a failure that slips through one layer is caught by the next.)

containment

The third branch has a somewhat different character. It is about safety and control, not cost or performance.

As agents gain more tools and move more autonomously, the blast radius of a single mistake grows with them. For an agent that can delete files, make external requests, and perform privileged actions on someone’s behalf, “Where do we stop the damage when it goes wrong?” matters as much as “How well does it perform?” The prominence of Anthropic’s May 2026 article on containment across its products can be read in this context. It divides agent risk into three branches: user misuse, in which a user maliciously or carelessly requests harmful work; model misbehavior, in which the model independently acts without being asked; and external attacks arriving through tools, files, or networks. One interesting observation is that smarter models do not simply reduce risk. Less capable models misread situations and make obvious mistakes; more capable models make fewer mistakes, but are better at discovering unexpected routes around constraints nobody explicitly wrote down.

Anthropic emphasizes the limits of “having a person supervise every action.” Claude Code initially sought safety by asking the user to approve each write, execution, and network-access request, but telemetry showed that users simply approved about 93% of the prompts. As approval windows multiply, approval fatigue sets in and people pay less attention to each one. A probabilistic defense that depends on human clicks will therefore retain holes. The center of gravity moves from “monitor what the agent is doing” to “restrict what the agent can do in the first place.” Defenses are layered across three levels: the environment layer, including sandboxes, VMs, and egress controls; the model layer, including system prompts and classifiers; and the external-content layer, including MCP, plugins, and search results. The central principle is to put the environment layer, which blocks actions deterministically, in place first. This is not because model-layer defenses are weak. On Gray Swan’s benchmark for prompt injection, the one-shot attack success rate is about 0.1%, which is best-in-class. But 100 adaptive attempts raise it to 5–6%, and a probabilistic defense inherently cannot achieve a 100% hit rate. A hard final boundary is therefore placed behind it. (Anthropic says introducing OS-level sandboxing reduced approval prompts by 84%. The safety mechanism actually reduced friction.)

This may seem remote from token efficiency, but the two share the same root. Both ask, “What should the agent be given, and how much?” Anthropic demonstrates the connection with two cases. In one, an internal red team phished an employee into running Claude Code with a malicious prompt. A quietly inserted instruction made it read ~/.aws/credentials and POST the contents externally; it succeeded in 24 of 25 attempts. Because the user had typed the instruction directly, the model classifier saw nothing suspicious. What stopped it was not a smart model, but the environmental boundary that kept credentials outside the sandbox in the first place, together with egress control. The other case is subtler. An egress allowlist correctly allowed api.anthropic.com, but a file planted by an attacker used the attacker’s own API key to call Anthropic’s file-upload API, causing data to leave through the attacker’s account. The sandbox worked perfectly, yet the data still leaked. The lesson was that an allowlist must be understood not as a “destination filter,” but as “permission for every capability available through that domain.” (Anthropic repeatedly stresses this principle: proven hypervisors, syscall filters, and container runtimes held up; what actually broke were the components Anthropic had built on top of them.) Removing unused MCP tools belongs to the same context. It saves money while also reducing the attack surface. A design kept lean is not only cheaper and more accurate, but safer too.

Putting it all together

The three branches can be condensed into one sentence. The unit of attention is expanding outward, one step at a time: from the prompt (one instruction), to context (what to show on each turn), and then to the whole agent system (how to operate it, measure it, and contain it). Harness asks, “How should it run?”; eval asks, “How do we know it ran well?”; and containment asks, “How do we stop it when it runs wrong?”

To stress the point once more, these three branches are a shift in emphasis that I infer from already published material, not a prediction that “this will become the standard in the second half of 2026.” Some currents will grow, while others will be absorbed under different names. What is clear is that neither prompts nor context will disappear; they will remain parts of a larger frame. The field has advanced by adding another layer on top rather than erasing old terms with new ones, and it seems likely to continue that way.

We started with cost and arrived here. The previous article recommended opening up your token ledger; this one goes a step farther. It is worth examining together how you verify the savings (eval), what framework makes those savings repeatable (harness), and how far that framework remains safe (containment). In the end, what lasts longest will probably not be any particular cost-saving technique, but the habit of measuring and controlling one’s own system.

This is a rewrite of a retrospective I wrote in June 2026. The original post's headline claim, "seven pickers behind one API, smaller than a single calendar from a competing library," turned out on re-examination to be half wrong and half not a differentiator. So I am reorganizing it in three parts: why I built it, how it differs from existing options, and a technical definition.

To give the conclusion first, what sets Kalyx apart is not the number of components but its value model. The reference versions are @kalyx/react 1.4.7 and @kalyx/core 1.4.8 (MIT, React 19 only), and code excerpts come from the GitHub repository at main as of 2026-09-16 (0bb302e).

I wanted to use date pickers declaratively

I had two reasons for building it. I wanted to use complex, hard-to-use date libraries in a more declarative and simple way, and I wanted to learn how such a library is built on the inside. "Hard to use" is vague, so I went back to each library's type definitions to check whether the places I got stuck are still there in the latest versions.

An API that switches modes with props

react-datepicker turns on time selection with showTimeSelect, month selection with showMonthYearPicker, year selection with showYearPicker, and range selection with selectsRange. One component becomes a different thing depending on its prop combination. The cost shows up in the types. In the 9.1.0 type definitions, depending on the value of selectsRange, the onChange signature forks.

// react-datepicker/dist/index.d.ts (발췌)
    selectsRange?: true;
    selectsMultiple?: false | undefined;
    formatMultipleDates?: never;
    onChange?: (date: [Date | null, Date | null], event?: React.MouseEvent<HTMLElement> | React.KeyboardEvent<HTMLElement>) => void;

It is a well-crafted union, but branches multiply as modes are added, and the consumer has to reconstruct which combination is active just from prop names. (This has the same shape as "an abstraction that bundles the wrong things increases coupling," which I covered in my post on abstraction.) I wanted "an input, a popover, and a calendar" to be readable from the JSX structure itself.

Where value types and timezones leak

react-datepicker and react-day-picker pass native Date objects around. Both do have an IANA timeZone prop. react-datepicker requires the optional peer date-fns-tz and react-day-picker's is experimental, but either way the value type is still Date. Because a Date is interpreted in the runtime's local timezone, in Seoul, new Date(2026, 3, 15) run through toISOString() gives 2026-04-14T15:00:00.000Z. You pick April 15, and the server sees the 14th. react-datepicker's "Date Selected is One Day Off" issue was opened in September 2017 and closed in December 2025.

On the other side, Ark UI and React Aria use @internationalized/date objects such as CalendarDate and ZonedDateTime. The semantics are precise, but in an app where form state and server responses are all strings, conversion code appears at every boundary.

Timezone support can also be tied to your choice of date library. Looking at the adapter code in MUI X Date Pickers 9.13.0, the dayjs, Luxon, and Moment adapters have isTimezoneCompatible = true, while the date-fns family has false. An app on date-fns has to bring in a second date library to use the timezone prop.

Modes were scattered across prop combinations, values across Date objects whose interpretation depends on the runtime, and timezone support across the choice of date library. It was hard to express intent in a single declaration.

Learning by building

A date picker looks small, but it contains calendar arithmetic, locales, timezones and DST, keyboard navigation, and SSR. So I set two goals: on the consumer side, you should be able to read what is being built just from the JSX; on the implementation side, the code should be narrow enough that I can explain where values get converted.

So was there really no library that had already solved these needs?

How it differs from existing options

The short answer is that there was. I started out believing that no library was both headless and shipped multiple pickers, but when I researched again, that premise was wrong.

The value model each option chose

The following was verified on 2026-09-16 by installing the latest version of each package from npm and checking its type definitions and official documentation. The size column was measured with the same method described below.

react-day-picker states in its official guide that "DayPicker does not include a built-in time picker." The MUI size includes @mui/material and emotion, so if you already have a MUI app, the increase is much smaller.

Complete headless options already exist

The rows that matter in the table are Ark UI and React Aria. Ark UI offers single, multiple, and range selection plus month- and year-level selection through a dot notation composition API such as DatePicker.Root. React Aria is Adobe's accessibility-focused implementation. So Kalyx's niche is narrower than I first thought.

Complete headless options already exist. However, Ark UI and React Aria exchange values as @internationalized/date objects and offer time input as segmented fields. Kalyx fixes values as UTC instant strings that go straight into JSON, and puts a list-based TimePicker along with month, year, and week pickers into the same composition API.

Still, "values are strings, which is nice" is a weak claim. A CalendarDate becomes a string with a single toString(). The differentiator is not the format but the contract that the string is always a real moment (an instant) and never mixed up with a calendar cell (a coordinate), and the tests that keep that contract.

Re-measuring bundle size with one method

The bundle size on the README badge was not a quantity you could compare with other libraries. The badge's roughly 19.5KB is a single file in @kalyx/react's dist, and that file leaves @kalyx/core, @kalyx/adapter-date-fns, and @floating-ui/react as external imports. I had placed that number next to other libraries' numbers that included their dependencies.

So I re-measured everything as "how much does the bundle grow when a consumer app adds one import line?"

echo "import { DatePicker } from '@kalyx/react'; export default DatePicker;" > entry.jsx
npx esbuild entry.jsx --bundle --minify --format=esm --platform=browser \
  --external:react --external:react-dom --external:react/jsx-runtime | gzip -6 | wc -c

Measured on 2026-09-16 with esbuild 0.28.2, and KB here means bytes divided by 1024. For react-datepicker I excluded the CSS. For React Aria Components I measured twice: with the 12 exports needed to assemble a single DatePicker (DatePicker, DateInput, Calendar, Popover, Dialog, and so on), and with 14 exports that add range and time (including DateRangePicker, RangeCalendar, and TimeField).

Under the same conditions, one statement holds. One DatePicker (18.9KB) is about the same size as DayPicker (20.0KB), and all seven together (25.6KB) are larger. (Even then, DayPicker is just a calendar, while Kalyx DatePicker also includes the input and popover.) Size is sensitive to the import combination and gzip level, so it is better read as an order of magnitude, and size is not the main reason to choose Kalyx.

Defining Kalyx technically

Kalyx is a headless React date picker that fixes all input and output as UTC instant strings, confines conversion between calendar coordinates and instants to two functions, and checks that round trip with property-based tests across every timezone the runtime knows.

Here is the consumer-facing API. value and onChange use string | null, and displayTimezone decides which zone's calendar is shown.

<DatePicker value={iso} onChange={setIso} displayTimezone="America/New_York">
  <DatePicker.Input />
  <DatePicker.Popover>
    <DatePicker.Calendar />
  </DatePicker.Popover>
</DatePicker>

Values are instants, calendar cells are coordinates

Two kinds of ISO strings with the same format flow through Kalyx. A coordinate is a single cell of the calendar grid, written as YYYY-MM-DDT00:00:00.000Z but carrying no notion of timezone. Grid computation runs only in UTC, so it is independent of the runtime environment. An instant is the value that leaves through onChange: the actual moment that is midnight of that day in displayTimezone. The same January 15 is 2026-01-15T05:00:00.000Z in New York and 2026-01-14T15:00:00.000Z in Seoul.

Only two functions in @kalyx/core connect them: civilMidnightFromUtcDay, which turns a coordinate into an instant, and its inverse, calendarDayFromInstant. The latter reads "what day is this instant in this zone?" and writes it back as a UTC-midnight coordinate.

// packages/core/src/utils/timezone.ts:187-190
export function calendarDayFromInstant(iso: ISODateString, timeZone: string): ISODateString {
  const p = partsInTimezone(new Date(iso), timeZone);
  return new Date(Date.UTC(p.year, p.month - 1, p.day)).toISOString();
}

The rule is about direction. When a cell is picked and the value is committed, a coordinate becomes an instant; when the stored value determines which month to show and where focus goes, an instant becomes a coordinate. The DatePicker Root's selectDate converts, checks constraints against the converted instant, and commits if it passes.

// packages/react/src/components/DatePicker/Root.tsx:182-194
const normalized =
  coordinate && displayTimezone
    ? civilMidnightFromUtcDay(coordinate, displayTimezone)
    : coordinate;
 
if (normalized && isDateDisabled(normalized, disabledRules, adapter, displayTimezone)) {
  return;
}
 
if (!isControlled) {
  setUncontrolledValue(normalized);
}
onChange?.(normalized);

This is not the only call site. Searching packages/react/src shows the two functions scattered across the Root and Calendar of DatePicker, RangePicker, and DateTimePicker, the Presets, the keyboard navigation utilities, and six headless hooks. Even so, every path that commits a date cell or decides the view goes through one of the two. Committing a time is handled by setTimeInTimezone, which converts internally through the same internal function, resolveCivilDateTime.

This contract is guarded by property-based tests. For every coordinate c and zone z, calendarDayFromInstant(civilMidnightFromUtcDay(c, z), z) === c must hold.

// packages/core/src/__tests__/timezone.property.test.ts:86-94
it('round-trips every UTC calendar coordinate through civil midnight', () => {
  fc.assert(
    fc.property(utcCalendarCoordinate(), zone(), (coordinate, timeZone) => {
      const instant = civilMidnightFromUtcDay(coordinate, timeZone);
      expect(calendarDayFromInstant(instant, timeZone)).toBe(coordinate);
    }),
    RUNS,
  );
});

fast-check randomly generates dates between 2020 and 2045 and combines them with 14 representative zones, including +5:45 Kathmandu, +14 Kiritimati, and -11 Niue, for 300 runs. The very next test checks the same round trip 12 times per zone for every zone returned by Intl.supportedValuesOf('timeZone'). On my local Node 24.16, that list has 418 entries. 1.4.8 added an exhaustive test that compares conversions at the boundary times of every DST transition against a Temporal implementation.

Then why not have the library normalize a user-supplied "2026-01-15T00:00:00.000Z" into an instant on its own? That road is closed because you cannot tell from the string alone whether it is a coordinate or an instant. Take the Seoul value 2026-01-14T15:00:00.000Z, which is already an instant, apply civilMidnightFromUtcDay to it again, and you get 2026-01-13T15:00:00.000Z; in positive-offset zones like Seoul, the date keeps slipping by a day on every render. (The New York value stays the same when applied again.) Switching to startOfDayInTimezone, which gives the same result no matter how many times it is applied, removes the slipping but can no longer do the original job of turning a coordinate into an instant.

So I gave up on normalization and fixed the contract in documentation. Consumers must pass back exactly the value the picker emitted. Honestly, that is a cost pushed onto consumers, and because ISODateString is an alias of string, the compiler cannot stop mistakes either.

Solving DST with Intl alone

To turn a coordinate into an instant, you need to know when "00:00 on that day in that zone" is in UTC, but you can only get the offset once you know the instant. On top of that, on DST transition days a local time may not exist (spring forward) or may occur twice (fall back).

Kalyx solves this without a library like date-fns-tz. It asks Intl.DateTimeFormat(...).formatToParts "what local time is this UTC moment in that zone?" to measure the offset, and reads that offset at two points: one day before and one day after the requested time.

// packages/core/src/utils/timezone.ts:251-254, 259
const candidate = (probeEpoch: number) =>
  civilEpoch - getTimezoneOffsetMinutes(new Date(probeEpoch).toISOString(), timeZone) * 60_000;
const epochBefore = candidate(civilEpoch - 86_400_000);
const epochAfter = candidate(civilEpoch + 86_400_000);
 
if (epochBefore === epochAfter) return new Date(epochBefore).toISOString();

civilEpoch is the desired local time read as if it were UTC. Offsets stay within ±14 hours, so one day before and one day after give the offsets on either side of any nearby transition. If the two agree there is no transition, and the work ends with two formatToParts calls; since the grid calls this once for each of its 42 cells, this fast path determines the cost. (It assumes at most one transition within 48 hours, which holds in every zone from 2020 to 2045.)

If they differ, it re-reads both candidates in that zone and checks which matches the requested time (timezone.ts:261-279). In a fall back overlap both match, so it picks the earlier one built from the pre-transition offset; in a spring forward gap neither matches, so it picks the same side and moves the time forward by the length of the gap. Here is what it actually returns.

"Gaps move forward, ambiguity takes the earlier one" matches the disambiguation default "compatible" described in MDN's Temporal.ZonedDateTime documentation. Why the 1.4.7 London row is off is covered later.

The adapter boundary is 21 string methods

If core computes timezones, what do date-fns or dayjs handle? Date arithmetic and parsing. That boundary, DateAdapter, has 21 methods, and every date argument and return value is an ISO string.

// packages/core/src/types.ts:71-102 (발췌)
export interface DateAdapter {
  parse(value: string, format?: string): string;
  format(iso: string, formatStr: string, timezone?: string): string;
  addDays(iso: string, n: number): string;
  isSameDay(a: string, b: string, timezone?: string): boolean;
  startOfDay(iso: string, timezone?: string): string;
  today(timezone?: string): string;
  // addMonths, isBefore, startOfMonth, getYear 등 15개
}

Four methods accept a timezone, format, isSameDay, startOfDay, and today, and even those four hand the computation to core. The date-fns adapter's format, when given a timezone, calls core's formatInTimezone directly (packages/adapter-date-fns/src/index.ts:112-115). So whichever adapter you use, timezone answers come from the same code, and the three adapters (date-fns, dayjs, luxon) each run @kalyx/core/test-helpers's runAdapterConformanceTests to confirm they produce the same answers.

The two entry points also split along this boundary. The default entry, @kalyx/react, calls setDefaultAdapter(DateFnsAdapter) at module load, so it works right after installation (packages/react/src/index.ts:9-11). @kalyx/react/headless has no such call and is bundled separately, so no date-fns code gets in.

This boundary shape came at a price. In June 2026, I shelved a Temporal adapter. Temporal's value lies in types carrying meaning, as with PlainDate and ZonedDateTime, but when the boundary is a string, that meaning cannot pass through. Wrapping it would only flatten it into a string, so I judged there was no correctness gain. Looking back, the problem from the previous section, "coordinates and instants are the same string," is exactly what Temporal solves in the type system with PlainDate and Instant. The string boundary made swapping adapters easy, but in exchange it closed off the path to solving this with types.

Seven pickers are combinations of three contexts

Kalyx ships seven pickers: DatePicker, RangePicker, TimePicker, DateTimePicker, MonthPicker, YearPicker, and WeekPicker. What matters is not the count but the fact that the seven are not independent implementations. There are only three contexts: DatePickerContext, RangePickerContext, and TimePickerContext.

MonthPicker's Root merely changes the default display format to yyyy-MM and passes selectionGranularity="month" to DatePicker's Root implementation (MonthPicker/Root.tsx:11-20). DatePicker's selectDate, when the granularity is month, folds the coordinate to the 1st of that month and then goes through the same conversion and constraint check shown earlier. DateTimePicker provides the two contexts stacked.

// packages/react/src/components/DateTimePicker/Root.tsx:401-402
<DatePickerContext.Provider value={dateContext}>
  <TimePickerContext.Provider value={timeContext}>{children}</TimePickerContext.Provider>

So DateTimePicker.Calendar is the same component as DatePicker.Calendar, and it does not know which picker it is inside. A fix in one place reaches every picker that uses the same component, and so does a defect in one place. This shared foundation is also why a single DatePicker came to 74% of the whole in the earlier chart.

What keeping the contract cost

I spent the three months since 1.0 verifying this contract more than building new features, and it surfaced three defects that example-based tests would have let through. The first was found by a property test, the second by a cross-review of the code, and the third by fact-checking this post.

One hour on October 1 in Sydney

The first was that the property "reading the result of startOfDayInTimezone in that zone gives 00:00:00" broke in Australia/Sydney. The implementation at the time measured the offset only once. Sydney moves from +10 to +11 at 02:00 on October 1, 2034, and 00:00 that day is still +10, so the correct answer is 2034-09-30T14:00:00.000Z. But "the point where 00:00 on October 1 is read as UTC" is after the transition, so it returned +11, and the result was 23:00 on the previous day. This counterexample remains as a regression test (timezone.property.test.ts:276).

A test that passed only in Seoul

The second came out of a cross-review on August 3, 2026. The code that decides which calendar cell to mark as selected was converting twice. The cell was already a coordinate, yet both the cell and the stored value were converted to that zone's date. Here is what picking "January 15" produced.

In positive-offset zones the two shifts cancelled out and it happened to be right, and the existing tests covered only Seoul. The same review also found a violation in the opposite direction. When deciding which month to show, the code applied startOfMonth directly to an instant, so passing January 1 as the value in Seoul opened the December calendar.

The mechanisms differ, but both broke the same rule: convert only once per direction, with the function assigned to that direction.

This incident widened core's round-trip property tests from "a few representative zones" to "every zone the runtime knows." (Component tests on the React side still use representative zones such as America/New_York.) Defects that only appear where the sign flips are not caught by sampling.

London's 01:30 resolved late

The third came up while I extended this post's DST table to other zones. 1.4.7 started measuring the offset at the point where the requested time is read as UTC, and in zones whose post-transition offset is 0 or more, that point is already past the transition, so it converged on the later offset. Checking every transition from 2020 to 2045 in 418 zones against temporal-polyfill, 1,908 of the 3,395 overlap transitions resolved to the later instant, across 84 zones. (All 3,394 gap transitions were correct, and New York, with its negative offset, just happened to land on the right side.)

In #226 I changed it to read one day before and after, released it as @kalyx/core 1.4.8, and kept the same comparison as timezone.dst-oracle.test.ts. (temporal-polyfill is used only in tests.) The source comment that wrongly said 'earlier' now says 'compatible'. Once again, the samples were concentrated on one sign.

The 3KB spent on correctness

These fixes cost code. Kalyx sets a CI ceiling on the default entry bundle, and a PR that exceeds it fails a required check. That ceiling started at 12KB and went up 1KB each time a feature landed, and in August 2026, while overhauling timezone and constraint correctness, I raised it from 17KB to 20KB in one step.

Size was a selling point shown on the README badge. A date picker that slips by a day in negative-offset zones is unusable whether it is small or large.

If I have to choose between small and correct, I choose correct. The ceiling is tight now. According to the repository's bundle byte map document from 2026-09-11, dist/index.cjs measured with Node's default gzip is 20,259B against a ceiling of 20,480B, leaving 221B of headroom. (This is the size of its own file with dependencies left external, so it is a different quantity from the earlier chart.) The next feature will have to reclaim bytes before it can land.

Not the conclusion I started with

What I wanted was to use complex date libraries declaratively. After building Kalyx, I found that declarative composition APIs and complete headless options already existed. The remaining difference lay deeper. Fix the value as a single instant, narrow conversion between coordinates and instants to two functions, and guard that round trip with tests in every timezone. The Intl-based DST handling, the string adapter boundary, and the seven pickers built from three contexts are the consequences of that decision.

Measured against my goal of learning, the biggest lesson was how to claim that something is "correct." The original post's bundle comparison was measuring different quantities, and the test that passed in Seoul was wrong in New York. Unless you also record what you measured and which samples you checked against, numbers easily turn into bragging.

The limits are clear too. I am the only maintainer, it supports React 19 only, and npm downloads of @kalyx/react from September 5 to 11, 2026 were 200, of which 156 were concentrated on the single day 1.4.7 was released. The inability to separate coordinates and instants in the type system is guarded only by documentation, and I have not decided whether to guarantee the styling hooks, classNames and data-* attributes, as public API. For accessibility, it has the grid role on calendars, combobox on inputs, and listbox on time lists, with arrow key, Home/End, and PageUp/PageDown navigation, and CI runs the jest-axe checks in 8 test files, but I have no basis yet to say it is as proven as React Aria.

So if you already have a MUI app, look at MUI X first; if segmented input and proven accessibility come first, React Aria; if you only need a calendar, react-day-picker. If you have run into dates slipping by a day in forms where values travel as JSON, that is when I would be glad if you took a look at Kalyx, and if you know a better solution, I would be grateful if you let me know through a GitHub Issue.

pnpm add @kalyx/react

On the docs site's Playground, you can try the seven pickers and change locale and timezone settings yourself.

In the early days, I focused on results and process rather than performance and cost. AI-generated output had plenty of gaps, so it needed verification, and the pressure to produce results quickly led many people, myself included, to pay for more tokens or move to a higher subscription tier whenever they ran short. I did the same. (For the first few months, I did not even pay attention to how much tokens were costing me.)

After a while, however, I became increasingly conscious of token usage. Individuals felt the burden of monthly subscriptions, while companies grew more concerned about labor and operating costs. MCP and Function Calling covered the protocol that connects agents to external systems, Four Tiers of Code Intelligence covered code search tools and GitHub Trending, and Context Files covered the rule files that agents read. In this way, my articles so far have focused more on which tools exist and how to use them well. I still think those topics matter, but as time passes, cost will ultimately become the question people care about most.

I covered the mechanics of tokens—what exactly they are, how BPE creates them, and what happens inside a transformer when prompt caching lowers the unit price—in a separate article, How Tokens Work. Building on that foundation, this article first examines how costs are billed and where inefficiencies arise, then organizes proven cost-saving patterns. For the main patterns, I attached token usage I measured myself by running the same task under different strategies.

Where Do Token Costs Come From?

Let us take a quick, focused look at how token costs arise and how providers bill them.

Every piece of text we send—the system prompt, tool definitions, conversation history, and user messages—becomes input tokens, while the model's response becomes output tokens. To the model, every call is a new input it has never seen before. Whether the conversation happened yesterday or the previous call was one minute ago, a new call sends the same content again in full as input. (This simple fact is the heart of token costs: the model has no memory, so we tell it everything again each time.)

One more variable enters the picture. Resending the static portion of a repeated call from scratch is expensive, so major LLM providers introduced prompt caching. Static input is stored in a cache once, and on subsequent calls, tokens read from that cache are billed at a much lower rate. (I covered the underlying mechanics—how BPE creates tokens and how the cache reuses the transformer's KV cache to lower the unit price—in How Tokens Work. Here, the focus is on how those mechanics translate into cost.)

Input Tokens, Output Tokens, and Cache Tokens

Let us look at four fields in the usage object returned by the Anthropic SDK.

• input_tokens the portion of the submitted input excluding cache reads
• output_tokens the response generated by the model
• cache_creation_input_tokens tokens stored in the cache for the first time on this call
• cache_read_input_tokens tokens read again from an existing cache

Each of these four fields is multiplied by a different rate. Output tokens are the most expensive, while tokens read from cache are the cheapest. According to Anthropic's official documentation, cache reads cost 0.1 times the base input rate—exactly 10%. Cache writes cost 1.25 times the base rate with a five-minute TTL (Time To Live, or cache validity period) and 2 times the base rate with a one-hour TTL. In other words, you pay slightly more for the first call, then save 90% from the second call onward.

Caching also has several less widely known constraints. The minimum cacheable token count varies by model and even by version within the same family. According to Anthropic's official documentation, Sonnet 4.6 and Opus 4.8 require 1,024 tokens, Opus 4.7 requires 2,048, and Haiku 4.5 plus the older Opus 4.5/4.6 require 4,096. A shorter prompt quietly goes uncached even if you add cache_control. A request can contain at most four cache_control breakpoints, and the cache is read hierarchically in the order tools → system → messages. As a result, changing even one tool definition near the front invalidates the entire cache that follows it.

The 90% discount comes from reusing a transformer's KV cache across calls, a mechanism covered in detail in How Tokens Work. From a cost perspective, there is one key point to remember: a cache hit occurs only when the prefix matches exactly. Therefore, put static content first and dynamic content that changes on every call last so the cache stays intact. Even a single timestamp character near the beginning of a prompt invalidates the entire cache after it.

This naturally raises a question: “But the input changes on every turn in a multi-turn conversation. Wouldn't that break the cache almost every time?” The short answer is no. A conversation's input is not rewritten as a fresh block on each turn; it uses an append structure that preserves everything accumulated at the front and adds only the new utterance at the end. Once the system prompt, tool definitions, and previous questions and answers are fixed, they remain unchanged, and only the latest question is appended. Compare the current turn's input with the previous turn from the beginning, and the first difference will always be the new question at the very end. The full input changes every time, but most of the prefix stays the same, so the cache survives. (The cache breaks on every turn only in a flawed design that inserts dynamic values near the front of the prompt. Tools such as Claude Code and opencode are designed to keep the front fixed and append only at the end, so users do not need to configure anything separately for caching. Typos and spelling in the user's input do not affect the cache for the same reason: that variation always occurs at the very end, outside the cacheable prefix.)

Comparing AI Providers

Putting the official Anthropic, OpenAI, and Google price lists side by side as of June 2026 makes the pattern immediately clear. (Prices are in USD per million tokens, focusing on models commonly used in everyday coding workflows.)

For all three providers, output tokens cost five to six times as much as input tokens. That means longer output quickly drives up the cost, even when the answer is substantively the same. Within the same tier, model prices also differ by a factor of three to six. The arithmetic alone points to three major principles for reducing token costs: request shorter output, use a cheaper model when it can produce the same answer, and cache static input. (One more calculation is revealing. Suppose you reuse a static context of 10,000 tokens 100 times. At GPT-5.5's base rate, that costs $5; as cache-read tokens, it costs $0.50. The cache therefore saves $4.50 across 100 reuses of the same context, and because the first call has almost no cache-write surcharge, the investment pays off from the second call.)

How Cache Mechanisms Differ

Even when the discount rates are similar, the internal design differs by provider. Understanding why those differences exist makes policy decisions easier.

The design philosophy of each provider is easy to see. Anthropic asks users to mark cacheable sections explicitly, charges a modest premium on the first call (1.25 times), and then applies a deep discount. Because the prefix position is easy to control, the cache-read rate is predictable. OpenAI takes the opposite approach and automates everything. Inputs of at least 1,024 tokens are cached automatically at no additional cost, but users have less control over cache behavior. Google offers both approaches and charges a separate storage fee when users explicitly manage the cache. Implicit caching is advantageous for short, frequent reuse, while explicit caching plus storage fees suits large contexts that must remain available for an hour or more.

Tool Definitions and Tokenizers

Two more variables have a surprisingly large impact.

The first is tool definitions. In an environment with several MCP servers attached, every call includes every tool name and parameter schema in the input—we will measure how much this inflates costs later. It is also worth noting that even with the exact same tools, changing only the model changes the cost. Anthropic's official pricing documentation shows that the tool system prompt itself differs in length by model. At tool_choice: auto, Sonnet 4.6 and Haiku 4.5 use about 497 tokens, Opus 4.7 uses 675 in the same position, and Opus 4.8 drops back to 290. Before model routing, checking how verbosely the current model expands tool definitions can reveal a surprisingly large difference.

The second is the efficiency of the tokenizer, the tool that breaks human-written text into token units the model can process. As discussed in How Tokens Work, the same text can produce different token counts depending on the tokenizer. OpenAI's o200k_base uses far fewer tokens than cl100k_base for non-English text, and Anthropic states that after switching to a new tokenizer beginning with Opus 4.7, the same text may be billed as up to 35% more tokens than with previous models. Choosing a model based only on price can allow costs to leak back in through tokenizer inefficiency. An honest comparison multiplies the unit price by the expected token count.

The difference is especially pronounced in Korean. When I tokenized the same sentences with two OpenAI tokenizers, the old and new versions produced identical token counts for English technical sentences, but the older cl100k_base used 31–43% more tokens for Korean than the newer o200k_base. One Korean paragraph was especially striking: 167 characters became 169 tokens with the older tokenizer, producing more tokens than characters. On average, each Korean character consumed more than one token.

The older tokenizer splits Korean more aggressively at the byte level, while the newer one treats frequently used chunks such as “개발” and “입니다” as complete tokens. That means even at the same unit price, a Korean-heavy workload can cost more than 1.5 times as much solely because of tokenizer efficiency.

At this point, another question arises naturally: “Where exactly are we creating all this inefficiency?”

Common Ways Tokens Get Wasted

Token waste is surprisingly common even when we believe we are using AI well. Patterns I have observed in my own work and among other developers fall into several groups.

Too Many MCP Servers and Tool Definitions

I mentioned this in the previous section, but it bears repeating. Once people attach MCP servers such as Linear, GitHub, Notion, Figma, Slack, and Sentry, they rarely remove them. Unused tool schemas inflate the input token count on every call. To address this, Claude Code enables MCP Tool Search by default: only tool names and server descriptions enter the context at the beginning of a session, and the full schema is loaded only when the model actually calls that tool.

I measured how large the difference could be. Using 27 MCP tools from a Claude Code session I was working in—10 from Serena, eight across four claude.ai OAuth integrations, two from Figma, and seven from agentmemory—I sent the same user message under two configurations. One had no MCP servers installed. The other had all 27 attached but unavailable for the model to call. In both cases, the number of tool calls was zero.

With the same question, the same model, and an answer with the same meaning, input tokens alone rose from 41 → 10,335 (+10,294) on Opus 4.7. The cost of a single call increased from $0.0048 → $0.0563, about 12x. The 250-fold increase in input also raised prefill load, adding +783ms to response time. More striking than the dollar amount was that this was a cost paid on every call even though the user did not invoke a single MCP tool on that turn. This is the overhead that Tool Search prevents. (Since taking this measurement, I have continuously pruned MCP servers I do not use regularly.)

Context Accumulation and Lost in the Middle

Carrying a long conversation forward does more than increase input tokens per call; it also lowers the model's accuracy. The “Lost in the Middle” paper by Liu's Stanford research team quantified a U-shaped curve: models retrieve key information best when it appears at the beginning or end of the context, and perform noticeably worse when it is buried in the middle. It is the worst possible combination—spending more tokens for a poorer answer. Because a transformer's self-attention computation grows with the square of the token count, the absolute share of attention available to each token becomes more diluted as context length increases. The middle weakens first because training distributions tend to place important information near the beginning and end.

Looking one level deeper, the model's treatment of token “position” contains two tendencies that bury the middle. They may sound technical, but the intuition is straightforward.

First, the model listens more closely to nearby tokens. RoPE (Rotary Position Embedding), the positional representation used by most current open models, is designed so that two tokens connect more weakly as the distance between them increases—a decay effect in which the signal fades with distance. Distant tokens therefore receive less attention naturally.

Second, the model sends an unnecessary amount of attention to the very first token. At every moment, the model must distribute a full 100% of its attention—the softmax forces attention weights to sum to 1. When there is nothing in particular to focus on, the remaining attention must be discarded somewhere, and that destination is usually the first token in the sequence. This phenomenon, in which the first token absorbs surplus attention, is called an attention sink. (Xiao's MIT research team first quantified this structure in the StreamingLLM paper. It is less a sign that the first token contains important information than a byproduct of using it as a drain for spare attention.)

Together, these two tendencies concentrate attention at both ends—the nearby recent tokens and the first token—while information buried in the middle receives the weakest treatment. Importantly, this is not a bug in one particular model. Most open models, including LLaMA, Mistral, and Qwen, use RoPE-family mechanisms, and proprietary models such as Claude and GPT are believed to use similar approaches. Lost in the middle is therefore closer to a shared bias in modern transformer architecture.

More recently, this phenomenon has been called context rot. An analysis by the Chroma research team put 18 frontier models—including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3—through the same NIAH (needle in a haystack) task. It quantified accuracy declines of 20–50%, depending on the model, when the input grew from 10k to more than 100k tokens. All 18 models performed worse as context length increased, with the Claude family declining most gradually. Anthropic also explains this as an “attention budget” problem arising from the transformer's n² attention, with the budget being depleted across tokens. Keeping context light therefore reduces costs and protects accuracy at the same time.

Calling Subagents Indiscriminately

Delegating every task just because subagents are useful is another trap. A subagent starts in a separate context from its parent, so it pays the fixed cost of loading the system prompt and tool definitions from scratch. If you delegate a lightweight task such as a short shell command or a simple git lookup, that startup cost can exceed any savings from keeping the main context clean. According to Anthropic's published report on its multi-agent research system, a single agent uses roughly four times as many tokens as ordinary chat, while a multi-agent system uses roughly 15 times as many. Delegation makes sense only when the accuracy gain justifies that four- to fifteen-fold overhead.

The same trap applies to attaching MCP tools. Every enabled tool adds its schema to the system prompt on every turn, and the cost is billed even on turns when the model does not use the tool. “Enable everything just in case” may feel intuitively safer, but actual measurements tell a different story.

I tested both accuracy and cost on the same batch of questions about the reconciler source in facebook/react v19, divided into three setups:

• No tools
• One tool (CodeGraph, Serena, ripgrep, or bare grep)
• All four tools attached at once

Three findings stood out. (Here, recall means how completely the correct answers were found.)

• No tools achieved an average recall of only 0.31, confirming that tools themselves clearly provide value.
• Serena (LSP) alone achieved 1.00 recall at a cost of $0.38, making it the most efficient of all single-tool strategies.
• All four tools attached reduced recall to 0.89 while raising the cost to $0.47. On multi-hop questions in particular, the all-tools score matched CodeGraph alone (0.78 / 0.88) to the second decimal place. The model gravitated toward one of the four tools and inherited that tool's weaknesses.

One principle applies equally to subagents and tool attachment: the extra cost matters only when an accuracy improvement justifies it. Choosing the one tool that fits the task domain is cheaper and more accurate than enabling everything just in case.

So what are the proven ways to save tokens? Do effective methods really exist?

Proven Ways to Save Tokens

Each savings pattern attacks a different cost axis. Some reduce input, some lower the unit price of the same input, and some assign the same work to a cheaper model. Let us examine them one by one.

Prompt Caching

This produces the most immediate effect. As discussed above, cache reads cost 0.1 times the base input rate. After paying a small write premium of 1.25 times on the first call, you can reuse the same static section from the second call onward at one-tenth the price.

From a cost-saving perspective, one point deserves emphasis: under any caching approach, the parts that remain nearly identical across calls—tool definitions, code snippets, and RAG context—are exactly what the cache can capture. The benefit remains intact as long as those static blocks are not mixed with dynamic content such as the current turn's question or a tool result that just arrived. If you call the API directly, grouping static blocks at the front and moving dynamic parts behind them delivers most of the savings. If you use a finished tool such as Claude Code or opencode, the tool handles that ordering for you.

Batch Asynchronous Work with the Batch API

When a call does not need to finish immediately, the Batch API is the simplest way to lower the rate itself. Anthropic, OpenAI, and Google all apply the same 50% discount to both input and output tokens in exchange for returning results within 24 hours. With Anthropic, most batches finish within an hour in practice, but the SLA itself is 24 hours. The calls remain unchanged while the unit price is cut in half, so the implementation cost is minimal.

What makes that 50% especially interesting is that it compounds multiplicatively with other savings. Anthropic's official documentation explicitly states that caching and batch discounts stack. For static input, the calculation is standard rate × 0.5 (batch) × 0.1 (cache read) = 0.05 times, meaning the static portion can fall to 5% of the standard rate. The key is that the discounts multiply rather than add.

I calculated the difference directly. Suppose 100 jobs each contain 10,000 tokens of static context, 500 tokens of dynamic input, and 1,000 output tokens. Using Opus 4.8 pricing, I compared the cost of four strategies.

Caching alone cuts the cost by 57%, batching alone by 50%, and using both by as much as 79%. The discounts multiply over the static-input portion rather than simply adding together. (Output tokens are not cacheable and receive only the 50% batch discount, so workloads with a high share of output see a total reduction smaller than 79%. The larger the static-input share, the stronger the compounding effect.) There are more areas than one might expect where nobody needs to wait in front of a screen: overnight codebase indexing, pre-publication article evaluation, data extraction, and recurring reports.

Not every task can be deferred to asynchronous processing, however. Batch does not fit work where response time itself creates value, such as the main session of a coding agent, where the user reads an answer before choosing the next action, or an interactive chat UI. Simply dividing work into two lanes—“results needed immediately” and “results needed by the next workday”—can cut the bill in half.

Isolate Verbose Work with a Subagent

To use the wording from Claude Code's official documentation, a subagent “operates in its own isolated context window, with intermediate tool calls and results remaining inside the subagent and only the final message returning to the parent.” Delegating an entire verbose task therefore leaves only a clean summary in the parent context. According to Anthropic's article on context engineering, even when a subagent spends tens of thousands of tokens exploring, it typically returns a compressed summary of only 1,000–2,000 tokens to the parent. The benefit of keeping the parent context light is clear.

A verbose task is one that produces a vast number of tokens in the process of finding a one-line answer. Running tests, searching documentation, and analyzing logs are typical examples.

One misconception needs clarification: a subagent does not automatically reduce total cost. Anthropic's own report says agents use about four times as many tokens as ordinary chat and multi-agent systems about 15 times as many. A subagent protects accuracy and the cost of an expensive long-running parent context by removing verbose output from it; it is not magic that always reduces total token expenditure. That is why the earlier principle still applies: delegate only when the cleanup savings in the main context exceed the startup cost. For short tasks, it is cheaper for the parent to handle them directly.

There is also a domain consideration. Anthropic's multi-agent research system report says that in its internal evaluation, an Opus 4 leader with Sonnet 4 subagents improved performance by 90.2% over a single Opus 4 agent. That improvement, however, is not uniform across domains. In the same report, Anthropic explicitly says that domains where agents must share the same context or have many interdependencies are poor fits for multi-agent systems—and identifies coding as exactly such a case. Research can explore independent directions in parallel, but code operates on a dependency graph where changing one function affects another. (If you use a multi-agent setup for coding, it is worth asking whether the workflow truly resembles parallel exploration. A single agent plus one isolated exploration subagent may be a safer default for the coding domain.)

compact and Progressive Disclosure

Claude Code's /compact command compresses the entire conversation so far into a summary and restarts with a fresh context. According to Anthropic's official explanation, compaction is not simple truncation but semantic summarization: it retains the context of ongoing work and recent changes while discarding material unlikely to be referenced again, such as repetitive tool output. Unlike /clear, which removes everything, /compact preserves a summary. When the context window approaches 95% full, auto-compact performs the same operation automatically. Tidying a long session at an appropriate point stops input tokens from accumulating.

Looking further inside, /compact is only the final stage a user can invoke explicitly; before it, a four-stage context-compression pipeline runs automatically. According to an external study of Claude Code's internals, “Dive into Claude Code,” query.ts checks the following five stages in order before every call.

• Budget Reduction truncates portions of individual tool outputs that exceed their size limit.
• Snip cuts off older history along the time axis.
• Microcompact performs fine-grained compression while preserving cache awareness.
• Context Collapse projects extremely long history again at read time to reduce its dimensionality.
• Auto-Compact triggers semantic compression at the 95% mark as a last resort.

The higher stages are lighter and cheaper; the lower ones are heavier but more effective. This five-stage design reflects the recognition that no single compression strategy can resolve every kind of context pressure. Explicitly invoking /compact is roughly equivalent to triggering the final stage of this automatic pipeline early.

The same way of thinking appears in Claude Code's Skills architecture. Where /compact reduces context after it has accumulated, the three-stage loading used by /skills prevents that context from accumulating in the first place.

According to Anthropic's documentation, a skill loads in three stages. Only its name and one-line description—about 100 tokens—enter the context at session start. Its body (SKILL.md, fewer than 5,000 tokens) loads only when the skill is triggered. When bundled scripts or resources are executed through bash, only their output returns; the code itself never enters the context. Installing dozens of skills therefore adds almost nothing to the initial context.

Model Routing: Use a Cheaper Model for the Same Answer

The input price of Opus 4.8 is five times that of Haiku 4.5. Using the largest model for every simple search, exploration, or short summary is a major waste of money. Routing by task difficulty—Haiku → Sonnet → Opus—and calling Opus only for genuinely reasoning-heavy stages is becoming standard practice. LMSYS's RouteLLM research demonstrated a router that preserved 95% of GPT-4 quality while reducing strong-model calls to 14%, though the benchmark measured general reasoning rather than coding specifically.

The tooling landscape has also settled quickly. Commercial gateways include OpenRouter, Martian, and NotDiamond, while the open-source self-hosted ecosystem includes LMSYS's RouteLLM and cost-observability layers such as LiteLLM and Bifrost. Within the Anthropic ecosystem, the Agent SDK's model-selection argument, the Claude Code subagent model field—Explore defaults to Haiku—and the /model slash command all support this kind of routing. One caveat matters: when you call an API directly, it does not inspect input difficulty and choose a model automatically. The Anthropic, OpenAI, and Google APIs all use exactly the model name the user specifies by default. Automatic routing is an optional product-level feature, available in places such as Cursor's Auto, ChatGPT's auto mode, and OpenRouter's openrouter/auto. To save money through routing, you must build it yourself, whether with a gateway or a classifier.

That does not mean adding an external automatic router indiscriminately is the answer. A router such as OpenRouter's Auto Router, which dynamically changes the model on each call, conflicts directly with prompt caching. Anthropic's ephemeral cache hits only when the same model and the same prefix match. If the model key changes on every call, the cache key misses every time. You lose the 90% discount from cache reads, and it is common to save nearly half through routing only to give the money back through cache misses. This is why OpenRouter recognizes the issue and recommends a session_id-based stickiness option.

The pattern that production teams have converged on is surprisingly simple: do not classify dynamically on every call; route statically by task type. The subagent model field discussed earlier follows this pattern. An Explore subagent always uses a Haiku-powered code-exploration lane, while a code-review subagent always uses an Opus-powered review lane. Within each lane, the same model, system prompt, and tool definitions repeat, allowing every model to accumulate and hit its own separate cache. When people say “routing is becoming standard,” they increasingly mean static task-type branching in a “coordinator + executor” structure, not a classifier that runs on every call. That is how routing and caching can coexist without contradiction.

Cursor Composer 2.5

This is a slightly different way to save, but one option is to use an agent such as Cursor. Composer 2.5, Cursor's proprietary model released on May 18, 2026, is based on Moonshot AI's open-source Kimi K2.5 checkpoint and fine-tuned specifically for coding. In Cursor's own benchmarks, the team says it delivers coding performance comparable to Claude Opus 4.7 at roughly one-tenth the price. Published base pricing is $0.50 for input and $2.50 for output, exactly one order of magnitude below Opus 4.8's $5.00 input and $25.00 output rates.

Because Cursor measured the specialized model's benchmark itself, it is safer not to treat the absolute figures as definitive. The broader trend is what matters: a smaller model trained specifically for coding can deliver practically comparable performance to a general-purpose frontier model on the same work while reducing costs by an order of magnitude. This has become one of the clearest cost-saving trends of 2026.

The truly interesting part is that this approach does not merely reduce the unit price: when the model is co-designed with an IDE, it also reduces the input token count itself. Models such as Composer and Cascade are trained to consume codebase context sent by the IDE—current files, adjacent files, and indexed symbols—efficiently. On the same task, that reduces the input-token inflation caused when a general-purpose model repeatedly asks to see relevant files and performs more grep and read operations. The unit price drops by an order of magnitude and the token count falls slightly as well, so real savings can exceed the price difference alone.

Move Context Outside the Context

One of the most interesting trends of 2026 is loading “only what is needed, when it is needed” into context. Anthropic calls this just in time (JIT) context. Instead of placing every resource in the context in advance, the agent carries lightweight references such as file paths or queries and retrieves the actual material with a tool only when necessary. Claude Code follows this pattern by reading files on demand with glob and grep rather than indexing and loading the entire codebase into context at once.

The memory tool and context editing that Anthropic released with Sonnet 4.5 follow the same philosophy. One distinction is important: the “memory” in the memory tool differs from what people commonly imagine. It is neither the conversation history that accumulates within a session—remaining in the context window and disappearing when the session ends—nor a configuration file such as CLAUDE.md that a person writes in advance and the tool loads automatically at startup. The memory tool is a tool through which the model directly writes and reads files.

At a deeper level, the two tools expose surprisingly simple interfaces. The memory tool lets Claude create, read, update, and delete files in a dedicated memory directory hosted in the user's infrastructure. In other words, the model handles notes like an ordinary file system rather than being told to “write this down somewhere in the text.” Keeping storage in user infrastructure is the crucial design choice. It gives Claude persistent memory outside the context window that can carry across sessions, while letting the user control how the data is stored and retained. Session memory is volatile inside the context; this memory remains outside it and survives into the next session. Context editing does the same work in the opposite direction. As the token limit approaches, it automatically clears old tool-call results that are no longer referenced, preserving the conversation flow while allowing the agent to run longer.

In Anthropic's own evaluation, using both tools improved performance by 39% over the baseline, while context editing alone improved it by 29%. In a 100-turn web-search evaluation, token consumption fell by 84%. (These are vendor benchmarks and should be read accordingly, but the direction is clear: designing context to be emptied matters more and more than filling it.) This trend aligns with Claude Code's five-stage /compact pipeline. The difference is that /compact reduces what is inside the context, while the memory tool creates separate storage outside it. The two approaches complement rather than conflict with each other.

Conclusion

Viewed together, token savings ultimately reduce to three axes: send less for the same work, pay less for the same input, and assign the same answer to a cheaper model. Prompt caching, Batch API, subagent isolation, /compact, the memory tool and context editing, model routing, and specialized models differ only in which of these three axes they attack. If the direction of 2026 had to be summarized in one sentence, it would be this: the center of gravity is shifting from the technology of filling context to the technology of emptying and selecting it—context engineering. As context rot demonstrates, a lighter context improves not only cost but accuracy.

There is no single correct answer, of course. Every team has different work patterns, and even one person's cost structure changes between writing prose and writing code. One thing is certain: as the ecosystem moves quickly, assuming that “a savings pattern that worked yesterday will still work today” is becoming increasingly risky. Keeping up with new models, tools, and price lists—and building a habit of validating them through a small POC in your own workflow—may ultimately be the least conspicuous yet most durable way to save. I encourage readers to take out their own token ledger at least once. (If you want to understand why these strategies work at the root level, read the discussion of BPE and KV cache in How Tokens Work as well.)

One question remains. Now that we have techniques for emptying and selecting context, what comes next? Following the available research, the focus is moving one step beyond context to the agent system as a whole: how to operate it (harness design), how to measure whether it operated well (eval), and how to confine it when it goes wrong (containment). Interestingly, when the “measure it yourself with a POC” mindset emphasized throughout this article scales to the system level, it becomes exactly an eval. I plan to explore that next direction in detail in Beyond Context.

Until now, I have mainly written about how to use AI tools effectively, which tools are gaining popularity, and why. But while putting together my guide to saving tokens, I was reminded of something important: before explaining how to reduce costs, I first needed to explain what tokens are and how they are billed. Yet I had never properly covered that foundation. (As I wrote the cost-saving guide, the explanation of how tokens work grew substantial enough to become a post of its own.)

This post, then, lays the groundwork before we get into cost-saving techniques. We will go through what tokens actually are if they are neither words nor characters (BPE), what form they take when they enter a model (embeddings), why output is more expensive than input (prefill/decode), where inside the Transformer architecture prompt caching gets its cost advantage (KV cache), and why longer contexts become more expensive (the quadratic cost of attention). If you are looking for practical ways to save, continue with How to Save Tokens after this article.

Tokens Are Neither Words nor Characters

Let us start with the most fundamental fact: a token is neither a word nor a character. It is a unit in a vocabulary built by compressing character sequences that occur frequently in the model's training data.

Our intuition for asking “How many words are in this sentence?” differs from the way a model splits text. A model groups character combinations that frequently appear together into single units, then segments incoming text according to that vocabulary. As a result, one word may be represented by a single token while another is split into several.

The algorithm used to build this vocabulary is BPE, a common foundation of nearly every modern LLM.

The BPE Algorithm

BPE (Byte Pair Encoding) is an algorithm that grows a vocabulary by repeatedly merging the most frequent pair of adjacent symbols into a new symbol.

Interestingly, it was not originally created for natural language processing. Philip Gage first proposed BPE in 1994 as a data compression technique. The method replaced the most frequently occurring byte pair in a dataset with a single byte not otherwise used in the data, then stored those replacement rules separately in a table.

A research team led by Sennrich at the University of Edinburgh brought the technique to the vocabulary problem in neural machine translation. In “Neural Machine Translation of Rare Words with Subword Units,” released in 2015 and presented at ACL 2016, they used BPE to overcome the inability of a fixed vocabulary to handle rare or previously unseen words. The idea was that instead of memorizing whole words, a model could represent them as combinations of smaller subword pieces, allowing even out-of-vocabulary words to be encoded from known parts.

The process itself is surprisingly simple.

1. Initially, each individual character is one token.
2. Find the pair of tokens that appears adjacent most often in the corpus.
3. Merge that pair into a new token, then replace every occurrence of the pair in the corpus with the new token.
4. Repeat steps 2–3 until the vocabulary reaches its target size.

At first, each character is its own token. Over time, frequently occurring combinations such as “th,” “the,” and “tion” are merged into single tokens. In other words, the more common a pattern is, the more likely it is to become a larger unit.

The GPT family takes this one step further with byte-level BPE. It first converts text into a stream of UTF-8 bytes, then applies BPE. This allows the base vocabulary to start with only 256 entries—one for each possible byte—while still being able to encode any text representable in UTF-8 without an unknown token. Whether the input contains emoji, Chinese characters, or special symbols, it is always representable at least at the byte level.

Same Text, Different Token Counts

Because of this structure, the same text can produce very different token counts depending on the tokenizer, the tool that splits human-written text into tokens a model can process. Depending on how its vocabulary was trained, a tokenizer may split the same sentence into finer pieces or larger chunks.

The material OpenAI published alongside GPT-4o illustrates this difference clearly. The o200k_base tokenizer used by the GPT-4o family represents the same text with fewer tokens than cl100k_base, the tokenizer used by the GPT-4 family. It is only about 1.1 times as efficient for English, but the gap widens for other languages. In the short examples OpenAI published, token counts fell by about 1.4 times for Chinese and Japanese, 1.7 times for Korean, and as much as 2.9 times for Hindi. (For Korean users, it is striking how much more noticeable the difference between tokenizer generations is than it is for English users.)

Looking more broadly, there are also disparities between languages themselves. In Yennie Jun's comparison of multiple languages using cl100k_base, English almost always required the fewest tokens for sentences with the same meaning, while languages with their own writing systems tended to require more. Hindi and Bengali could use five times as many tokens as English, and Burmese more than ten times as many. Korean and Chinese also tokenize into more pieces than English, though not as extremely as those languages.

This has an important practical implication. When Anthropic introduced a new tokenizer starting with Opus 4.7, its official pricing documentation stated that the same text could be billed as up to 35% more tokens than with previous models. Even when the per-token rate stays the same, the bill rises in direct proportion to the token count. An honest model comparison therefore has to consider not just the listed rate, but the rate multiplied by the expected number of tokens.

But when these segmented tokens enter the model, they do not go in as literal characters or words. So what form does a token take as model input?

How Tokens Enter the Model

Neural networks cannot work with text directly. Ultimately, a model can compute only with numbers—more precisely, with vectors, which are arrays of numbers. Before entering the model, tokens therefore pass through several stages that convert them into numerical form.

Here is the sequence.

1. Text → tokens: A BPE tokenizer splits the text into token pieces.
2. Token → token ID: Each token maps to an integer ID indicating its place in the vocabulary. For example, if " the" is the 1,234th token in the vocabulary, its ID is 1234.
3. Token ID → embedding vector: The ID selects the corresponding row from the embedding matrix. This matrix is a huge table with dimensions vocabulary size × model dimension (d_model): each token ID corresponds to one row, or one vector. You can think of this vector as a set of coordinates that captures the token's “meaning.”
4. Add positional information: On its own, self-attention has no awareness of token order. In other words, it would view “I like you” and “You I like” as identical. Positional information is therefore injected into each token vector to tell the model the sequence order.

That can feel abstract in words alone, so I have included a widget that lets you interact with all four stages. Change the input sentence or click a token, and you can follow its path from ID to embedding vector, then see positional information added to produce the final input vector. (Only a few dimensions are displayed, but keep in mind that the actual GPT-4 vector has 12,288 dimensions.)

Here, the model dimension (d_model) is the length of the vector used to represent one token. The original Transformer paper used a value of 512, but modern LLMs are much larger. (Even the 7B version of Llama 2 uses 4,096 dimensions.) A larger model dimension can represent subtle differences in token meaning more richly, but it also increases computation and memory requirements.

Methods of injecting positional information have also changed across generations. The original Transformer added position vectors built from sine and cosine functions directly to the embeddings. RoPE (Rotary Position Embedding), now the de facto standard in open LLMs, instead injects relative position during the attention calculation by rotating the Query and Key vectors. (The mechanism differs, but the purpose is the same: giving the Transformer a sense of order.)

In short, the flow is text → tokens → token IDs → embedding vectors (+ positional information) → Transformer input. The “token count” on a bill refers to the second stage of this pipeline: how many tokens the text was split into.

At this point, we know how tokens are created and what form they take when they enter a model. But when the model processes them, why do input and output cost so differently, and how can the cost of sending the same input repeatedly be reduced?

Why Input Is Cheap and Output Is Expensive

Anyone who has worked with tokens has probably wondered about this at some point: on pricing tables, output tokens cost several times more than input tokens. For Anthropic, the output rate is exactly five times the input rate across every model. (Opus is $5 for input and $25 for output, while Haiku is $1 for input and $5 for output.) Why should the same token have a different price depending on whether it is going in or coming out?

The answer is that models process tokens in completely different ways for input and output. LLM inference has two stages.

• prefill (input processing): The model processes all prompt tokens at once and in parallel. Whether the prompt contains 1,000 or 10,000 tokens, the GPU sweeps through them together and calculates the K/V (Key/Value) vectors for each token. The total computation is substantial, but parallel processing makes it efficient per token.
• decode (output generation): The model generates response tokens sequentially, one at a time. After producing one token, it appends that token to the input and generates the next. This repeats until the response is complete.

That asymmetry creates the cost difference. According to Databricks' analysis of inference performance, prefill is a compute-bound stage, while decode is memory-bound. During decode, the model has to read all of its weights from GPU memory again for every token it produces, yet each pass yields exactly one token. Most of the GPU's enormous compute capacity sits idle. (To borrow Databricks' phrasing, you are paying to keep the GPU running without using its available compute.)

In other words, input tokens are processed efficiently together in parallel during prefill, while output tokens must be squeezed out inefficiently one by one. Commercial factors such as demand and margins may also affect pricing, but at the technical level, this decode-stage inefficiency is part of the underlying reason. The principle of “keep outputs short” is therefore more than a generic savings tip: it directly reduces the most expensive stage.

Is there any way to make this inefficient decode stage even slightly less inefficient? This is where KV cache enters the picture.

How Caching Lowers the Price

Prompt caching stores static input in a cache once, then lets subsequent calls read it back at a much lower price. According to Anthropic's official documentation, a cache read costs 0.1 times the base input rate—exactly 10%. A cache write costs 1.25 times the base rate with a five-minute TTL (Time To Live, the period for which the cache remains valid), or twice the base rate with a one-hour TTL. You pay a little extra on the first call, then save 90% from the second call onward.

But how can the price fall by 90%? And why is the requirement that “the prefix must match exactly” so strict? Both answers become clear once we look inside the Transformer.

KV Cache

When processing input tokens, the model creates corresponding Query/Key/Value (Q/K/V) vectors. Self-attention calculates attention scores using the dot product of Query and Key, then uses those scores to produce a weighted sum of the Values. You can think of it as the calculation that determines how much one token should “refer to” every other token.

This is exactly where the decode stage described above becomes a problem. To generate each new token, the model needs the K/V vectors for all preceding tokens. Recalculating them from scratch every time would repeat the same K/V computation endlessly.

KV cache was introduced to eliminate this duplication. It stores previously calculated K/V vectors, so when generating a new token, the model reads and reuses the preceding tokens' K/V values from the cache instead of calculating them again. Within a single sequence, this reduces the cost of generating one additional token from O(n²), which would recalculate the entire sequence at every step, to O(n), which only requires reading the cache. (Strictly speaking, this describes the cost per step, but the central idea is simple: do not recalculate what has already been computed.) This is also why decode is memory-bound: every step must read the KV cache back from memory.

Reuse Across Calls

If KV cache was originally designed to accelerate decoding within a single response, prompt caching extends the same memory structure beyond one call, reusing it across calls.

The K/V values of a static prefix—such as a system prompt, tool definitions, or code snippets whose beginning remains nearly identical—stay in GPU memory for a five-minute or one-hour TTL. If the next call starts with the same prefix, the model skips that prefix's K/V computation entirely and reads it directly from the cache. More precisely, the input token rate is not reduced by magic: the GPU work required to compute those tokens disappears altogether. The 90% price reduction reflects the computation that has been avoided.

Once you understand this mechanism, the strict requirement that “the prefix must match exactly” also makes sense. A cache hit is determined by whether a cumulative hash of the prefix matches. Because self-attention is causal, changing even one earlier token changes the K/V values of every token that follows. Put just one timestamp at the beginning of a prompt, and its value changes on every call, causing the prefix hash to diverge and invalidating the entire cache after it.

Anthropic reads its cache in the hierarchy tools → system → messages. Changing even one tool definition near the beginning therefore invalidates the whole cache that follows. The conclusion is straightforward: put static content first and dynamic content that changes on every call last if you want the cache to survive.

Caching also has several lesser-known constraints. The minimum cacheable token count differs by model, and even by version within the same family. A prompt shorter than that threshold silently goes uncached even when caching is enabled. (Because these minimum token counts and provider-specific caching policies directly affect cost design, I cover them in detail alongside the pricing tables in the token-saving guide.)

Why the Context Window Sets a Token Limit

The final concept no discussion of tokens can omit is the context window: the maximum number of tokens a model can accept at once. Understanding why this limit exists also clarifies the familiar warning that “longer context costs more.”

The key lies in the computational structure of self-attention. Attention involves every token referring to every other token once. With n tokens, there are n × n token pairs, so the computation and memory required by self-attention grow with the square of the sequence length n. Double the number of tokens, and the attention cost quadruples. (It is worth clarifying that the entire model is not O(n²); the attention stage is.)

Hugging Face's estimates show just how steep this quadratic growth becomes. With standard, unoptimized attention, storing the attention score matrix alone requires only about 50 MB for an input of 1,000 tokens, but about 19 GB for 16,000 tokens and nearly 1 TB for 100,000 tokens.

The KV cache described earlier also consumes memory in proportion to the number of tokens. As the context grows, the K/V values the model must retain accumulate linearly. In the end, the context-window limit is a boundary set by physical constraints: beyond that point, the memory and computation become unmanageable.

This is why carrying a long conversation forward unchanged is not simply a matter of adding more input tokens. Attention costs grow quadratically, and as the token count approaches the limit, the model's ability to handle information accurately also declines. (This loss of accuracy is directly related to context rot, which I discuss in the token-saving guide. It is precisely why keeping context lean benefits both cost and answer quality.)

Conclusion

Putting it all together, the way tokens work comes down to a few simple facts. A token is a vocabulary unit created by BPE from frequently used patterns, and it enters the model as an embedding vector after being mapped to a token ID. Input is processed all at once through parallel prefill, while output is more expensive because it must be squeezed out one token at a time, and the KV cache reduces that inefficiency by reusing K/V values. Caching lowers the price not through magic, but because the Transformer does not recalculate K/V values it has already computed, and the demanding requirement for an exact prefix follows directly from this architecture. Finally, longer context costs more because attention cost grows with the square of the token count.

Once you understand this foundation, the economics become much clearer. Strategies such as shortening output, arranging prompts so the cache remains intact, keeping context lean, and switching to a cheaper model when it gives the same answer all rest on these principles. If you have read this far because you wanted to understand the mechanics, I recommend continuing with the token-saving guide to see how these principles translate into real cost reductions.

This is for developers whose projects have piled up CLAUDE.md, AGENTS.md, SKILL.md, and .cursor/rules side by side, and who are unsure when each is read and what should go in it. By the end, you will know how the files differ, how agents load them, and a rule of thumb backed by an ETH Zurich study: write down only what the agent cannot infer.

As a frontend developer, I use Claude in my day-to-day work. At some point, that meant a CLAUDE.md appeared at the project root, an AGENTS.md created by someone else sat beside it, a .cursorrules file lingered elsewhere, and I even created a .claude/skills/ directory after following an article I had come across. (By the time I stopped to take stock, I had about five files containing roughly the same information.)

Agents Without Persistent Memory

AI coding agents have a fundamental limitation: they have no persistent memory. Every session starts from a blank slate. In the next conversation, they cannot remember a convention you agreed on yesterday or a directory structure you explained an hour ago. Context files are the simplest mechanism for addressing this problem. If a project contains a file that is read automatically whenever a session starts, you no longer need to repeat the same explanation every time.

The problem is that each tool created its own file around the same basic idea. Claude Code reads CLAUDE.md, Cursor reads .cursorrules (now deprecated, with .cursor/rules recommended instead), GitHub Copilot reads .github/copilot-instructions.md, and OpenAI Codex reads AGENTS.md. When a team uses several tools, it can end up copying the same information into four different places.

CLAUDE.md

CLAUDE.md is a file that Claude Code reads automatically at the start of a session. According to Anthropic’s official documentation (code.claude.com/docs/en/memory), Claude Code looks for CLAUDE.md at the following three levels.

• User memory (~/.claude/CLAUDE.md): global defaults that apply to every project on the machine
• Project memory (CLAUDE.md at the project root): committed to git and shared across the team
• Local memory (CLAUDE.md in a subdirectory): loaded in addition to the others only when working in that directory

When all three levels exist, Claude reads and concatenates all of them. It does not choose only one according to precedence; instead, more specific instructions are layered on top, much like the CSS cascade. (This is a merge, not an override.) As a result, scattering rules about the same topic across several levels can create conflicts. (Anthropic’s official documentation explicitly states that behavior is not guaranteed when instructions conflict.)

There is one frequently overlooked detail here: Claude reads every CLAUDE.md it encounters while walking from the current working directory up to the repository root. If you work inside packages/ui/ in a monorepo, both the root CLAUDE.md and packages/ui/CLAUDE.md are loaded. (That is powerful, but it also means the context can quietly expand without you noticing.)

AGENTS.md

AGENTS.md is a standard created to address the proliferation of tool-specific files described above. In December 2025, OpenAI donated this standard to the Linux Foundation’s Agentic AI Foundation (AAIF). According to the Linux Foundation’s announcement, the AAIF was co-founded by Anthropic, Block, and OpenAI, and its founding projects are three: Anthropic’s MCP (the protocol that connects agents to external systems), Block’s goose, and OpenAI’s AGENTS.md. The official site (agents.md) states that more than 60,000 open-source repositories have adopted the file.

The list of supported tools makes the picture even clearer. It includes OpenAI Codex, Google Jules, VS Code, GitHub Copilot, Cursor, JetBrains Junie, Aider, Devin, Zed, Factory, Warp, goose, opencode, Amp, RooCode, Gemini CLI, Kilo Code, Phoenix, Semgrep, Ona, Windsurf, and Augment Code, among many others. GitHub Copilot began supporting AGENTS.md natively in August 2025. One interesting detail is that native AGENTS.md support in Claude Code is still an active feature request. Claude Code continues to treat CLAUDE.md as its primary file.

You might still wonder whether this supposed standard is actually being adopted. The strongest evidence is dogfooding—the organizations behind the standard using it themselves.

• The canary branch of Vercel/Next.js has an AGENTS.md at its root. It is actually a symbolic link to CLAUDE.md, whose contents cover the monorepo structure, one-to-two-second iteration with pnpm --filter=next dev, testing guidance for both Turbopack and Webpack, the pr-status script, and rules for handling environment variables and secrets. The fact that create-next-app now generates both AGENTS.md and CLAUDE.md for new projects reflects the same trend.
• The OpenAI/codex repository maintains its own AGENTS.md.

A conventional strategy is beginning to emerge: use AGENTS.md as the single source of truth, while keeping CLAUDE.md minimal, with a one-line reference to AGENTS.md plus instructions specific to Claude Code. This eliminates duplication, and because Claude Code reads both files, nothing is lost.

SKILL.md

SKILL.md belongs to a different category from the two files above. Whereas CLAUDE.md and AGENTS.md are persistent instructions that are always present in the context, a Skill is an on-demand capability invoked only when needed.

A Skill is organized as a directory. The directory contains one SKILL.md, scripts executed by the Skill, and any additional Markdown documents. Claude loads that directory only when the current task matches the Skill’s description. This is called progressive disclosure, a concept established in the UX field by Jakob Nielsen in 1995: advanced or rarely used functions are deferred to secondary screens so users can focus on one task at a time, reducing cognitive load and errors. In the context of Claude Skills, it refers to the mechanism of bringing a Skill’s body into the context only when needed. The result can be a dramatic reduction in context-window costs.

The frontmatter in SKILL.md includes several distinctive fields.

• description: explains when the Skill is needed and acts as the trigger the model uses to decide whether to invoke it
• allowed-tools: restricts which tools may be used inside the Skill (for example, "Read, Glob, Grep, Bash(python:*)")
• disable-model-invocation: true: prevents the model from invoking the Skill; only the user can trigger it with a slash command. This is used for operations with side effects, such as deployments and commits
• user-invocable: false: hides the Skill from the user’s slash-command menu and allows only Claude to invoke it autonomously, for use as background knowledge

Claude Skills launched simultaneously across Claude.ai, Claude Code, the API, and Agent SDK on October 16, 2025. Then, on December 18, 2025, Anthropic published the Skills specification itself as an open standard (agentskills.io). Simon Willison even called it “Skills are awesome, maybe a bigger deal than MCP,” citing the format’s dramatic simplicity compared with MCP and its use of progressive disclosure to address context-window costs.

MCP (Model Context Protocol), which Skills were compared against here, is a standard protocol that connects an agent to external systems such as Slack, GitHub, or a database so it can call them. If context files are about what to tell the agent, MCP is about what to enable the agent to do. How MCP differs from function calling is covered separately in MCP and Function Calling.

The tools that cut the cost of an agent finding relevant code (Repomix, Aider, CodeGraph, Serena) cut different costs depending on how deeply they understand the code. That comparison is covered separately in Four Tiers of Code Intelligence.

Files Used by Other Tools

Cursor’s .cursorrules has been deprecated since version 0.43. The current official recommendation is to use the .cursor/rules/ directory and place multiple .mdc files inside it. Each .mdc file has YAML frontmatter.

• description: information the agent uses to judge whether the rule is relevant
• globs: automatically attaches the rule when a matching file is included in the conversation (auto-attach)
• alwaysApply: when true, includes the rule in every conversation without exception (globs is ignored in this case)

GitHub Copilot has evolved in a similar direction. Repository-wide instructions live in .github/copilot-instructions.md; instructions that require path-specific scope go in .github/instructions/*.instructions.md, with globs specified through the frontmatter key applyTo:. (Copilot code review has officially supported path-scoped instructions since September 2025.)

Tools beyond Cursor and Copilot are converging on similar patterns. The following table summarizes them.

Aider’s CONVENTIONS.md is particularly interesting. Because the official documentation says this entire file is included in the context with every request, it explicitly instructs users to “keep it under 200 lines.” (In effect, Aider recognized this limitation early and tells users about it directly.)

MEMORY.md

Separate from the files above, another pattern is appearing with increasing frequency: MEMORY.md. It is not an official standard, but an organically developed community convention for recording decisions and mistakes over time.

## 2026-04-10
Pages Router에서 App Router로 이전. 신규 라우트는 App Router 컨벤션 사용.
 
## 2026-04-22
Prisma 쿼리 결과에 optional chaining 쓰지 말 것 — null은 if-check로 명시적 처리.
(이전에 옵셔널 체이닝으로 null을 흘려보내 프로덕션 이슈 발생.)

If CLAUDE.md or AGENTS.md records the rules as they stand today, MEMORY.md records the history of why those rules were created. (The two are complementary, not interchangeable.)

How Agents Read These Files

So far, we have cataloged the files that exist. But one surprisingly common question remains unanswered: where, exactly, do agents load these files, and how? Understanding this mechanism makes it easier to see why the ETH Zurich results discussed later—showing that context-file instructions are not followed reliably—emerged.

First, one essential fact: CLAUDE.md is injected as a user message, not as part of the system prompt. Anthropic’s official documentation states the following.

CLAUDE.md content is delivered as a user message after the system prompt, not as part of the system prompt itself. Claude reads it and tries to follow it, but there's no guarantee of strict compliance.

CLAUDE.md content is delivered as a user message after the system prompt, not as part of the system prompt itself. Claude reads it and tries to follow it, but there's no guarantee of strict compliance.

In other words, it is contextual guidance rather than an enforced rule. The official guidance recommends using a separate mechanism, such as a PreToolUse hook, when you need to enforce a specific behavior.

The load order progresses from broad → specific. More precisely, it is managed policy (organization-level settings) → the user’s global file (~/.claude/CLAUDE.md) → the project file (./CLAUDE.md) → the local file (./CLAUDE.local.md). Within the same directory, CLAUDE.md comes before CLAUDE.local.md. By taking advantage of the fact that the nearest instructions are read last, you can make more specific rules exert a stronger influence, thanks to the LLM’s recency bias.

The @import syntax is especially interesting. If you put @path/to/file anywhere in the body of CLAUDE.md, that file is expanded in place and loaded with it. The maximum recursion depth is 4 hops, and relative paths are resolved from the file containing the import statement. That is why the official recommendation is to bridge to @AGENTS.md. If you leave CLAUDE.md almost empty and put only @AGENTS.md in it, Claude Code will naturally read AGENTS.md as well. (Given that Claude Code does not yet support AGENTS.md natively, this is the cleanest workaround.)

The token implications also deserve attention. CLAUDE.md has no explicit token limit, so the entire file is loaded if it exists. The official recommendation, however, is to keep each file under 200 lines. Beyond 200 lines, it is said to “consume more context and may reduce adherence.” Interestingly, in Claude 4.x, merely enabling tool use automatically adds 346 tokens via a special system prompt (with tool_choice: auto). Context leaks away in ways that are easy to miss.

Cursor takes a different approach. Rules in .cursor/rules/*.mdc operate in four modes.

• Always Apply: included in every chat without exception; ignores globs/description
• Apply Intelligently (Agent Requested): the agent reads description, determines relevance, and pulls in the rule
• Apply to Specific Files (Auto Attached): activated when a file matching the glob pattern enters the context
• Apply Manually: explicitly invoked by the user with @rule-name

Other tools differ again. OpenAI Codex walks from the git repository root toward cwd, collects every AGENTS.md, and injects them immediately before the user prompt. GitHub Copilot inserts .github/copilot-instructions.md at a middle priority within the context window: “after edit context and explicit references, but before loosely related open files.” Because the load timing, precedence, and merge rules vary by tool even for the same AGENTS.md file, there is no guarantee that three tools will interpret it in exactly the same way.

That leaves a more fundamental question: why does a model follow only some instructions that are present in its context? Simply saying “because the instructions are long” is not enough. The phenomenon is rooted in structural limitations of LLMs.

Hallucination and Context Forgetting

If you have ever seen an AI agent confuse the conversational context or forget something that was clearly stated earlier, that is a form of hallucination. People usually think of hallucination first as “making up facts that do not exist,” but the academic literature divides it into three categories. A 2023 survey by Yue Zhang and colleagues (“Siren’s Song in the AI Ocean”) classifies them as input-conflicting (generating something inconsistent with what the user explicitly provided), context-conflicting (contradicting something the model previously generated), and fact-conflicting (disagreeing with world knowledge). Ignoring instructions in a context file belongs to the first category, not the third. The model processes the input while treating part of its information as though it were not there.

The deeper problem is that hallucination cannot be eliminated in principle. A research team at the National University of Singapore proved this mathematically using learning theory. No LLM can learn every computable function; therefore, as long as it is used as a general-purpose problem solver, it must hallucinate at some point.

Position effects matter as well. Stanford researchers demonstrated experimentally that models reference relevant information most effectively when it appears at the beginning or end of the context window, while performance drops sharply when that information is buried in the middle. This maps directly onto context files. CLAUDE.md is inserted somewhere in the middle by the load order, and as a conversation grows longer, its instructions are pushed further into the “middle” of the context. This also connects to the other side of the recency bias mentioned earlier: within the primacy-recency effect, the middle is the weakest region.

Taken together, these phenomena form a coherent picture. A context file is merely additional text inserted outside the system prompt before the LLM’s first user turn. It is not a mechanism that forces the model’s decisions; it is simply another block of tokens dropped into the context window. The longer the file and the longer the conversation, the farther its instructions drift toward the “middle,” and the less reliably they are referenced. The ETH Zurich results quantitatively confirm this structural limitation.

The ETH Zurich Study

Many people may have thought, “Then I should put as much as possible into these files.” A recent study directly challenges that intuition: the ETH Zurich research referenced throughout the preceding section.

The paper, “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?”, was published by an ETH Zurich research team in February 2026. The researchers evaluated four agents—Claude Code (Sonnet-4.5), Codex (GPT-5.2 / GPT-5.1 mini), and Qwen Code—on a benchmark of 138 real-world Python software-engineering tasks (AGENTBENCH) and on SWE-bench Lite. The results were unexpected.

• LLM-generated context files actually reduced task success rates by about 0.5% on SWE-bench Lite and 2% on AGENTBENCH
• Even human-authored files produced only a modest average improvement of about 4%
• Adding a context file increased inference costs by more than 20% per instance
• Context files had an even smaller effect on the stronger model (GPT-5.2), because more capable models already have sufficient parametric knowledge and additional context can become noise

There was one exception: specifying a nonstandard tool. When the Python package manager uv was named in the context, for example, the agent’s use of uv rose from 0.01 times per instance to 1.6 times—an increase of roughly 160×.

Aider’s “200-line recommendation”—keep the file short because it enters the context every time—is practical guidance, while the ETH Zurich study quantitatively demonstrates that long context files reduce performance statistically. I see the following practical implications in the research.

• A huge, automatically generated context file can do more harm than good. If you cram coding standards, architecture, and workflows into a 300-line CLAUDE.md, the agent will follow some of them and ignore the rest. That inconsistency can produce worse results than having no context at all.
• What absolutely belongs in the file is information that cannot be inferred. This includes nonstandard tools, project-specific conventions, and past failure cases. The model already knows general coding best practices.
• Use AGENTS.md as the single source, keep CLAUDE.md limited to brief tool-specific instructions, and move detailed workflows into Skills.

So, What Does This Mean?

Context files differ from tool to tool in name, location, and when they are read. This article alone covered CLAUDE.md, AGENTS.md, SKILL.md, Cursor rules, copilot-instructions.md, and MEMORY.md, and which tool supports which file keeps changing. Memorizing a list of files goes stale quickly.

That is why my goal here was not to recommend a particular file format, but to develop an eye for how agents actually read these files. Once you understand why CLAUDE.md is injected as a user message, why different tools read the same AGENTS.md differently, and why instructions weaken in the middle of the context, a new context file format becomes much easier to read: “This is when it gets loaded, and this is how strongly it acts.”

What remains, ultimately, is one intuition from the ETH Zurich study: the model already knows a great deal. Stuffing a context file with everything you can think of does not make the agent follow it more faithfully. It is better to keep only what the model is unlikely to know—project-specific conventions, nonstandard tools, and past mistakes—and remove the rest. Writing a long context file and writing a good one are different problems.

Rather than expanding CLAUDE.md to hundreds of lines right away, I encourage readers to dig, at least once, into when, where, and how strongly the tools they already use read that file. I believe that understanding provides a stable foundation, however file formats change.

This is for developers who have watched an agent burn tokens by repeating grep and file reads in a large codebase, and who are wondering which tool to add, such as Repomix, CodeGraph, or Serena. These tools split into four tiers by how deeply they understand code (context packing, tree-sitter repository maps, knowledge graphs, and LSP), and each tier cuts the search cost in a different place. For the knowledge graph I include its maker's benchmark; for the other three tiers, measurements taken on this blog's own repository.

Ever since I saw codegraph on GitHub Trending and installed it myself, I have wondered, every time I come across a new tool, how exactly it saves tokens. Many of the tools in this post, too, I first came across on GitHub Trending, which I usually browse weekly, filtered to TypeScript and Python.

Code Intelligence Tools

Before an agent changes any code, it first has to find where the relevant code lives. It greps, reads files, filters, and greps again. In the CodeGraph benchmark we will look at later, the side answering without the tool used up to 43 tool calls for a single question. Code intelligence tools are attempts to cut this search cost.

"Cost" does not point to one thing, though. Tokens the model processed, the number of tool calls, and tokens still sitting in the context window after the work is done all move separately. For each tier, this post looks at which of the three it cuts.

I divide these attempts into the four tiers below, based on how deeply each tool understands code. There is no established industry classification for this; the grouping is my own. Every measured value in this post was taken on 2026-10-08 at commit 36e5cfa of this repository, and tokens were counted with tiktoken's o200k_base. The values differ from Claude's tokenizer, so use them for comparison rather than as absolute numbers.

Context Packing

The simplest solution begins with the idea: "Put everything into one context window." It builds no graph and performs no indexing. It simply serializes the entire repository into a block of text and hands the whole thing to the model.

The representative tool is Repomix. Its default output format is XML, and its README links to Anthropic's documentation on XML tags. It ships a CLI, a web app, a browser extension, and an MCP server.

GitIngest's strength is that it needs no installation. Change github.com to gitingest.com in a GitHub URL, and the whole repository becomes a single text page. code2prompt (made by Mufeed VH), a Rust CLI, lets you change the output format with templates.

rtk (rtk-ai/rtk) takes a slightly different direction. Where the tools above pack the whole repository at once, rtk compresses the output of the CLI commands an agent runs. It is a single Rust binary, registered in the hooks of several agents including Claude Code, Cursor, Copilot, Gemini CLI, and Codex, so when the agent calls git status, it runs rtk git status instead. It applies filtering, grouping, truncation, and deduplication to more than 100 commands. rtk's hook targets only Bash tool calls. Claude Code's Read, Grep, and Glob pass straight through.

The size of the reduction needs careful reading. The rtk README says it cuts Bash output by up to 90%, then immediately adds that this does not mean cutting your bill by 90%. From the model's side, command output is part of the input tokens, and input tokens are part of the bill, so the reduction dilutes at each step. The figure the official site puts forward is 56% on average, for the commands rtk rewrites. Where the tools above shrink the text going in, rtk shrinks the text coming back as tool call results.

The limit of this tier is that large repositories hit the token ceiling. Packing just the 109 files in this blog's src/ comes to 94,596 tokens. That still fits in the window, but a repository with thousands of files would hit the limit right away. Repomix answers this with --compress. According to the README, it uses Tree-sitter to keep function and class signatures and drop the implementation bodies.

# repomix 1.18.1, 이 리포 커밋 36e5cfa, 2026-10-08
npx -y repomix@1.18.1 src -o out.xml              # Total Tokens: 94,596 tokens
npx -y repomix@1.18.1 src --compress -o out.xml   # Total Tokens: 30,320 tokens

That is a 68% reduction. Comments stay, so visit-counter.ts, which has long JSDoc, only went from 1,642 to about 1,220 tokens, a 26% cut. The compressed output keeps import statements and signatures, but the calls inside function bodies disappear. You can see in the text which file imports what, but you cannot ask who calls whom. So the boundary between this tier and the next lies less in whether a tool sees syntax than in whether it can be asked about relationships.

tree-sitter Repository Maps

The next tier uses tree-sitter to analyze code structure without running a separate index server.

What tree-sitter, the tool this tier relies on, builds is a CST (Concrete Syntax Tree), a tree that represents the structure of source code. The official tree-sitter documentation also says it builds a concrete syntax tree. That tree keeps even parentheses and punctuation as nodes, and it is the tree the tools below work with.

tree-sitter is an open-source parser generator and incremental parsing library. GitHub's code navigation uses tree-sitter. Because it reparses only the edited part, changing one line in an editor does not reparse the whole file; it fixes only the changed part of the tree. That advantage belongs to editors, where edits keep happening. Aider, below, keeps a cache keyed on file modification time so it does not reparse files that have not changed.

Aider, an AI pair programming tool used in the terminal, is the representative example of this approach. Aider uses tree-sitter to extract definitions and references of functions, classes, and methods from each file. It then builds a graph with files as nodes. When file A references an identifier defined in file B, an edge runs from A to B.

To pick the important files in this graph, Aider uses PageRank. PageRank is an algorithm that gives a node a higher score the more links it receives, and the heavier they are. Aider uses a variant called personalized PageRank, which tilts the scores toward chosen nodes. It then fits the definitions and signatures of the top-ranked files into the token budget.

What goes into the budget changes with the current conversation. In Aider's repomap.py, an edge weight is the square root of the reference count times a multiplier. Identifiers mentioned in the conversation get 10x, and camelCase, snake_case, or kebab-case identifiers of 8 or more characters also get 10x. Identifiers starting with _ and identifiers defined in more than 5 files each get 0.1x, and edges going out of files currently added to the chat get 50x. Files in the chat, files mentioned in the conversation, and files whose path or file name matches an identifier in the conversation also receive PageRank personalization scores.

The budget is not fixed either. Aider's documentation gives the --map-tokens default as 1k, but the code clamps one eighth of the model's input limit to between 1,024 and 4,096. And when no files are in the chat, it grows up to --map-multiplier-no-files (default 2) times. I checked this with this repository's src/. The 0.86.1 release I ran and the commit linked above have the same ranking code. The printed maps were counted with o200k_base, including the three lines of guidance text Aider adds.

# aider-chat 0.86.1, 리포 루트에서 시작한다, 2026-10-08
mkdir /tmp/aidermap && cp -R src /tmp/aidermap/ && cd /tmp/aidermap && git init -q && git add -A && git commit -qm init
# 키는 더미 값이다. --show-repo-map 은 map 만 출력하고 LLM 을 부르지 않는다
export OPENAI_API_KEY=dummy
aider_map() { uvx --python 3.12 --from aider-chat==0.86.1 aider --no-check-update --analytics-disable --no-gitignore --model gpt-4o "$@"; }
aider_map --show-repo-map                                              # Repo-map: using 4096 tokens
aider_map --map-tokens 1024 --show-repo-map                            # Repo-map: using 1024 tokens
aider_map --map-tokens 1024 --show-repo-map src/lib/filter-posts.ts    # 이 파일을 채팅에 올린 상태

gpt-4o has a 128k input limit, so one eighth was clamped to 4,096, and with an empty chat the map came out within twice that. Once one file was added to the chat, the map shrank to fit the budget; only 10 of the earlier 27 files remained, and PostList.tsx and SearchModal.tsx came in new. A repo map is not a fixed summary of the repository but an excerpt fitted to the conversation at that moment.

There is one more tool worth noting in this tier: ast-grep (ast-grep/ast-grep). It is a tree-sitter-based CLI for structural search and rewrite, and it matches syntax tree nodes rather than text. For example, the pattern console.log($A) catches every call to console.log with a single argument, regardless of how line breaks or whitespace look. What it catches is the same syntactic structure. It does not know the type of $A or where the name console comes from. There is also an ast-grep-mcp server, so you can make an agent use structural search instead of text grep.

Knowledge Graph

The third tier goes one step further. It parses the entire codebase in advance, builds a knowledge graph, and stores it on disk; the agent then sends queries to that stored graph. The most talked-about example is a tool called CodeGraph.

The structure the CodeGraph README describes is simple. It parses code with tree-sitter to extract symbols, edges, and file information, and stores them in a local SQLite database. Name search goes through SQLite's FTS5 index. The agent asks this graph through MCP. The steps the README describes involve no LLM. So I take this extraction to be deterministic, meaning the same code always yields the same result.

If an LLM summarizes code to build the graph, results can vary even for the same code, and hallucinations can creep in. Parsing the syntax tree directly, by contrast, extracts symbol relationships only by the rules of the language grammar, leaving no room for that kind of interpretation.

FTS5 (SQLite Full-Text Search 5), which appears here, is a full-text search extension provided as an SQLite virtual table. According to the SQLite documentation, it has been included in the amalgamation since 3.9.0 (2015-10-14); you create a table with CREATE VIRTUAL TABLE ... USING fts5(...) and query it with the MATCH operator. You can keep a full-text index in a single SQLite file without running a separate search engine such as Elasticsearch.

Deterministic is not the same as complete, though. The same README lists route recognition for frameworks that lean on convention and reflection as ranging from 74.1% for Django to 83.9% for ASP.NET, and calls this the honest static-analysis ceiling. The same code gives the same result, but that result can be missing edges.

The benchmark was measured by CodeGraph itself. The same README's 2026-08-05 re-measurement ran Claude Opus 4.8 headless and asked one architecture question each on 7 open-source repositories. The side with CodeGraph MCP enabled cut average cost by 44%, processed tokens by 62%, and tool calls by 88%. This re-measurement blocked both sides from calling the codegraph CLI through Bash. In a measurement setup without that block, the side without the tool found and used the CLI in 26 of 28 runs, and the README states that earlier published figures were produced without the block.

The size of the savings did not follow repository size. On questions where the side without the tool used 28 to 43 tool calls, cost fell by 57 to 78%, and on Gin, which finished in 7, it was about even. VS Code, with about 11k files, came in at 71%, and Excalidraw, with about 640, at 78%. The README attributes this to how much searching each question required. With only one question per repository, I read only the direction from it.

The README also records a number pointing the other way. Processed tokens go down, but at the end of a multi-turn session, the retrieval results still sitting in the context window are about 80% larger on the CodeGraph side across the 7 repositories. The gap varies by repository; on VS Code alone it is 67k versus 18k tokens, about 3.7 times. It returns dense source text in one go, and that text stays in the window. CodeGraph cut tool calls and processed tokens, at the price of leaving more in the window.

Cursor took a different road and then changed course. The indexing described in the January 2026 Cursor blog post was not a syntax graph but semantic search based on vector embeddings. It split files into chunks locally, synced with the server via Merkle tree hashes, and turned the chunks into embeddings for semantic search. In July 2026, a Cursor Community Support Engineer replied on the forum: "Semantic/embeddings indexing is being turned down in favor of grep-based retrieval". In the same thread, another staff member wrote that as models got good at using grep, the older semantic search path was no longer helping in a meaningful way. The Cursor documentation now says Instant Grep builds and queries its index on your machine and does not store embeddings of your codebase for search.

LSP

The last tier relies directly on a language server. If tree-sitter knows "that a symbol exists," LSP knows "what that symbol is."

LSP (Language Server Protocol) is an open, JSON-RPC-based protocol that standardizes communication between editors and language analysis tools (code completion, go to definition, find references, refactoring, and so on). In 2016, Microsoft, Red Hat, and Codenvy announced a collaboration on it. The core idea is "don't reimplement a language analyzer for every editor; keep one server per language and have every editor query it." rust-analyzer and Python's pyright are LSP servers, while TypeScript uses typescript-language-server, which wraps tsserver (which speaks its own protocol) in LSP.

Serena (oraios/serena) is an MCP server in this tier. As of 2026-10-08 it has 30,093 stars, and the repository was created in March 2025. Serena's core idea fits in one line: show the agent code as symbols. Its core tools include find_symbol, find_referencing_symbols, and get_symbols_overview. You can choose one of two backends. The default is a language server implementing LSP (free/open source); the other option is a paid plugin that uses JetBrains IDE code analysis (with a free trial).

Measuring on this repository shows where the difference comes from. I looked for usages of isHiddenPost (src/lib/filter-posts.ts:12), which filters out private posts, in two ways. The text side is grep.

# 커밋 36e5cfa, 2026-10-08. 이 글도 같은 이름을 담고 있어 content/ 는 뺐다
git grep -n isHiddenPost -- ':!content'   # 16줄, 파일 9개
git grep -n -C1 isHiddenPost -- ':!content'   # 같은 16곳을 앞뒤 1줄과 함께

On the LSP side, Serena's find_referencing_symbols was called. LSP's find references asks by position (file, line, and column), not by name, so this tool also takes both name_path, the symbol name, and relative_path, the file holding the definition. Rather than running Serena as an MCP server, the tool was called directly from Python on the same commit.

# serena 3b99f8b, 커밋 36e5cfa, 2026-10-08. 리포 루트에서 실행한다
# uvx --python 3.12 --from git+https://github.com/oraios/serena@3b99f8b024dafd58c962ea6e74f37c8a730ef532 python refs.py
from serena.agent import SerenaAgent
from serena.config.serena_config import SerenaConfig
from serena.tools.symbol_tools import FindReferencingSymbolsTool
agent = SerenaAgent(project=".", serena_config=SerenaConfig().with_headless_mode_overrides())
agent.execute_task(lambda: None)  # 언어 서버가 뜰 때까지 기다린다
tool = agent.get_tool(FindReferencingSymbolsTool)
print(agent.execute_task(lambda: tool.apply(name_path="isHiddenPost", relative_path="src/lib/filter-posts.ts")))

The output is a single line of JSON. Here is just the first entry. Line numbers count from 0.

{"src/lib/filter-posts.ts": {"Function": [{"name_path": "filterPublishedPosts", "body_location": {"start_line": 22, "end_line": 24}, "content_around_reference": "...  22:export function filterPublishedPosts(posts: Post[]): Post[] {\n  >  23:  return posts.filter(post => !isHiddenPost(post))\n...  24:}"}]}, ...

Serena returned 10 references. It leaves out the definition at filter-posts.ts:12, and for the import lines in three files it returns the file itself as the symbol (File). The 11 code locations grep caught are these 10 plus the one definition line. This name is unique in the repository, so grep neither missed nor added any code location. The 5 extra places grep caught were sentences in docs explaining this function: the root CLAUDE.md, the .claude/commands/ files audit.md and write-post.md, and, in docs/research/snapshot-20260816/, the leftover copies of CLAUDE.md and write-post.md.

Serena returns each reference with one line before and after. So grep was run two ways, git grep -n, which returns only line numbers, and git grep -n -C1, which also returns one line before and after, and all three outputs were counted in full with o200k_base.

Compared with a grep that returns one line before and after, Serena uses 683 fewer tokens. The difference comes from not returning the 5 doc locations. But Serena attaches the symbol name and location information to each reference as JSON, so on the code side alone it actually uses 159 more tokens. And Serena's 892 tokens are more than the 703 of git grep -n, which returns only line numbers. What the LSP tier saves in this repository is not the search result but the next step, where the agent reads surrounding lines or files after seeing the grep result.

In that next step, if the agent read all 9 files grep hit in full, it would read 47,034 tokens: 3,716 from the 4 code files and 43,318 from the 5 doc files. That is an upper bound. In a real session, CLAUDE.md is likely already in context and would not be read again. With a common name like isLocale, results could differ even in code locations.

Wrapping Up

To sum up, the four tiers split by how deeply they understand code, and they cut different costs. Context packing hands over code as text, and even with --compress it keeps only syntax, cutting the tokens going in. A tree-sitter repository map knows that a symbol exists, puts a budget cap on the tokens going in, and lets the conversation at that moment decide what fills it. A knowledge graph stores relationships in advance and cuts tool calls and processed tokens, but by the vendor's measurement, what stays in the window actually grew. LSP knows what a symbol is; in this repository it did not return the same name found in docs, but attached metadata to every result instead. Where it saves is not the search result but the reading that follows.

So when I pick a tool, I look first at which cost is the problem right now, rather than at how deep the tier goes. If the window is small and sessions are long, I look at what stays behind; if round trips are slow, at the number of tool calls; if docs and code share names in the repository, at how many files I end up reading after a grep result. I read Cursor stripping out embedding-based semantic search and moving to a local text index (Instant Grep) as a sign that an expensive index is not always better than the model's own ability to search.

If these tools cut the cost of an agent finding code, how much of the project rules an agent should know from the start to write, and in which file, is a separate problem. I cover that in Context Files.

This is for developers who use MCP servers with Claude Code or Cursor but have found it hard to explain where MCP parts ways with the function calling in LLM APIs. The short answer is that MCP does not replace function calling. The host application converts the tool list it receives from an MCP server into the tools parameter of function calling, and passes the call the model chooses back to the MCP server. By the end, you will know which part of the protocol each of the four differences MCP adds on top comes from, which differences survive the 2026-07-28 revision, and what attack surface that structure opens.

I work as a frontend developer and use Claude every day, yet each time I added another MCP server, I was never quite sure how these tools actually entered the model's field of view.

MCP (Model Context Protocol)

MCP (Model Context Protocol) solves the question of “what should we enable the agent to do?”

Put more concretely, an AI agent needs to be able to call the Slack API to send a message to Slack. It needs to call the GitHub API to create a GitHub issue. It needs to handle a database connection to query Postgres. MCP unifies integrations with all these external systems under a single standard protocol. (The point is that clients and servers connect through the same specification.)

MCP is an open standard first released by Anthropic on November 25, 2024. Then, on December 9, 2025, Anthropic donated MCP to the Linux Foundation’s Agentic AI Foundation (AAIF). The AAIF was co-founded by Anthropic, Block, and OpenAI.

MCP is a protocol built on JSON-RPC. JSON-RPC 2.0 is a stateless, lightweight RPC (Remote Procedure Call) protocol that uses JSON as its wire format. It is transport-independent and can run over HTTP, TCP, or standard input/output. This post describes the 2025-11-25 revision, in which MCP is a stateful protocol that sets up a session for each connection. What changed in the revision after that is covered once we have walked through the call flow.

Six Primitives

The overview of the 2025-11-25 specification lists three features that servers offer and three features that clients offer. This post calls these six primitives. Here, a primitive has nothing to do with JavaScript's primitive types (such as string or number); it refers to a basic type of interaction defined by the protocol.

Server-side primitives

• Tool (model-controlled): an action the model autonomously decides whether to invoke. Such actions may have side effects
• Resource (application-controlled): data identified by a URI. The specification only has resources/read for reading its contents and no method for writing. How those resources go into the context is decided by the host application
• Prompt (user-controlled): a reusable template explicitly triggered by the user, for example through a slash command

Client-side primitives

• Sampling: a mechanism that allows the server to request a completion from the client’s LLM, making the client-server architecture bidirectional. It lets a server that needs generated text while running a tool borrow the model the client uses, without an API key of its own. It was deprecated, that is, scheduled for removal, in the 2026-07-28 revision.
• Roots: workspace boundary information through which the client tells the server, “This is the extent of the area you may work in”
• Elicitation: a feature that allows the server to request additional user input in a structured form while executing a tool

This distinction matters because who decides to invoke or provide something differs. A Tool runs at the model’s discretion, so an incorrect invocation carries risk, while a Prompt is explicitly chosen by the user. A Resource is chosen by the application by default, but the specification also allows implementations that include resources automatically, based on heuristics or the model’s selection. The three client-side primitives run in the opposite direction: the server asks, and the client decides whether to respond.

Two Transports

There are two standard transport mechanisms, and the specification also allows other custom transports (MAY). The first is stdio, which runs an MCP server as a local subprocess and communicates through standard input and output. It is well suited to locally operating tools such as filesystem and git integrations. The second is Streamable HTTP, which layers SSE (Server-Sent Events) streaming over HTTP POST and GET to create near-bidirectional communication. SSE is a way for a server to push data to a client in one direction over an HTTP connection. It is suited to scenarios that occur across a network boundary, such as remote servers, OAuth authentication, multi-client connections, and cloud deployments.

How an LLM Invokes an MCP Tool

Now that we have covered the primitives and transports, let us trace how an LLM actually discovers and invokes an MCP tool.

In the 2025-11-25 revision, the following handshake takes place when a connection begins. As of 2026-10-08, the TypeScript SDK’s 1.32.1 and the v2 line’s 2.3.1 also follow this order in their default configuration.

• Client → server: initialize request (sends the supported protocol version and client capabilities)
• Server → client: initialize response (server capabilities plus an optional instructions field)
• Client → server: notifications/initialized notification
• Client → server: tools/list request → receives the list of available tools
• (Later) The LLM decides to invoke a tool → the client sends tools/call → receives the result

The initialize response’s instructions field is where the server sends text describing how its tools should be used. The instructions line in the demo output below shows this value.

How, then, does the tool definition itself enter the LLM’s field of view? An MCP tool definition takes the following JSON Schema form.

{
  "name": "get_weather",
  "description": "Get current weather information for a location",
  "inputSchema": {
    "type": "object",
    "properties": { "location": { "type": "string" } },
    "required": ["location"]
  }
}

The host transforms the list returned by tools/list into the tools parameter of the Anthropic Messages API or the tools parameter of OpenAI function calling, then includes it in the LLM API request. With Anthropic, supplying the tool parameter automatically adds a special system prompt that teaches the model how to invoke tools. Its length varies by model. In the documentation table I checked on 2026-10-08, current models added 286 to 675 tokens with tool_choice: auto.

When the LLM decides that it should invoke a tool, its response contains a tool_use block ({"type": "tool_use", "name": ..., "input": ...}), and the response ends with a stop_reason of tool_use. The host receives this, sends tools/call to the actual MCP server, receives the result, places it in a tool_result block in the next user message, and sends it back to the LLM. This loop continues until stop_reason changes from tool_use to another value, such as end_turn or max_tokens. What we commonly describe as “the agent working” is essentially a sequence of these call-result-call loops.

Drawn as a picture, there is no line between the model and the MCP server. The model only sees the definitions it received through tools, and the MCP server receives requests from the host, not from the model. What connects the two protocols is the host in the middle.

I ran this conversion to see how short it really is. It ran on 2026-10-08 with Node v24.16.0, @modelcontextprotocol/sdk 1.32.1, and zod 4.6.5. Instead of stdio, the transport is InMemoryTransport, which connects the server and client inside one process, and no LLM was called. The tool_use block was built by hand in the shape shown in Anthropic’s documentation.

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
import { z } from "zod";
 
// MCP 서버: 도구 하나와 instructions 를 둔다
const server = new McpServer(
  { name: "weather", version: "1.0.0" },
  { instructions: "Use get_weather for current conditions only." }
);
server.registerTool(
  "get_weather",
  { description: "Get current weather information for a location", inputSchema: { location: z.string() } },
  async ({ location }) => ({ content: [{ type: "text", text: `${location}: 15C, partly cloudy` }] })
);
 
// 호스트: 같은 프로세스 안에서 서버와 잇고, 클라이언트가 보내는 메서드 이름을 찍는다
const [clientT, serverT] = InMemoryTransport.createLinkedPair();
const send = clientT.send.bind(clientT);
clientT.send = (m) => { console.log("C->S", m.method, m.params?.protocolVersion ?? ""); return send(m); };
await server.connect(serverT);
const client = new Client({ name: "demo-host", version: "1.0.0" });
await client.connect(clientT);
console.log("instructions:", client.getInstructions());
 
// 1. tools/list 결과를 LLM API 의 tools 파라미터 모양으로 바꾼다
const { tools } = await client.listTools();
const anthropicTools = tools.map((t) => ({ name: t.name, description: t.description, input_schema: t.inputSchema }));
const openaiTools = tools.map((t) => ({ type: "function", name: t.name, description: t.description, parameters: t.inputSchema }));
console.log("anthropic:", JSON.stringify(anthropicTools[0]));
console.log("openai:", JSON.stringify(openaiTools[0]));
 
// 2. 모델이 이런 tool_use 블록을 돌려줬다고 가정한다. input 이 그대로 tools/call 의 arguments 가 된다
const toolUse = { type: "tool_use", id: "toolu_demo", name: "get_weather", input: { location: "Seoul" } };
const result = await client.callTool({ name: toolUse.name, arguments: toolUse.input });
console.log("tool_result:", JSON.stringify({ type: "tool_result", tool_use_id: toolUse.id, content: result.content }));
await client.close();

The output of node post-demo.mjs is as follows.

C->S initialize 2025-11-25
C->S notifications/initialized 
instructions: Use get_weather for current conditions only.
C->S tools/list 
anthropic: {"name":"get_weather","description":"Get current weather information for a location","input_schema":{"type":"object","properties":{"location":{"type":"string"}},"required":["location"],"$schema":"http://json-schema.org/draft-07/schema#"}}
openai: {"type":"function","name":"get_weather","description":"Get current weather information for a location","parameters":{"type":"object","properties":{"location":{"type":"string"}},"required":["location"],"$schema":"http://json-schema.org/draft-07/schema#"}}
C->S tools/call 
tool_result: {"type":"tool_result","tool_use_id":"toolu_demo","content":[{"type":"text","text":"Seoul: 15C, partly cloudy"}]}

Converting the tool definition is nothing more than renaming fields. MCP’s inputSchema becomes input_schema in Anthropic and parameters in OpenAI. The output also shows that the SDK adds $schema to the schema. The other direction is just as short. Anthropic’s tool_use.input is an object, so it goes straight into tools/call as its arguments. On the result side, text blocks have the same shape and pass straight through, but image and error results have a different shape, so the Anthropic SDK’s MCP helper converts them separately. The OpenAI shape above is the Responses API format. The arguments of a call OpenAI returns is a JSON-encoded string, so it needs one pass through JSON.parse before being handed over. The code above builds only an Anthropic-shaped tool_use block, so this parse step does not appear in the output.

Four Things MCP Adds

So what does MCP add to function calling? Under the 2025-11-25 revision, there are four things.

• Dynamic discovery: the tool list is retrieved at runtime through tools/list rather than known at build time. The server can announce through notifications/tools/list_changed that the list has changed during a connection
• Stateful session: a connection is established with initialize, and requests are exchanged within it. There is no dedicated JSON-RPC shutdown message; the session ends by closing the transport
• Primitives beyond Tool: Resource, Prompt, Sampling, Roots, and Elicitation are exposed through capability negotiation. Capability negotiation is the step in initialize where each side announces the features it supports
• Bidirectionality: the server can request a completion from the client’s LLM in reverse through Sampling (deprecated in the 2026-07-28 revision)

After the 2026-07-28 Revision

As of 2026-10-08, the revision the official site opens as latest is the 2026-07-28 revision, and half of this list changed there. First, the handshake made of initialize and notifications/initialized and protocol-level sessions are gone. Instead, every request carries the protocol version and client capabilities in _meta. That field is a slot MCP reserves for attaching metadata alongside a message’s regular parameters. Servers MUST implement server/discover. It is an RPC that returns the server’s supported protocol versions, capabilities, and server information, and a client can call it before any other request to check the supported versions and capabilities up front.

Requests the server used to send first were replaced by a pattern called Multi Round-Trip Requests (MRTR). Instead of sending a separate request, the server returns an interim result saying it needs more input (input_required), and the client fills in that input and sends the original request again.

Sampling and Roots were deprecated along with Logging. They remain in the specification and still work, but new implementations should not adopt them, and the specification recommends integrating directly with LLM provider APIs instead of Sampling.

The SDKs’ default behavior, however, still follows the old way. TypeScript SDK 1.32.1 has 2025-11-25 as its latest version constant and does not know the 2026-07-28 revision, and 2.3.1 supports this revision but defaults to legacy version negotiation. The first line of the output above, initialize 2025-11-25, is the result.

What remains of the four, then, is dynamic discovery and the primitives beyond Tool (Resource, Prompt, Elicitation). Dynamic discovery has changed shape slightly as well: list change notifications go only to clients that opt in to the subscriptions/listen stream. In the end, where MCP parts ways with function calling is less about sessions or bidirectionality and more about a contract for exchanging the tool list and context at runtime.

When the API Becomes the MCP Client

Anthropic’s MCP connector (beta) shows how much of this contract reaches the model. It is a feature in which the Messages API connects directly to a remote MCP server, and the Limitations section of its documentation says that of the MCP specification’s features, "only tool calls are currently supported", and that "Local STDIO servers cannot be connected directly". The same documentation advises that if you need local servers, MCP prompts, or resources, you manage the connection yourself with an MCP SDK and use the Anthropic SDK’s conversion helpers.

In other words, when MCP is consumed at the function calling layer, only Tool remains. Resource and Prompt mean something only when there is a host to carry them into the screen or the context. OpenAI’s function calling guide also introduces a way to use the functionality of an MCP server as a built-in tool. OpenAI’s MCP servers guide describes only how to list and call tools, and does not say whether Resource or Prompt is supported.

The Attack Surface Dynamic Discovery Opens

Tool descriptions and tool call results pass through the host into the model’s context. So whatever a server writes there, the model reads. Both representative attacks come from this.

• Tool Poisoning Attack (TPA): an attack named and demonstrated in a PoC by Invariant Labs in April 2025. If malicious instructions are hidden inside an MCP server’s tool description, the model reads that text, which the user never sees, and may follow it without the user knowing.

• Rug Pull: an attack, described by Invariant Labs in the same post, in which the server changes a tool definition after the user has approved it. As in Elena Cross's example quoted by Simon Willison, you approve a safe-looking tool on day 1, and by day 7 that tool has been changed to send your API keys to an attacker; it happens because tool definitions are fetched from the server at runtime rather than at install time. The tool specification recommends (SHOULD) UI that shows which tools are exposed to the model but does not require re-approval of a changed definition, so re-approval falls to the host.

Wrapping Up

In short, MCP does not replace function calling; it is a standard layered on top of it. The model still invokes tools through the tools parameter and the tool_use loop, and the host translates between the two protocols. In the 2025-11-25 revision, what MCP added was dynamic discovery, a stateful session, primitives beyond Tool, and calls that go from the server to the client. With sessions gone and Sampling deprecated in the 2026-07-28 revision, what remains is a contract for exchanging the tool list and context at runtime. Because of that contract, attacks that poison tool definitions or quietly change them arise in the same place. When you add another MCP server, I recommend checking not only what that server can do but also whether your host tells you when its definitions change.

If MCP is about what to enable the agent to do, what to tell the agent is the job of context files such as CLAUDE.md and AGENTS.md. How those files are read by the agent, and how far their instructions are followed, is covered in Context Files.

AI tools such as Claude, ChatGPT, Cursor, Gemini, and Copilot have become deeply embedded in our daily work. Development has accelerated exponentially, but if I am honest, the quality of the services we build does not seem to have kept pace. We are increasingly confronted with as many new bugs as new features, and I often hear people say, "I have no idea why this ended up this way."

As development speeds up, we spend less time examining every line of code in detail. That makes it even more important, in my view, to have the fundamental skills needed to guide AI in the right direction. To preserve the quality of the result, we need to recognize problems in AI-generated code and steer it back toward the outcome we want. Those fundamentals may include domain-driven development, abstraction, TDD (Test-Driven Development), effective use of libraries, and building performance advantages.

Yet whenever I ask frontend colleagues—and colleagues in other IT roles—"What is the hardest part of frontend development?" the answer I hear most often is remarkably consistent: "Managing state flow."

This article explores why managing state flow is so difficult and what kind of judgment and intuition we need to cultivate to handle it well.

What Is State?

Before getting into the main discussion, let us begin with the most fundamental question: what exactly do we mean by "state"?

While studying frontend development, I often read articles by hoseung.me. There, state is described as "any data that can affect the UI." A like count, the contents of a shopping cart, whether a modal is open, an input value, information about the signed-in user, the currently selected tab, search results, and whether something is loading—all of these are state.

The official React documentation defines it more formally. The page is titled "State: A Component's Memory", which can be understood as "the mechanism by which a component retains data between renders and tells React to trigger a re-render when that data is updated." In other words, it is data that persists over time, changes in response to events, and causes the UI to be redrawn when it changes. One more important point: state is isolated to each component instance. Even if the same component appears ten times on a page, each instance has its own independent state. This fact connects directly to the later discussion of where state should live.

Both definitions point to the same idea: state is "a value that changes over time and affects rendering." A constant that never changes is not state. Primitive design tokens fixed at build time are not state, but a dark mode setting that the user can toggle is. (Strictly speaking, the values themselves resolve according to the dark or light theme state. The "theme selection" is the state, while the tokens are more accurately understood as a mirror reflecting that state.)

There is one important caveat: not all state lives in components. Some state lives in cookies; some in localStorage, sessionStorage, or IndexedDB; and some in the URL. When server-resident data is brought to the client and cached, that cache also becomes a form of state. Even browser-managed values such as scroll position and the history stack sometimes need to be treated as state because they determine how our application behaves.

Why Is It So Difficult?

At first, handling state sounds simple enough. Create the state you need, pass it where it is needed, and correctly handle updates and resets. What could be so difficult about that?

Keep that question in mind and open a page from the service you are currently working on.

How many components does that page contain? Even a simple page may consist of dozens of components, while a complex one may contain hundreds arranged in a tree. Each component may hold its own state, share state with a sibling, or receive state from a parent. State can also carry across pages. Some state must survive a refresh, while other state should disappear when the tab closes.

This is the real reason state is difficult to manage: we cannot see at a glance where countless pieces of state are declared, how they are updated, or when they cease to exist. As the number of components with similar roles grows, both naming state and tracing the code that changes it become increasingly difficult.

An invisible web begins to form. A click in component A invalidates B's data; invalidating B closes C's UI; and closing C discards the form input. If that chain is not explicit anywhere in the code, we have to reconstruct the web in our heads whenever we debug a bug.

How, then, should we untangle this web? I believe the first step is recognizing that "state comes in different kinds."

Not All State Is the Same

Kent C. Dodds divides state into Server Cache (information that exists on the server and is held by the client for fast access) and UI State (information that exists only in the UI to control interface behavior). We often make mistakes when we lump the two together.

The official TanStack Query documentation defines TanStack Query as a server-state library that manages asynchronous work between the server and the client, while tools such as Redux, MobX, and Zustand are client-state libraries. (They can store asynchronous data, but doing so is inefficient.)

The central point is clear: Server State and Client State are different problems. Server State is asynchronous, can be changed by other users, and becomes stale over time. Client State is synchronous, under our control, and disappears on refresh. (More precisely, when the page unloads, the JavaScript runtime restarts, and the component tree and its state in heap memory are reclaimed together. When the component mounts again, it starts over from the initial value of useState.) If we try to handle both with the same tool, we must implement patterns such as cache invalidation, background refetching, and optimistic updates ourselves.

I take this one step further and divide frontend state into seven categories. To be clear, these seven categories do not separate neatly along a single axis. They mix storage location, source, lifecycle, and role, so a single piece of state may belong to several categories at once. Rather than treating this as a perfect taxonomy, think of the categories as questions to ask when deciding how state should be managed.

• Local State: State used only within one component or a narrow subtree
• Global State: State that must be shared across the application
• Server State: State whose source of truth is the server and whose client-side copy is a cache
• Form State: Temporary state that exists while the user is entering data
• URL State: Shareable state that lives in the address bar and survives refreshes
• External State: State outside React, such as cookies, localStorage, sessionStorage, and IndexedDB
• State Guard: Logic that blocks, allows, or validates access and actions based on combinations of state, rather than state itself

Beyond these categories, there is workflow state that may need to be modeled as a state machine and real-time collaborative state built on WebSocket or CRDTs.

Let us examine why each category calls for different tools and what kind of judgment we should apply to it.

Local State

This is the simplest kind of state. It is used only within a single component, and the outside world neither needs nor is entitled to know about it. Examples include whether a modal is open, whether a toggle button is on or off, hover state, and a search term currently being entered.

function SearchBox() {
  const [query, setQuery] = useState("");
  return <input value={query} onChange={(e) => setQuery(e.target.value)} />;
}

This much is probably familiar. The genuinely difficult question with Local State is deciding "where should this state live?"

In Kent C. Dodds's article on State Colocation, he explains that people are accustomed to "lifting up" state, but when the code changes, they rarely move that state back to where it belongs by "colocating" it.

Lifting state is the natural thing to do when sibling components need to share it. Because both siblings need to see the same data, we move the state to their common parent and pass it down through props.

The problem arises when those siblings no longer need the state. We are much less likely to move it back down into a child. As a result, parent components accumulate state that has little to do with them, and every parent re-render causes the entire child tree to re-render as well.

The first principle for Local State, then, is this: to make the code faster and simpler, keep state as close as possible to the code that uses it. If state is used by only one child of a component, there is no reason for the parent to own it. Move it into that child. The parent becomes lighter as a result.

Global State

Global State must be accessible from anywhere in the application. Authentication information, theme, language, and notifications such as toasts are common candidates.

The difference between Local State and Global State is not simply where they live. They make different contracts about how they may be referenced. Local State promises that "this has meaning only inside this component," while Global State publishes a codebase-wide promise that "this value can be referenced by this name from anywhere in the application." The cost of that promise is the essence of Global State.

Creating a piece of Global State is, in effect, adding another implicit dependency across the entire application.

Server State

We put data returned by an API into Client State and manage loading and error states ourselves with booleans—until we finally ask, "Why am I writing the same boilerplate every time?"

Tanner Linsley, the lead maintainer of TanStack, puts it this way: "Client State is synchronous and predictable. Server State is asynchronous, shared across multiple components, and requires careful handling of caching, background refetching, and error states." Server State is therefore fundamentally a different species from Client State. It should not be handled with the same tool.

What makes Server State difficult is not the tooling but the nature of the data itself.

The data the client displays belongs to the server. What the client holds is merely a snapshot from a particular moment in time. As time passes, this data becomes stale. It is also asynchronous, can fail, and moves through states such as pending, error, and success.

The most important fact is that responses are not guaranteed to arrive in the order requests were sent. Imagine typing "react" quickly into a search box. Requests for r → re → rea → reac → react are sent in that order, but if the response for "react" arrives first and the response for "rea" arrives afterward, the UI will display the results for "rea". Preventing this requires accounting for concurrency hazards (race conditions) that would otherwise force us to implement AbortController logic or request ID tracking by hand every time.

Form State

Forms are a peculiar kind of state. Their values change intensely while the user is typing, but usually disappear once the form is submitted. They are not shared elsewhere, and in most cases there is nowhere else they need to be stored.

The problem is that this intense rate of change can be expensive. If every keystroke causes a React re-render, input lag can become noticeable in a large form. And a form does more than simply hold values. Validation, dirty checks, submission status, error messages, and multistep flows all bring several kinds of state to life within a single form at once.

A multistep form such as a three-step checkout flow is often expected to preserve progress even if the page is refreshed midway through. If its values live only in useState, a refresh wipes them all out. It is more natural to store them in sessionStorage (temporary, tab-scoped storage) or the URL (for shareable steps). In other words, depending on its lifecycle requirements, Form State combines with External State or URL State.

URL State

Imagine a search page where users can filter by category, sort order, and page number. If we keep these values in useState, three problems arise at once.

• Refreshing the page resets every filter
• Sharing the URL with a friend shows them the unfiltered page
• Pressing the Back button does not restore the previous filters

A natural solution is to put the state in the URL. The URL is effectively a free persistent store with built-in support for refreshes, sharing, and navigation history.

/products?category=shoes&sort=price-desc&page=2

That single URL contains the complete state: "page 2 of the shoes category, sorted by price in descending order." There is no need to keep a separate copy in useState.

When is the URL the right place for state? A URL is a public interface. Passwords, authentication tokens, and temporary notes that users would not want others to see must not go into it. Nor should values that change too frequently—such as a search term updated on every keystroke—be written directly to the URL, because they will fill the history stack with noise. In those cases, apply the change after a debounce or avoid polluting history by using push only when appropriate and choosing replace for updates that should not add an entry.

Values in a URL are always strings. Numbers, booleans, arrays, and objects must be serialized and deserialized. URLs must also follow the rules of percent encoding, which gives characters such as &, =, Korean text, and spaces special handling. Implementing all of this by hand every time quickly becomes a breeding ground for bugs.

const params = new URLSearchParams(location.search);
const page = Number(params.get("page") ?? "1");
params.set("page", String(page + 1));
navigate(`?${params.toString()}`);
 
const [page, setPage] = useQueryState("page", parseAsInteger.withDefault(1));

Libraries such as nuqs solve both problems through the concept of a parser. Parsers such as parseAsInteger, parseAsBoolean, and parseAsJson handle serialization, deserialization, and types together. The library supports most common environments, including both the App Router and Pages Router in Next.js, React Router v6/v7, TanStack Router, and Remix.

Does that mean we can put unlimited amounts of state into a URL? Separate from serialization and typing concerns, one final constraint remains. RFC 7230 does not set an exact limit, but recommends that servers support at least 8,000 octets (an octet is the term used in networking and data communications to refer unambiguously to a group of eight bits, or one byte). Browser limits also vary. Modern browsers generally permit anything from 8 KB to tens of thousands of characters, but search engines, social media Open Graph and sharing pipelines, and some gateways may truncate URLs at around 2 KB. So do not treat the URL as boundless storage. Keep only the essential, shareable filters there, and leave the rest to sessionStorage or server-side storage.

External State

React knows only about state within its own world. Our applications, however, constantly communicate with the world outside React. State that lives there can persist and change independently of React's lifecycle. Here, External State includes Cookie, localStorage, sessionStorage, and IndexedDB.

How should we choose among these storage mechanisms? I usually evaluate them along four dimensions: lifetime, capacity, synchronicity, and security.

For authentication tokens, the OWASP recommendation is to prefer HttpOnly + Secure cookies. Because JavaScript can access localStorage, an XSS compromise can expose the token directly. Some security guides recommend a hybrid pattern: keep the access token in memory and the refresh token in an HttpOnly cookie. Use localStorage for persistent, non-sensitive data that changes infrequently, and sessionStorage for data that should disappear with the tab. IndexedDB is commonly used for offline caches, large datasets, and files.

Cookies and Web Storage (local/session) store strings only. Storing an object therefore requires JSON.stringify/JSON.parse. JSON, however, has limitations.

JSON.stringify({ when: new Date() });
// → { "when": "2026-05-19T..." } — Date becomes a string
 
JSON.stringify({ map: new Map([["a", 1]]) });
// → { "map": {} } — Map is lost entirely
 
JSON.stringify({ value: undefined });
// → "{}" — the undefined field is omitted

Date becomes a string during a JSON round trip, while Map, Set, and undefined can lose data. With the default behavior, BigInt makes JSON.stringify throw a TypeError, so serialization fails entirely. Whenever you put an object into external storage, remain conscious of which types may be lost, transformed, or cause serialization to fail, and introduce a serialization adapter when necessary.

The real difficulty of External State is that React does not detect its changes automatically. Writing a value to localStorage does not cause a React component to re-render. There are generally three patterns for solving this.

• Wrap it in a custom hook (useLocalStorage) that synchronizes External State with React state. This is lightweight, but a hand-rolled implementation must handle edge cases such as multiple tabs, SSR, and tearing.
• Use the useSyncExternalStore hook introduced in React 18 to "synchronize with state outside React." This ensures that tearing does not occur during concurrent rendering. It is the standard tool for connecting React to localStorage, browser APIs, and external stores.
• Use the first-class external storage integrations provided by state libraries, such as Zustand's persist middleware or Jotai's atomWithStorage.

Here is one more important principle: the moment we bring External State into React, synchronization becomes our responsibility. What happens when another tab updates it? What if the server changes a cookie? What if a user edits localStorage directly through the browser's developer tools? These scenarios often become some of the richest sources of bugs.

State Guard

The final category is somewhat different. It is not state itself, but logic that uses a combination of state to block, allow, or validate a particular flow.

The most common example is an Auth Guard.

function ProtectedRoute({ children }) {
  const { isAuthenticated, isLoading } = useAuth();
 
  if (isLoading) return <Spinner />;
  if (!isAuthenticated) return <Navigate to="/login" replace />;
  return children;
}

Here, the isAuthenticated state controls the routing flow. That is guard logic. There are many kinds of guards: Auth Guards (authenticated or not), permission guards (specific roles or permissions), flow guards (entry-point branching), and validation guards (enabling a step), among others.

Guard logic tends to accumulate in one place. It is common for a single component to contain every condition: "if the user is not signed in, go to the login page; if they lack permission, show a 403; if the cart is empty, go to the product page; if the user is suspended, show the suspension notice." As a guard grows, it becomes harder to debug which condition blocked the flow and where.

A good guard checks only one thing. Combine guards through Composition.

<AuthGuard>
  <RoleGuard role="admin">
    <FlowGuard require={["cartHasItems"]}>
      <CheckoutPage />
    </FlowGuard>
  </RoleGuard>
</AuthGuard>

Each guard makes only one decision, while the tree structure handles composition. Adding a new guard does not require changing any existing guard.

When designing guards, what happens next—where the user is sent and how the flow continues—deserves even more thought than the act of blocking access itself. A guard that blocks without a fallback ends in a blank screen or an infinite spinner.

One of the most common bugs occurs when protected content briefly flashes before the guard's asynchronous check finishes. Validating an authentication token and fetching permissions are usually asynchronous. During that interval, isAuthenticated may temporarily be undefined or false. Unless the loading state is handled explicitly, protected content may be exposed during that gap, or the user may be redirected to the login page incorrectly.

// Ignores loading and handles only missing data => incorrect
if (!user) return <Navigate to="/login" />;
 
// Treat loading as a first-class state (early return) => correct
if (isLoading) return <Spinner />;
if (!user) return <Navigate to="/login" replace />;
return children;

Two models are commonly used when implementing permission guards.

• RBAC(Role-Based Access Control): Grants permissions by role—for example, "an admin can view all user information." It is simple and fast, but the number of roles can explode as permissions become more granular
• ABAC(Attribute-Based Access Control): Determines permissions from combinations of attributes—for example, "the user is the author of the post, belongs to the same team, or is an admin." It is highly expressive but more difficult to implement and debug

Patterns such as the TanStack Router RBAC guide, which places guards in beforeLoad at the router level, are recommended. The key is that permission checks should be expressed as data—a list of roles and permissions—rather than scattered throughout the code. That way, changing an authorization policy remains a data change.

Conclusion

Let us sum up. State Management is difficult not because the libraries are difficult, but because we often forget that state comes in different kinds and overlook the fact that each kind requires different tools and a different way of thinking.

Keep Local State as close as possible; question whether Global State truly needs to be global; treat Server State as a cache; separate forms from the domain; use the URL more actively; understand the responsibility that comes with external storage; and compose small, focused guards. These are the fundamentals of working with the seven categories.

The judgment that ties them together can ultimately be reduced to four questions.

• Where is the Single Source of Truth for this data?
• Is this a value that can be derived, or does it truly need to be stored?
• Are any combinations of this state impossible?
• Does this state really belong in this location?

I believe the surest way to develop sound judgment and intuition is to ask these questions whenever we build a new screen, review a PR, or receive code generated by AI.

As I said at the beginning, AI will remain beside us for a long time. We will spend less and less time inspecting every line. Yet that will only make the ability to answer small questions such as "Where should this state live?" more valuable. Asking AI to "add another useState here" is easy. Knowing what new strand that single line adds to the web of our application depends entirely on the judgment of the person reading the code.

There is no single right answer. But there is a clear difference between "creating state without understanding what state is" and "creating state with deliberate awareness of its category and location." Before readers write their next line of useState, I hope they will pause for a moment and ask, "Which category of state is this?"

This is for frontend developers whose business rules, such as tax calculations or filing status checks, are scattered across components, hooks, and utils, so that every rule change means hunting down several files. By the end, you will know how to move those rules into pure functions that do not depend on React, and how to judge where that separation should stop.

All the examples use the comprehensive income tax calculation, a domain I have been paying attention to.

Where Should Domain Logic Live on the Frontend?

Before answering, let us agree on one term. In this post, a domain model holds business concepts together with the rules that operate on them. It differs from a data model, such as an API response type, which only defines the shape in which data travels. This difference is also why, when components depend directly on the response structure, the whole frontend is shaken every time the backend schema changes. How the four terms (domain, domain model, domain object, and domain object model) differ is covered separately in Domain Terms.

Khalil Stemmler, who has a deep interest in software design, initially argued that "business logic does not belong on the frontend." He later revised his position, saying, "Almost everything we do architecturally on the backend can and should also be done on the frontend."

I agree with this position. Of course, the frontend must not become the Single Source of Truth for business logic. That is the backend's role. But the frontend clearly has domain logic of its own.

Consider a case where "the estimated refund must update in real time based on the information the user enters." If this calculation logic exists only on the backend, an API call must be made every time the user changes a single digit in their income. The UI pauses for a network round trip, and a fast typist can trigger a flood of unnecessary requests. Even with debounce, a delay of a few hundred milliseconds is enough to undermine the experience of a "real-time preview." Ultimately, calculations that require immediate feedback have to be performed directly on the frontend, which means some logic can only be performed there.

When Domain Logic Is Mixed into a Component

Consider a comprehensive income tax preview screen. It shows the estimated tax in real time as the user enters income information. The code below is a common example in which domain logic and UI logic are intertwined.

function TaxPreviewPage() {
  const [총수입, set총수입] = useState(0);
  const [경비율, set경비율] = useState(0.641); 
  const [인적공제대상인원, set인적공제대상인원] = useState(1); 
 
  const 종합소득금액 = 총수입 - 총수입 * 경비율;
 
  const 소득공제합계 = 인적공제대상인원 * 1_500_000;
  const 과세표준 = Math.max(0, 종합소득금액 - 소득공제합계);
 
  let calculatedTax = 0;
  if (과세표준 <= 14_000_000) {
    calculatedTax = 과세표준 * 0.06;
  } else if (과세표준 <= 50_000_000) {
    calculatedTax = 과세표준 * 0.15 - 1_260_000;
  } else if (과세표준 <= 88_000_000) {
    calculatedTax = 과세표준 * 0.24 - 5_760_000;
  } else if (과세표준 <= 150_000_000) {
    calculatedTax = 과세표준 * 0.35 - 15_440_000;
  } else {
    calculatedTax = 과세표준 * 0.38 - 19_940_000;
  }
 
  const 기납부세액 = 총수입 * 0.033;
  const refundOrPayment = 기납부세액 - calculatedTax;
 
  return <div>...</div>;
}

Can you see the problem? Business rules established by tax law—"KRW 1.5 million in personal deductions per person," "eight progressive tax brackets," and "3.3% withholding"—are embedded directly in a React component. Tax law changes every year. When such rules are scattered across components, each revision sends us hunting for every place that needs to change. And if the QA team maintains E2E scenarios, the testing cost will be substantial as well.

Eventually it becomes difficult to distinguish view logic from business logic, and the code turns into a tangle of conditionals and custom hooks.

Let Us Separate the Domain Logic

Let us borrow the central principle from Alex Bespoyasov's Clean Architecture approach: separate domain logic into pure functions that do not depend on a framework.

The domain is the core that distinguishes one application from another. You can think of the domain as something that will not change if we move from React to Angular.

The domain is the core that distinguishes one application from another. You can think of the domain as something that won't change if we move from React to Angular.

Let us refactor the tax calculation example above.

First, we define the domain types and rules, keeping related information together.

export interface Income {
  grossAmount: number;
  expenseRate: number;
}
 
export interface Deductions {
  personalCount: number;
  pensionPaid: number;
  additionalDeductions: number;
}
 
const PERSONAL_DEDUCTION_PER_PERSON = 1_500_000;
const WITHHOLDING_RATE = 0.033;
const TAX_BRACKETS = [
  { limit: 14_000_000, rate: 0.06, progressiveDeduction: 0 },
  /** ...구간들... **/
] as const;

Then we separate the domain logic into pure functions.

The logic for income calculation, deductions, tax base, tax, and refunds discussed above is extracted into a computeFullTax function. Each stage is then broken down into smaller pure functions. If the result type is inferred with ReturnType<typeof computeFullTax>, there is no need to declare a separate interface.

The component then does nothing but "use" the domain logic.

import { computeFullTax } from "../domain/tax";
 
function TaxPreviewPage() {
  const [income, setIncome] = useState<Income>({
    grossAmount: 0,
    expenseRate: 0.641,
  });
  const [deductions, setDeductions] = useState<Deductions>({
    personalCount: 1,
    pensionPaid: 0,
    additionalDeductions: 0,
  });
 
  const result = computeFullTax(income, deductions);
 
  return (
    <div>
      <IncomeForm value={income} onChange={setIncome} />
      <DeductionForm value={deductions} onChange={setDeductions} />
      <TaxResultSummary result={result} />
    </div>
  );
}

What has changed?

• The table of eight progressive tax brackets (TAX_BRACKETS) is gathered in one place, so when tax law changes, only domain/tax.ts needs to be updated.
• The calculation pipeline is cohesive within a single computeFullTax function, making the entire flow visible at a glance. (It is grouped into one function to keep the example simple, but in a real project it would be appropriate to divide it further by purpose, such as income calculation, deduction calculation, and tax calculation.)
• The component focuses only on "how to present it." A change in tax rates does not require modifying the component.
• Even if we migrate from React to another framework, domain/tax.ts does not change.

Once domain logic is separated, testing becomes surprisingly simple. This is especially important in the tax domain, where calculation accuracy directly affects the user's money.

Pure functions containing tax calculation logic need neither React Testing Library nor render nor screen.getByText. Supply an input and check the output—that is all. Cases such as "a 6% rate up to KRW 14 million," "zero tax when the tax base is zero," and "the refund on KRW 30 million of freelance income" can each be expressed as a one-line it. Domain unit tests naturally establish boundaries for component separation, while the test code also serves as documentation.

Anemic Domain Model

In the previous section, we separated calculation logic. But domain logic also includes state-transition rules and authorization decisions. Questions such as "Can this filing be edited right now?", "Can it be submitted?", and "Can the claim method be changed?" fall into these categories. Separating these rules presents an easy trap: the Anemic Domain Model, a term coined by Martin Fowler.

An Anemic Domain Model is a state in which types are well defined in the domain language, but the rules that operate on them have been scattered outside the domain. Consider the tax filing domain. The type is tidy.

// types/filing.ts
export interface TaxFiling {
  id: string;
  status: "draft" | "submitted" | "reviewing" | "completed" | "amended";
  taxYear: number;
  filingType: "regular" | "late" | "amendment";
  determinedTax: number;
}

But the rules for decisions and transitions concerning this type are buried elsewhere.

// utils/filingHelpers.ts
export function canAmendFiling(filing: TaxFiling) {
  return filing.status === "completed" && filing.filingType !== "amendment";
}
 
// components/FilingDetail.tsx
function FilingDetail({ filing }: { filing: TaxFiling }) {
  // 같은 도메인 규칙을 컴포넌트 안에 다시 작성한다
  const canEdit = filing.status === "draft" || filing.status === "reviewing";
  // ...
}
 
// hooks/useSubmitFiling.ts
export function handleSubmitFiling(filing: TaxFiling) {
  if (filing.status !== "draft") return;
  // ...
}

The same domain rule exists in three different forms across utils, a component, and a hook. If a requirement comes in saying, "The eligibility conditions for claims are changing," we must hunt down every place that needs updating, while any one we miss will make an incorrect decision somewhere on the site. Fowler criticized such code as "little more than procedural code wearing the skin of an object-oriented model."

The solution is the same one we applied to calculation logic in the previous section: put the rules next to the type.

export interface TaxFiling {
  id: string;
  status: FilingStatus;
  taxYear: number;
  filingType: FilingType;
  determinedTax: number;
}
 
export type FilingStatus =
  | "draft"
  | "submitted"
  | "reviewing"
  | "completed"
  | "amended";
 
export type FilingType = "regular" | "late" | "amendment";
 
// 도메인 규칙은 도메인 옆에 둔다
export function canEdit(filing: TaxFiling): boolean {
  return filing.status === "draft";
}
 
export function canSubmit(filing: TaxFiling): boolean {
  return filing.status === "draft" && filing.determinedTax >= 0;
}
 
export function canAmend(filing: TaxFiling): boolean {
  return filing.status === "completed" && filing.filingType !== "amendment";
}

Now every filing-related rule is managed in a single place: domain/filing.ts. Any component can call canAmend(filing), and when a rule changes, only this one file needs to be updated. The key is to treat the type and the rules that operate on it as one unit. A partial separation that puts only the type in the domain folder while moving its rules to utils may look clean, but it is still anemic.

The Translation Layer Between API Responses and the Domain Model

There is one more factor to consider in practice: the backend API response structure and the frontend domain model do not always match. This is even more true for a tax service integrated with government systems. Data from Korea's National Tax Service Hometax system is full of abbreviations and coded values, so it is unlikely to arrive in the same shape as the frontend domain model.

What we need here is a transformation layer (Mapper). Rather than passing the API response type all the way into components, we first refine it into a domain type. A single pure function is enough.

import type { Income } from "../domain/tax";
 
interface HometaxIncomeResponse {
  총수입금액: number;
  경비율: number;
  소득유형코드: string;
  // ... 나머지 약어 필드들
}
 
export function toIncome(response: HometaxIncomeResponse): Income {
  return {
    총수입_금액: response.총수입금액,
    경비_비율: response.경비율,
  };
}

This converts abbreviated fields such as 총수입금액 and 경비율, as well as code-based classifications in the API response, to suit the frontend domain in one place. Values such as the income-type code that need to be expanded into an enum can be handled with a small lookup table inside the mapper. Even if the field names in the Hometax API change, only the mapper needs to be updated.

Utility Functions and Domain Logic

When separating domain logic, one question inevitably arises: "Isn't this just a utility function?"

Consider the following two functions.

function formatCurrency(amount: number): string {
  return `${amount.toLocaleString()}원`;
}
 
function calculateTax(taxableBase: number): number {
  const bracket = TAX_BRACKETS.find((bracket) => taxableBase <= bracket.limit);
  return Math.floor(taxableBase * bracket.rate - bracket.progressiveDeduction);
}

formatCurrency is pure presentation logic that converts a number to a string. Adding the unit "won" and thousands separators is not a business rule; it concerns how a value is shown to the user. By contrast, calculateTax contains a business rule grounded in tax law: applying eight progressive tax brackets. It is a domain rule that must apply in exactly the same way even if there is no UI.

Here is the criterion I use in practice.

If this logic disappears, does the business break, or only the screen?

If the business breaks, it is domain logic. If only the screen breaks, it is presentation logic. This one question can identify most boundaries.

Reality, however, is not this tidy. The hardest cases are those that look like domain logic but are actually presentation logic.

Consider the code below. It was classified as domain logic because it takes the domain concept FilingStatus as an argument. But is it really domain logic?

// domain/filing.ts
function getStatusBadgeColor(status: FilingStatus): string {
  const colors: Record<FilingStatus, string> = {
    draft: "gray",
    submitted: "blue",
    reviewing: "yellow",
    completed: "green",
    amended: "purple",
  };
  return colors[status];
}
 
function getStatusDisplayText(status: FilingStatus): string {
  const labels: Record<FilingStatus, string> = {
    draft: "작성 중",
    submitted: "제출 완료",
    reviewing: "검토 중",
    completed: "신고 완료",
    amended: "경정청구",
  };
  return labels[status];
}

Although getStatusBadgeColor and getStatusDisplayText use the domain concept FilingStatus, what they do is screen presentation. Changing a badge color does not break the business at all. Putting such functions in domain/filing.ts makes the domain module increasingly bloated and mixes genuine domain logic with presentation logic.

Separating the Domain Model and ViewModel

There is a practical way to solve this problem: separate the ViewModel into its own file within the same domain folder. Rather than .ui.ts, using the name .viewModel.ts connects naturally to the ViewModel concept in the MVVM pattern. The name immediately communicates its role as "a layer that transforms domain data for the screen."

domains/
└── filing/
    ├── filing.ts              # 순수 도메인 모델 + 도메인 로직
    ├── filing.viewModel.ts    # ViewModel (표현 변환 계층)
    ├── filing.test.ts         # 도메인 로직 테스트
    └── filingMapper.ts        # API ↔ 도메인 변환

Move the getStatusBadgeColor and getStatusDisplayText functions we saw earlier directly into filing.viewModel.ts. Transformations that expand filing types into Korean labels, such as getFilingTypeLabel(type: FilingType): string, are gathered there as well. filing.ts is responsible only for business rules, while filing.viewModel.ts is responsible only for screen presentation.

The key is the direction of dependency. filing.viewModel.ts imports filing.ts, but filing.ts never imports filing.viewModel.ts. The domain knows nothing about presentation; presentation knows about the domain. This can be seen as a miniature version of Robert C. Martin's Dependency Rule.

I placed the files in the same folder because I believe files that change together should live in the same directory. If the FilingStatus type gains a new value (such as 'rejected'), both filing.ts and filing.viewModel.ts must be updated. Since they are in the same folder, the scope of the change is visible at a glance.

Boundaries and Cohesion

Just as important as separating domain logic is deciding where to draw the boundaries. Here are several boundary decisions I often encounter in practice.

The data handled by a frontend comes from roughly four sources.

• Server data: received in an API response
• Derived data: calculated from server data
• UI state: used to control the screen and reflect user interactions
• User input: values currently being entered in a form

Combining all four in one type pollutes the domain model.

// 안티패턴: 모든 것이 섞인 타입
interface TaxFiling {
  // 서버 데이터 (도메인)
  id: string;
  status: FilingStatus;
  determinedTax: number;
 
  // 파생 데이터 (도메인)
  refundAmount: number;
  canAmend: boolean;
 
  // UI 상태 (표현)
  isExpanded: boolean;
  activeStep: number;
 
  // 임시 상태
  editingDeductions: Deduction[];
}

This type puts domain concepts, UI state, and temporary data in one basket. Every change to activeStep effectively updates the Filing domain. (Changing a form step is not a business event.)

The improvement is to divide the types along their boundaries. The domain model contains only business concepts such as id, status, and determinedTax; UI state (FilingFormViewState) contains only screen controls such as isExpanded and activeStep; and form state (DeductionEditForm) contains only temporary data being entered.

This gives each type a single reason to change. A domain type changes only when tax law changes, UI state only when the screen design changes, and form state only when the input UX changes.

Keep Things That Change Together Together

Eric Evans's DDD includes the concept of an Aggregate: "a cluster of related objects treated as a single unit." We do not need to apply this concept wholesale on the frontend, but its central principle—keep data and rules that change together together—is worth borrowing.

In a tax service, for example, Income and ExpenseRate always change together. When the income type changes, the applicable expense rate changes, and the comprehensive income calculation is affected as well. These should therefore be kept together in one file, domain/tax.ts.

By contrast, TaxFiling can change independently of tax calculation. A change to the filing's state-transition rules does not affect tax-rate calculation logic. It is therefore appropriate to separate it into domain/filing.ts.

이렇게 묻자: "A가 변할 때 B도 반드시 변해야 하는가?"
  → Yes: 같은 모듈에 둔다 (Income + ExpenseRate + TaxBracket)
  → No: 분리한다 (Tax 계산 ↔ Filing 상태관리)

Class vs. Functional Style

At this point, a fundamental question may come to mind. Every example so far has used a combination of interface and pure functions. Would cohesion be more natural if we represented the domain with a class?

That is a fair point. A class-based representation of the domain binds data and behavior into a single object, making cohesion immediately visible in the code structure.

class TaxFilingModel {
  constructor(
    public readonly id: string,
    public readonly status: FilingStatus,
    public readonly taxYear: number,
    public readonly filingType: FilingType,
    public readonly determinedTax: number,
  ) {}
 
  canEdit(): boolean {
    return this.status === "draft";
  }
 
  canAmend(): boolean {
    return this.status === "completed" && this.filingType !== "amendment";
  }
 
  canSubmit(): boolean {
    return this.status === "draft" && this.determinedTax >= 0;
  }
}
 
const filing = new TaxFilingModel(
  "F-001",
  "completed",
  2025,
  "regular",
  547200,
);
 
filing.canAmend();

With a class-based approach, behavior belongs to the data. The subject is also explicit at the call site. filing.canAmend() is as intuitive as reading natural language: the subject (filing) and verb (canAmend) are clearly bound together. It is like writing jihoon.eat('감자탕') and immediately reading, "Jihoon eats gamjatang."

By contrast, the functional style looks like this.

canAmend(filing);
eat("jihoon", "감자탕");

In the functional style, the data exists outside the function. The first call takes filing as an argument and performs an action, while eat takes jihoon and 감자탕 as its arguments.

As a result, the bond between subject and verb is looser. To know that the canAmend function concerns TaxFiling, we have to open the file or inspect its type signature. If functions such as canAmend(filing), canEdit(filing), and calculateTax(taxableBase) are mixed in the same file, it may be difficult to see at a glance which domain each function belongs to.

So Should We Use Classes?

Honestly, the answer is "it depends." In my experience, however, there are practical reasons classes are not a silver bullet in a React + TypeScript environment.

Friction with React State Management

React state management fits most naturally with plain objects. useState and useReducer can technically hold any value, and Redux DevTools does not itself remove a class instance's prototype. If Redux/Zustand persistence middleware stores and restores state as JSON, however, a class instance loses its methods and prototype in the JSON.stringify → JSON.parse cycle and becomes a plain object. The props boundary from a React Server Component to a Client Component has a different constraint: it accepts only supported serializable values, so an arbitrary class instance cannot be passed through it in the first place.

Consider the code below.

const [filing, setFiling] = useState(
  new TaxFilingModel("F-001", "draft", 2025, "regular", 0),
);

Updating React state alone does not stop filing from being an instance of TaxFilingModel. If Redux/Zustand persistence saves and restores it as JSON, however, the restored value may be a plain object without methods, so an innocent call to filing.canAmend() can produce a runtime error. Passing it from a React Server Component to a Client Component fails earlier because a class instance is not a supported serializable prop value.

The Difficulty of Guaranteeing Immutability

React detects state changes based on referential equality. If a method on a class instance mutates internal state with something like this.items.push(...), the reference stays the same and React does not trigger a rerender. In the end, addDeduction(item) has to return a new instance every time—something like return new DeductionList([...this.items, item]). That undermines the class advantage of "encapsulated state mutation" and leaves code that is not very different from functional updates.

Strategies for Achieving Cohesion in a Functional Style

Then how can we improve the loose cohesion of code such as eat('jihoon', '감자탕') in a functional style? Here are three approaches I have found effective.

Create Cohesion with a Module Namespace

This is the most straightforward approach. Make the file (module) itself correspond to a domain, and use a namespace when importing it. We can use the domain/filing.ts defined earlier as-is.