Compare Things, Not Designs

The first part of this series ended with a claim: when two designs feel equally good, the fact that separates them has not been found, and it is not in the question. This part is about where it is, and about the habit that finds it.

The habit is simple to state and slow to build. You cannot compare two designs. A design is too big to have a property. What you can compare is two answers to the same question about one thing. “Under option A, what is the request body? Under option B, what is the request body?” That is a comparison. “Which option is more robust?” is not, and never will be, no matter how long you think about it.

So the work is to find the things. I will call them items, because they need a name and because “thing” is what people say when they have not looked. An item is a noun you can point at in the code or in the system: a value, a field, an error, a lock, a thread, a person who gets paged. The set of items where the two options differ is the whole material of the decision. Everything else is a tie by construction and can be ignored.

Start with the verb

Write each option in one line. Not a paragraph. One line, with a verb in it.

A. The client sends the request and, on failure, retries it.

B. The client sends the request and returns the failure; the caller decides whether to retry.

The verbs are sends, retries, returns, decides. Sends is in both and can wait. Retries is the one the argument is about, so it goes first. Notice that under B it did not disappear. It changed hands.

Now expand the verb into nouns. A verb hides a lot. “Retry” sounds like one thing, and it is at least seven, because every action has an input, an output, a way to fail, something it waits on, something it holds while it waits, something that has to exist before it can start, and something that happens right after it finishes. Those are the slots. Fill them in, for each option, reading the list rather than trusting yourself to remember it, because the one you forget is the one that matters.

One more thing before filling them. There is more than one participant in this design, and each has its own verb. The client’s verb is “retry.” The caller’s verb is “call the client.” The test that exercises the client has a verb too: “build a client and make it fail.” Each participant stands in a different place and sees different nouns, so the slots get filled once per participant. Start with the client, since that is what the question is about.

Here is “retry” under option A, where the client does it.

Input. The request. The result of the previous attempt. A policy saying how many times and how long to wait.
Output. A response, or a final error after the policy is exhausted.
Error, and who names it. The client has to define “retries exhausted” as an error type. The individual attempt errors are consumed inside the loop; the caller never sees them.
What it waits on. The backoff sleep, then the network again.
What it holds while it waits. The request body, because it has to send it again. The caller’s thread or task, which is blocked or suspended inside the client. The clock: time is passing against whatever deadline the caller had.
What must exist first. A policy. A body that can be sent twice, which rules out a stream that has already been read. A rule for which failures are worth retrying. Some idea of whether sending this request twice is safe.
What happens right after. On the client side, it returns. On the caller side, the caller inspects the result, and may have a timeout of its own that expired two retries ago.

And under option B, where the client returns the failure.

Input. The request.
Output. A response, or the error from this one attempt.
Error. Whatever the network or the server said, named by the transport, passed through.
Waits on. One round trip.
Holds meanwhile. The body, once. The caller’s thread, once.
Must exist first. A request.
After, caller side. The caller decides what to do with the failure, using whatever it knows: what the request meant, how much time is left, whether anyone is still waiting for the answer.

That is the client’s view. Now stand where the caller stands and expand its verb, “call the client,” under option A.

Input. The request, and a client that somebody else configured with a policy the caller did not choose.
Output. A response, or a retries-exhausted error the caller now has to handle.
Error, and who names it. The client named it. The caller has to map it onto whatever its own errors look like.
What it waits on. The client, for a length of time the caller cannot bound from where it sits.
What it holds while it waits. Its own deadline, ticking. Its own caller, waiting. Possibly a database transaction it opened before making the call.
What must exist first. A client whose retry policy was decided at construction, by whoever constructed it.
What happens right after. The caller reacts to a result that may have arrived after its own deadline passed.

Compare the two lists. The open transaction, the policy chosen by someone else, the wait with no bound: none of those nouns appear in the client’s expansion, because from inside the client they are not visible. They are only visible from the caller. That is what it means to say each participant sees different nouns, and it is why the expansion is done once per participant and not once per option.

The test is the third participant, and it is worth a moment because it is easy to dismiss as a detail. The test is a caller with no privileges. It cannot rely on a real server or a real clock, so it has to construct, by hand, everything the client silently depends on. Under A, the client’s test has a “what must exist first” slot that reads: a fake server that fails on demand, and a fake clock, or the test sleeps through every backoff and takes thirty seconds. Under B the client’s test needs only a fake server, because the client no longer waits for anything. The clock did not go away. It moved with the verb, into every caller’s test, one fake clock per caller that bothered to write one. The list of things a test has to fake is a list of the component’s hidden dependencies, written down by someone who had no choice but to find them. That is why the test gets its own expansion.

Slots are questions, items are answers

I want to be precise about this because it is the step people skip. The slots are not the items. Input is not an item; nothing in the code is called “input.” The items are the nouns you wrote into the slots: the request body, the policy, the retries-exhausted error, the backoff sleep, the caller’s deadline, the fake clock. Those are things. You can find them in a file, or notice that a file is missing them.

Now the comparison, which is one line per item with two blanks:

under A, the ___ is ___ ; under B, the ___ is ___

Fill both blanks for every noun you found. Three things can happen.

The answers differ. Keep the item. “Under A, the request body must be buffered so it can be sent again; under B, the body can be a stream.” That is a real difference, and it is already pointing somewhere: at the size of the bodies, and at the memory budget.

The answers are the same. Strike it. “Under A, the TCP connection is opened by the client; under B, the TCP connection is opened by the client.” Both options do it the same way. It is not part of this decision, however important it is in general.

One side has no ordinary answer. This is the case to watch, because it looks like the second case and it is the opposite. “Under A, whether this request is safe to repeat is ___.” The honest answer is: not known to the client. The client sees a method and a URL and some bytes. Whether the operation behind them can be applied twice is a fact about the application, and it lives in the code that built the request, one layer up. Under B, the same blank fills in easily: the caller knows, because the caller wrote the request. That is not “no difference.” That is the largest difference in the set, and it is the one that gets struck by mistake, because “the client does not know” does not sound like an answer. It is an answer. “Cannot see,” “impossible,” “has no place to do it,” “must guess” are all answers, and in my experience they are the answer more often than not.

Here is what the surviving set looks like for retries, with the two blanks written out.

Whether the request is safe to repeat. A: unknown to the client. B: known to the caller.
The caller’s deadline. A: invisible to the retry loop, which keeps going after it has passed. B: in the caller’s hand.
The request body. A: must be replayable, so buffered. B: may be a stream.
The rule for which failures to retry. A: written once, in the client, in the transport’s vocabulary (connection reset, 503). B: written in every caller, in whatever vocabulary each caller has.
The retries-exhausted error. A: a type the client must define and every caller must handle. B: does not exist.
The backoff schedule. A: paced by the client’s clock. B: paced by whoever writes the loop.
What the test needs. A: a failing fake server and a fake clock. B: a failing fake server.

Seven items. The argument in part one had zero. That is the difference between two engineers and a decision.

What is not an item

The list above is built out of nouns you can point at. A few things look like items and are not, and it is worth being able to recognize them, because they are what people reach for when they have not done the expansion.

The client. A whole component is not an item. If you put its name in the first blank, look at what you wrote in the second. “Under A, the client is configured with a retry policy at construction.” The noun in that answer, the policy, is the item. Move it to the first blank and ask again: “under A, the retry policy is a constructor argument; under B, it does not exist.” Now both sides have an answer, and the comparison is about one thing.

The retry. The verb in the question is a tie by construction. Both options are about retrying. The item is never the verb. It is next to the verb.

Coupling. Or robustness, or simplicity. Qualities are verdicts. “Under A, coupling is higher” is the argument from part one wearing a slightly better coat.

Both need the URL. A shared requirement is struck by definition. It is true, and it is not a difference.

The policy. Watch this one. “The policy object” is option A restated: it exists only under A, so of course it differs. The items are what the policy drags in with it: a constructor argument every caller must supply, a body that has to be replayable, a clock the client now depends on, a fake clock the test now needs.

Look outside the question

The items so far came from expanding the verb. There is a second place to look, and it is where the largest surprises live: outside the boundary of the thing you are designing.

For each surviving item, ask two questions. Who else touches it? Not just code: the tests, the person reading logs at three in the morning, the engineer who inherits this in two years. And what happens immediately before it and immediately after it? “After” is where the caller’s deadline came from. It was not in the retry loop. It was in what the caller was doing while the loop ran.

Then walk the perimeter. Every component has one. It is the set of contracts the component takes part in but does not own: the things that can change without asking you. A change to your design will arrive through one of them, and the question to ask at each is: if this changed tomorrow, which files would I edit? If the two options give different answers, that edit set is an item, and it belongs in the list.

You do not have to imagine the perimeter. It can be read off the code in four passes.

Imports and dependencies. Every library, every system call, every RFC number the code mentions is a contract someone else controls.
Types that cross a public boundary. Every parameter and return type on a public function is shared with a caller you do not own. Every byte format you read or write is shared with a peer.
Reads of the environment. Configuration, environment variables, files, sockets, the clock, the operating system.
Readers of your output. The next component in the chain. The log. The dashboard. The person paged when the dashboard goes red.

Whatever the passes find will fall into six groups, and the groups are a check on completeness: if one is empty, look again.

Above. Whoever calls you, and the contract you offer them.
Below. Whatever you call, and the contract it offers you.
Beside. Other instances of you (topology, scale). Peers at the same level.
Data. The shape of what crosses each boundary, in and out.
Environment. Where and how it runs: the runtime, the resource budget, the threading model.
People. The operator, the maintainer, the tester, the attacker.

Time cuts across all six. Every contract has versions, and old clients, old data, and restarts are all ways the past reaches into the present.

The perimeter of the HTTP client, read off this way:

Above. A request handler serving a user who will wait two seconds. A batch job that will wait all night. A command-line tool run by a person watching it. Three callers with three different ideas of what a failure costs.
Below. The server, which has its own opinion about being retried and may say so in a Retry-After header. The network. DNS.
Beside. The other four hundred copies of this client, one per pod, all seeing the same failure at the same moment and all consulting the same backoff table.
Data. The request body, which is a few hundred bytes today and a file upload next quarter. The error type.
Environment. Whether the runtime is blocking or asynchronous, which decides what “sleep for the backoff” costs.
People. The operator who watches traffic to the server triple during the outage that took it down, and cannot tell from the graph whether that is users or retries.

Two more items fall out of this walk, and they are the ones neither engineer mentioned.

The other instances. A: four hundred retry loops with no knowledge of each other. B: four hundred callers with no knowledge of each other. The same on the surface. But the rule is in one place under A and in four hundred places under B, and a rule in one place can be given a budget.
The server’s own signal. A: Retry-After is honored in one file. B: it is honored in whichever callers thought of it.

Notice that these two point the other way from the first two. Safe-to-repeat and the deadline are things the caller knows and the client cannot see. The retry budget and the server’s signal are things that only make sense in one place, and the client is that place. The candidate set has not resolved the argument. It has shown that the argument was about two different things, and each engineer was looking at one of them.

When you are done

There is a completion test, and it is not “I filled in every slot.” It is this: every constraint that was stated in the problem must land on an item. Someone said “every service in the company will use it.” Which item is that? The other instances, and the callers above. Someone said “callers should not have to think about the network.” Which item? The rule for which failures to retry. If a constraint lands nowhere, an item is missing, and the place to look for it is the slot you filled in fastest.

The set for retries has nine items. Some pull toward the client, some toward the caller, and one of them, the request body, does not care yet and is waiting for a number. The third part of this series is about what to ask each of them, and about how to tell a real answer from a plausible one.