Latency budgets for teams without an SRE
You do not need error budgets and a reliability org to reason about latency. You need to know what you are spending and where.
Most of our customers are teams of three to twelve people. They do not have a reliability engineer and are not going to acquire one. The literature on latency is largely written for organisations that do, which makes it less useful than it should be.
Here is the version that fits on one page.
Start with the promise
Pick the user-facing thing that matters — a page load, a checkout, a search — and decide what you are willing to promise. Not an average: a number you are willing to be held to most of the time. Six hundred milliseconds at the ninety-fifth percentile is a real promise. "Fast" is not.
That number is your budget. Everything after this is spending it.
Write down what you are spending
List every network call in the path, with its p95, not its median. Add them up if they are sequential. Take the maximum if they are parallel, then add a little, because parallel calls are only as fast as the slowest and something is always slower than you think.
Most teams doing this for the first time find they are already over budget, and that one call they had not thought about is responsible for a third of it.
The three moves
There are only three things you can do with a call that costs too much.
Remove it from the path — do the work in the background and show the user something optimistic. Make it concurrent with something else already happening. Or cache it, if the data tolerates being slightly old, which more data does than people admit.
Making the call itself faster is usually not available to you, because it belongs to someone else. That is why we publish p95 and p99 per API rather than a single flattering median: you cannot budget against a number that hides the tail.
Set a timeout that means something
A timeout longer than your budget is not a timeout, it is a formality. If the promise is six hundred milliseconds and the call has a thirty-second timeout, you have decided that a slow call will break the promise rather than fail. Choose the failure. Timeout at the budget and have a fallback.
The fallback is the part teams skip, and it is the part that turns a slow dependency into a degraded feature rather than a broken page.