Operating at the Speed of Inference

Projects move fast when decisions are made early enough for agents to execute in parallel without constant interruptions.

I've started thinking about projects as moving between two states.

The first is decision mode. The direction is unclear, tradeoffs are unresolved, or somebody needs to approve the next step. Work is mostly interfacing with people.

The second is execution mode. Enough decisions have been made that people can hand work to agents, run several streams in parallel, and let each agent inspect, act, test, and revise without waiting for permission.

That second state runs at the speed of inference. The problem is that most projects keep falling out of it.

Stop-start work is the real bottleneck

Imagine an agent starts a feature, works for 20 minutes, then asks whether the new flow should replace the old one. It waits. Someone answers, so it continues until it needs approval for the schema change. It waits again. Then design needs to confirm an edge case.

The agent may be fast, but the project is not. Its real speed is set by the gaps between answers.

This gets worse with parallel agents. Ten agents do not create ten times the output if they all converge on the same unresolved product decision. They create ten ways of discovering that nobody decided it.

That is why unclear and deferred decisions are so expensive now. They do more than delay one task. They prevent work from fanning out. A whole project stays serialized behind the availability of a few people.

Same human input, placed differently: decisions unlock work instead of interrupting it.

Modern execution is a loop

Not every implementation detail needs to be decided upfront. Modern agents are more than fast typists waiting for instructions.

Modern agents run loops:

Inspect → plan → act → test → observe → revise

They can resolve local ambiguity by reading the codebase, trying an approach, running tests, inspecting the browser, checking logs, or asking another agent to review the result. OpenAI has described single Codex runs working for more than six hours, with agents reviewing one another and iterating until they are satisfied. Google DeepMind's AlphaEvolve generates programs, scores them with automated evaluators, and feeds the best results into the next round.

So the goal is not to plan every action. It is to settle the decisions that an execution loop cannot settle for you: what outcome matters, which constraints are real, who owns the tradeoff, and where the agent must stop.

Once those are clear, the loop can run without a person approving every turn.

Decisions unlock parallelism

Take a project to improve onboarding. If the brief is just "reduce drop-off," every stream quickly runs into the same questions. Which users are we optimizing for? What metric counts as success? Can the flow change for existing customers? Is adding a new verification step acceptable?

You could start agents anyway. One maps the funnel, another prototypes a flow, another changes the backend. But each deferred answer can invalidate work already underway.

Make the blocking decisions first. Pick the target users and success metric. Set the constraints and name the owner. Now one agent can implement the flow while another handles instrumentation, another tests edge cases, and a reviewer agent checks the combined result.

The decision does not merely unblock the next task. It lets several tasks start.

Get into inference mode and stay there

The fastest teams will get good at creating long, uninterrupted stretches of execution:

  • Front-load decisions that affect every downstream stream of work.
  • Delegate reversible decisions instead of escalating them.
  • Replace repeated approvals with boundaries for what agents may change, ship, or decide themselves.
  • Put human checkpoints where parallel work recombines. When a new decision surfaces, make it quickly and return the project to execution mode.

This is about using people at the right frequency. They should decide direction and resolve new tradeoffs, not act as a synchronous approval API for work an agent can verify itself.

I suspect a useful project metric now is simply: how much time can this work continue without waiting for a person?

If the answer is 20 minutes, a faster model will not save you. If the answer is a day, and ten agents can use that day in parallel, the project is operating at the speed of inference.

Make the decisions that fan work out. Then let it run.

Tilo Mitra

Tilo Mitra

@tilomitra

I'm a software engineer and engineering manager living in Toronto, Canada. I currently work at Square. Read more »