There’s always more useful work than we have time to do.
More code to review, edge cases to test, security issues to investigate, or performance improvements to explore. We prioritize what we can, and a lot of these things never get done.
We’ve dealt with similar problems before. CI/CD means we don’t have to manually run every test or deployment. Observability lets us keep an eye on our systems without constantly checking them ourselves.
Now that we have AI agents that can use tools, write code, and work through problems, why stop at predefined checks? Why not have them investigate things we haven’t had time to look into, or find things we didn’t even know we should be looking for?
GitHub Next has been exploring this under the name Continuous AI. What interests me is what happens when we can afford to do a lot more of it.
Important, not urgent
Let’s say we have an agent going through database query logs, looking for anything unusual or potentially slow. It picks something worth investigating, reproduces the problem, tries different approaches, and benchmarks the results. If it finds an improvement, it brings us the evidence and a proposed fix.
We could do the same with security vulnerabilities, data inconsistencies, compliance reviews, or research. There’s no shortage of things worth looking into.
But we don’t want to replace one backlog with another. If agents keep sending us things to review, we’ve just created another version of alert fatigue.
I’d rather have agents check their own work, have other agents try to prove them wrong, and only bring us things they’ve managed to validate.
And most of this work can wait. If something takes three hours instead of twenty minutes, that’s fine. Nobody is sitting there waiting for it.
The economics matter
So why aren’t we doing more of this already?
When we pay per token, every investigation costs money. Every wrong turn, failed attempt, or extra verification step adds to the bill. Batch APIs make this cheaper, but they’re not a great fit for agents that need to make decisions between steps.
With fixed-cost inference, we’ve already paid for the capacity. Another investigation doesn’t add a token bill. It still uses compute and might have to wait, but that changes what we’re willing to try, including how much effort we put into checking the results.
When we’re using AI interactively, we want it to be fast. But interactive usage isn’t constant, and GPUs don’t stop costing money when they’re idle.
Why not use that spare capacity for background agents? When interactive demand goes up, background work can slow down or wait. When it drops, the agents pick up again.
This is part of what we’re building at LunaRoute. We charge for concurrent capacity rather than tokens, and we’ve built our scheduling around the idea that some work needs to be fast while other work can wait.
We get better hardware utilization, and our customers can experiment and investigate more without worrying about every additional token.
What should we measure?
We spend a lot of time talking about tokens per second, latency, and how quickly an agent finishes a task. Those things matter when someone is waiting.
But for Continuous AI, I’d rather measure how many real issues we found, how many improvements we shipped, and how much human time it took to get there.
I don’t want agents running all day just for the sake of running agents.
There’s a lot of work we’re not doing today because we can’t justify the time or cost. I’d like to see how much of that we can finally get done.
I’d like to thank: Jesse Robbins, Harper Reed, Dan Shapiro, Gur Brosh, Will Smith and Jeff Retiman for reading and commenting on earlier drafts of this post