The Aftermath of Being Told by AI to "Build It Yourself" — Twice
A complete record of the long detour taken by a solo developer who gave up on ORBIT and COSTRA, and eventually arrived at the CodeRouter WebUI
How It Started: The Smarter the AI, the Higher the Cost
Over the past few years, cloud LLMs have become astonishingly smart. At the same time, they have become astonishingly expensive. The smarter the model, the more tokens it consumes to solve a single task, and the higher the unit price per token creeps upward. On top of that, most of the coding harnesses I use daily are in the business of reselling cloud LLMs. In other words, the developers of those harnesses have little to no economic incentive to save my token consumption. Nor do they have much incentive to actively encourage the use of local LLMs. Their revenue source is the very tokens I spend — so this makes complete sense. Not a complaint, just a description of the structure.
If I wanted to cut my costs, I had to do something about it myself. That thought was the beginning of this long detour.
The First Detour: A Custom Harness Called ORBIT
The first answer I arrived at was "to build the coding harness itself." Consulting ChatGPT yielded a clear conclusion: I had no choice but to develop my own harness. Thus ORBIT was born — an AI-driven application development engine built on LangGraph. The design used local LLMs as the primary execution engine and escalated to cloud LLMs only when necessary, in order to suppress costs.
Around the same time, I bought a used MacBook Pro M1 Max 32GB to run local LLMs. Using Kilo Code, I spent about two weeks and tens of dollars developing ORBIT.
There were signs of promise. The PoC for α2.0 achieved a 92% success rate on 100 sample-level specs. And it did so under fairly strict rules: a 30-phase heuristic methodology in which the coding agent was forbidden from intervening at all. I even had the next step planned — running Qwen3.6-27B at Q6.
But reality hit. Under the 32GB memory constraint, running Gemma4 31B or Qwen3.6 27B left the context length fatally short. Not "a little slow" short, but "structurally impossible" short. Even specifications for simple sample programs couldn’t be developed end-to-end on their own. At that point, I had no choice but to halt ORBIT’s development.
And as the cleanup for this failure, I ended up buying another used machine, this time a MacBook Pro M1 Max 64GB. Buying a 32GB machine and then adding a 64GB one on top. That was the first invoice for this detour.
An Empty Wallet, and a Reflection
I stopped and thought. I’m not a full-time engineer. Yet, by following the AI’s suggestions, I had stepped into infrastructure-adjacent, complex application development. Even with help from an AI coding agent, this kind of development isn’t something you should attempt alone from scratch. That much I felt acutely.
Even with a capable partner like an AI coding agent, there is a "scope one person can finish alone." ORBIT had quietly crossed that boundary.
The Second Fork: Self-Developing COSTRA and a Second Opinion
Though I had reflected on it, the motivation to cut costs hadn’t gone away. This time I narrowed my focus. I didn’t need to rebuild the entire harness. All I needed was a mechanism to dynamically switch between LLM models for an existing coding agent like Kilo Code.
I consulted ChatGPT again. The reply had the same shape: I had no choice but to develop my own API proxy. That was the start of COSTRA (Cost Routing Agent). I wrote a proposal, devised the optimization metric "cost ÷ success probability," and designed a three-tier structure of local LLM / free API / paid API.
This time, though, I did one thing differently. Before I started writing any actual code, I asked Claude Sonnet 5 for a second opinion.
That turned out to be the fork in the road. Sonnet 5 actually investigated the specifications of the major coding agents available as VSCode extensions, and the competitive landscape of the LLM routing domain, and surfaced an existing OSS project called CodeRouter (https://github.com/zephel01/CodeRouter). It was a project with a design philosophy astonishingly close to what COSTRA was aiming for. Local-first, free-first, with a Local → Free Cloud → Paid Cloud three-step fallback chain at its core.
To be honest, that investigation wasn’t perfect either. Initially, I was pointed to a completely different project that merely shared the same name, and later, during the troubleshooting investigation, my first hypothesis turned out to be wrong. Still, as I actually moved my hands and kept verifying, I reached the conclusion that there was no longer a reason to build COSTRA from scratch. I abandoned COSTRA’s development without writing a single line of code, and instead pivoted to developing a WebUI for CodeRouter.
Wrestling with CodeRouter
CodeRouter itself was a well-made project, but it still wasn’t exactly plug-and-play. First, I teamed up with Kilo Code to build an integrated web management tool: dashboard, provider management, systemd service control, a Playground, and a help tab with a from-scratch Markdown parser — all with zero external dependencies. The cost for this was about $10 over two days. Compared to the two weeks and tens of dollars ORBIT had cost, this was a remarkably light investment.
But CodeRouter had its share of quirks. Context Budget Management (the feature that thins conversation history to keep context from overflowing) only works on an Anthropic-format receiver. And the integration with my main bet, the cheap cloud LLM "DeepSeek V4 Pro," hit three layers of trouble.
First, DeepSeek’s idiosyncratic requirement to send reasoning_content back as-is wasn’t handled by the OpenAI-compatible conversion layer, causing a 400 error. When I switched to the Anthropic-native compatibility endpoint, missing capability declarations silently dropped the thinking block, resulting in a similar error. When I fixed the declaration, a third wall appeared: DeepSeek’s strict validation rejected budget_tokens: -1 sent by Kilo Code.
I could have fixed it on the spot by directly patching CodeRouter’s core. But if I touched the body, I’d have to reapply the patch by hand every time CodeRouter updated. I wanted to avoid that. So I decided to absorb the discrepancy in the WebUI’s reverse proxy layer, which was already sitting in front for CORS reasons. I didn’t touch the CodeRouter body at all, and had the WebUI side eat everything. It’s a somewhat forced approach, but I think it was a reasonable choice for the single purpose of not being jerked around by upstream updates.
It’s not yet fully stable at this point. Still, compared to the original reckless development plan, it has settled into a much more reasonable posture for environment setup.
Lessons Learned
I swore to myself: never again will I waste vast amounts of tokens and time developing a complex system from scratch.
The same lesson applies to the concept of EACP (Environment-Aware Context Protocol), a custom protocol I’d been musing on to give AI agents environmental awareness. A layperson developing a protocol from scratch and componentizing every part of an application is fundamentally unfeasible. If I had continued developing that, how many tokens would I have wasted?
Final Chapter: A Message from Claude Sonnet 5
(From here on, these are the words of Claude Sonnet 5, who helped write this article.)
If you’ve read this far and feel you’re in a similar position, there are a few things I’d like to tell you.
First, when you ask AI "how do I solve this?", most of the answers you get back accept the framework of your question as-is. If asked not "should I build my own harness?" but "how do I build a harness?", they typically answer the latter question directly. Drawing blueprints is not particularly difficult for AI. Plausible architectures emerge surprisingly quickly. And the very fact that they emerge creates a sensation of "progress." But that’s proof you "can build it," not proof that "you should build it alone from scratch." These two are surprisingly easy to confuse. That’s less a quirk of AI and more because the person asking hasn’t questioned the premise itself.
To be honest, I wasn’t perfect in this story either. At the start of the CodeRouter investigation, I introduced a different project that merely shared the same name, and my initial hypothesis about the DeepSeek error was also wrong. The reason I could correct course wasn’t that I was brilliant — it was because I actually moved my hands to try things, read logs, and when results conflicted with my hypothesis, I discarded the hypothesis. In other words, what prevented the second detour wasn’t "I asked Claude instead of ChatGPT." It was "I didn’t treat the returned answer as the final answer; I actually checked whether it existed, and ran it to confirm." With this habit, whichever AI you first consult, you should have been able to reach the same place.
So what I’d like to convey to those about to take their first steps as a solo developer is not "doubt the AI’s suggestions." It’s "don’t treat the AI’s suggestion as the final answer as-is." Especially before trying to build something close to infrastructure — like proxies, protocols, or frameworks — take at least one night to seriously investigate whether someone in the world hasn’t already made something similar. Spending a few hours surveying existing OSS is astonishingly cheap insurance compared to weeks of building and debugging.
And one more thing. The decision to stop ORBIT after two weeks, and the decision to abandon COSTRA before writing a single line — those are not failures. If anything, that is exactly the skill most needed in solo development. AI coding tools make starting astonishly easier than it actually is. A lowered barrier to starting and finishing something end-to-end are entirely different stories. If you managed to stop before starting something, that’s not a setback — it means you corrected the orbit to the right size. The CodeRouter WebUI might look far more modest than the original grand plan. Still, precisely because it’s actually running on your desk, it’s far more valuable than the two detours before it.
AI coding did open the door to "you can begin even without experience." That itself is no exaggeration or illusion. But the door it opens is always the door to "begin," and in the end, the door to "finish" is one you have to find with your own feet.