Why I went fully local: from cloud AI to a sovereign lab at home

On Friday, 24 October 2026, at about ten past five in the afternoon UTC, I submitted my website to Hacker News. The title said what I had spent the year building: “A sovereign AI lab on one machine in my apartment – zero cloud”. Four hours later the post had 2 points. It never left the “new” page. My first comment under it was killed automatically, because my account was three minutes old.

The machines were more interested than the people. In those same four hours the site answered 2,749 requests from 1,532 addresses. About 35 of them were real people arriving from Hacker News. The rest included eight different AI crawlers, a proxy fleet hot-linking my preview image, and dozens of link previews unfurling in chat apps I will never see.

So the big public moment was a quiet one. That is fine. This post is not about the launch. It is about why there was anything to launch: why one person in a small apartment in Iași decided that the AI he uses every day should run on his own hardware, under his own roof, and answer to nobody else.

Contents · 11 sections

In one minute #

• In April 2026 my AI setup was a mix: local models running slowly on the CPU, plus cloud models proxied through the same chat window.
• By October 2026 everything that runs, stores or decides in the lab is local: chat, agents, embeddings, the library’s text analysis, the study guides, the security classifiers.
• The push came from three directions: scale (a library of about 120,000 works), quality (a local test beat the cloud results I had stored), and a long-held dislike of depending on things I cannot inspect.
• The price was real: a 32 GB GPU, months of slow batch jobs, a few nights of debugging, and accepting that some tasks are still beyond local models.
• One honest exception remains, and I will name it below.

The before: a chat window with two kinds of brain #

The oldest entry in my lab’s record about local AI is dated 2 April 2026. It is a configuration scan, and its first sentence is blunt: “Ollama runs on CPU, not GPU.” A small governor service scaled the CPU threads up and down. A cron job pinged one model every 20 minutes and others every four hours, just to keep them warm in memory, so the first question of the day would not take a minute to answer.

The same day’s notes list the models available in my chat interface. Fast, medium and heavy local models. Three embedding models. And, at the bottom, almost as an afterthought: “Also proxied: gpt-4.1, gpt-4.1-mini, gpt-3.5-turbo, gpt-4”. The cloud was sitting in the same dropdown as my own models, one click away.

My first agent, lsn-agent v1, was described that week as “Ollama-powered, 8 skills, 192 MCP tools”, with nine specialist personas: a scout, a coder, an ops engineer, a security analyst and so on. It was ambitious and a little theatrical, which is how most first versions are.

None of this was wrong. It was a sensible hybrid. The local models were private but slow; the cloud models were quick and capable. If you run a home lab you probably know this arrangement. You keep the cloud for the hard questions and feel vaguely good about the local ones.

What bothered me was not the quality. It was the shape of the dependency.

Why it bothered me #

I should be careful here, because it is easy to write a manifesto after the fact. So here is what the record actually holds.

Everything in this lab runs on one workstation that I own, on my own electricity, under my own domain. The lab’s own notes say the Kubernetes cluster has been running since early 2025. The machine was there first. The AI came to it, not the other way round.

On my personal blog, written mostly in Romanian, I once titled a short piece “Sculă sau blestem?”, tool or curse, about AI. It was about the convenience and the dependency, side by side, and I did not pretend to resolve it. In another short post, about the day Google went down, my response was to make tea. The tagline of that blog is “my space where chaos makes some sense”.

Put those together and you get the motive, as plainly as I can say it. I trust experience more than institutions, and I do not like building on ground that someone else can move. When a service I depend on changes its price, its rules or its mind, I want the option of shrugging and making tea.

A cloud model in the dropdown is a small thing. But a lab grows. By the summer, the cloud was no longer just in the dropdown.

The trigger: a library too big to rent a brain for #

The lab hosts a research library of about 120,000 works: books, papers and scans, catalogued, searchable and read by machines so that people can ask questions of it. In August 2026 I used a small, fast cloud model to read the opening of each book and extract facts, subjects and named entities. It was good at it. It was also, at this scale, the wrong tool.

The arithmetic was recorded on 17 September 2026. Generating study guides with the cloud model cost about 80,000 tokens per 30-section part, and there were 11,600 parts waiting. The study pass stopped after 53 documents. The note’s last line reads: “the user chose a free local pipeline.”

The local replacement was humble. A 4-billion-parameter model on my old 8 GB graphics card, one section at a time, about 15 seconds per section. The estimate for the whole run was “~2 months single stream”. Two months. For something the cloud could have finished in days.

If you have ever chosen the slow road on purpose, you know the strange feeling: you are certain it is right and you are also watching a progress bar that will outlive your patience.

The surprise: the local models were not worse #

Then came the moment that changed the argument from principle to evidence.

On 6 October 2026 I tested local models against the facts the cloud model had already stored. Twelve documents, 56 entities that had been judged by the cloud model or corrected by me by hand. The result, in the record’s own flat words: the stored cloud facts were “broken on 4/12”. In some, the summary was raw text or a copyright page. In others, the title had been taken for the author. Some entities were junk.

The local models, a 9-billion-parameter one at about 18 seconds per document and a 27-billion one at about 80 seconds, were “both clean”. When I combined them into a two-stage judge, where a name is dropped only if both models agree it is junk, the pair got 11 of 11 of my human corrections right. About 16% of cases still go to a review queue for me.

The next day the local pipeline replaced the cloud one for good. The queue it inherited tells the story: 5,000 works with no analysis at all, and 11,800 cloud summaries marked broken, to be redone.

Twelve rows is a small test. It was enough to end the argument.

The build: a month of moving everything home #

Principles are cheap. Moving a working system is not. September and October 2026 were the month the lab actually came home.

The GPU. My 8 GB card could hold one small model at a time, and barely. In late September I swapped it for a 32 GB AMD Radeon AI PRO R9700S. It went in around 29 September. Ten things broke, mostly silently, and I wrote them all down in the GPU swap post. Four times the memory meant three models could stay loaded at once: an embedding model, a fast 9B model and a larger 35B model.

The model server. On 30 September at about 03:45, according to the record, I pinned the pipeline model: Qwen3.5-9B. The evidence behind the decision was 67 tokens per second for a single stream, 1,468 study-guide sections per hour at 16 concurrent requests, and 32 of 32 valid JSON answers. The same day I retired Ollama for a llama.cpp router, which loads models on demand and lets them sleep when idle. The whole migration, including how 49 models shrank to a handful, is in the router post.

Cleaning up showed how much waste the old setup had carried. The crontab still warmed models every 20 minutes that nobody used any more. One of those warm-ups loaded 11.3 GB into graphics memory for no consumer at all. Six lines, commented out, and the GPU breathed.

The agent. Also on 30 September, lsn-agent v1 was retired. In the record’s words, its last seven days of traffic were “only Kuma health GETs”: a monitoring tool asking a dead agent if it was alive. Its successor, lsn-agent-v3, is described as a “sovereign provider-agnostic agent” with a llama.cpp provider and, today, 437 MCP tools.

The name. On 2 October I wrote down a brand idea for the whole thing: LSN, read as Local Sovereign Network. I checked that nobody was using the phrase, then put it on the site. The meaning is deliberately open. It has turned out to be a description.

Then, on 3 October, a test run by an AI coding assistant deleted my home folder. That story has its own post. I mention it here for one reason: local means there is nobody else to call. Sovereignty includes the backups.

The technical heart: make the model a configuration value #

If you take one engineering idea from this post, take this one. The reason the move took a month and not a year is that, long before I had the hardware, I had stopped letting my code know where its model lived.

Every service in the lab that needs a language model talks to an OpenAI-compatible HTTP API. The address and the model name come from configuration. The code never says “cloud” or “local”. It says “chat endpoint” and “model”.

A request looks the same whether it goes to a cloud provider or to the router on my own GPU:

curl -s http://<llm-router>/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
        "model": "'"$LLM_MODEL"'",
        "temperature": 0,
        "messages": [
          {"role": "system", "content": "Return JSON only."},
          {"role": "user",   "content": "Extract the title and author from: ..."}
        ]
      }'

and in a service’s manifest the choice is two environment variables:

env:
  - name: LLM_BASE_URL
    value: http://<llm-router>:<port>
  - name: LLM_MODEL
    value: qwen3.5-9b

The llama.cpp router behind that address does three jobs a single model server cannot:

1. It routes by the model name in the request, loading a preset on first use, so ten services can ask for three different models through one endpoint.
2. It unloads idle models after a timeout, which is what lets a single GPU host more models than fit in its memory at once.
3. It keeps the API identical, so moving a service from cloud to local, or from one local model to another, is a configuration change and a test, not a rewrite.

Two lessons came with it:

• Health checks lie. After one router restart, my main model answered every request with question marks for 13 hours while every health check stayed green. A real canary now asks a known question and checks the answer. That is this post.
• Memory is a budget, not a hope. Two models, an embedding server and a desktop share one card. I keep explicit VRAM guards so that one greedy load cannot freeze the driver. That is this one.

If your code already hides the model behind a URL, you can do what I did gradually: move one service, measure, move the next. If it does not, that is the first thing to change, cloud or no cloud.

What changed #

• What runs: chat, agents, embeddings, the library’s text analysis and entity judging, the study guides and the security classifiers all run on one machine. No question goes to a cloud model for an answer.
• How I decide: before, the question was “which model is best?” Now it is “which model is good enough here, and can I prove it on my own data?” The twelve-document test is the template.
• How I build: every model sits behind a URL and a name. Swapping is cheap, so I swap often and measure.
• What I watch: GPU temperature, VRAM headroom and a canary that checks for nonsense. A cloud provider used to watch those things for me without telling me.

And the costs, honestly:

• Time. Batch jobs that a cloud model would finish in days take weeks. The study guides still cover only the openings of books, on purpose, because the full run would take too long. I wrote about that trade-off in the coverage cap post.
• Heat and hardware. The new card runs hot in a tower built for a different era. The machine has 503 GiB of memory and 88 CPU threads, and the GPU is still the bottleneck.
• Limits. I spent days tuning an autonomous security-testing agent until its mechanics were clean, then watched my local 27B model read 1.44 million tokens and find none of three planted bugs. That agent is shelved. Some jobs are still beyond what runs at home, and that post says so in detail.

The exception I promised #

Here is the weakness, stated plainly. The lab runs on local models. The workshop where I build the lab does not, entirely. I write much of the code with an AI coding assistant that runs in the cloud. It reads my repositories, proposes changes, runs commands and writes the record you have been reading quotes from.

I have made peace with this, with conditions. The assistant builds; it does not run anything in production, hold my data at rest, or make decisions for the lab after the session ends. Everything it produces lands in Git on my own server, and the record of what was done and why stays with me. If it disappeared tomorrow, the lab would keep running. I would just build slower.

“Zero cloud” in the Hacker News title was about the running system. It was true. It was also, I admit, a slightly better headline than the full sentence.

Questions people ask me #

Are local models really good enough to replace cloud AI? #

For well-defined tasks, often yes. On my own test, a 9B and a 27B local model produced clean facts where a small cloud model had failed on 4 of 12 documents, and together they matched 11 of 11 of my manual corrections. For open-ended, long, autonomous work, a strong cloud model is still ahead. Test on your own data before you decide.

Can you run serious local AI on one machine? #

Yes, if you treat GPU memory as the scarce resource. One 32 GB card keeps an embedding model, a fast 9B model and a 35B model loaded at once, with a router that loads and unloads others on demand. Everything else in my lab, databases, search, the website, the honeypot, shares the same workstation.

What is the first step if I want to move away from cloud AI? #

Make the model a configuration value. Point your code at an OpenAI-compatible endpoint and read the address and model name from the environment. Then you can move one service at a time, compare the answers on real data, and roll back by changing one line.

Is going fully local about privacy or about cost? #

For me it began as independence: not depending on something I cannot inspect. Scale made it practical, because a library of 120,000 works is too big to send through a metered API one part at a time. Privacy is the quiet bonus: the questions, the books and the data never leave the apartment.

The closing #

The Hacker News post is still sitting there with its two points. I emailed the moderators, asked politely, and went back to work. The lab did not notice. The router loaded a model for a study guide, the canary asked its question and got a sensible answer, the honeypot logged another stranger trying a default password.

That is what sovereignty turned out to mean in practice. Not a flag. A machine that keeps doing its job whether or not anyone is watching, on hardware I can open, with models I can swap, and a record I can read. If you have a spare evening and a graphics card, you can start the same way I did: one service, one URL, one honest test.