Writing everything down: one shared memory for me and my AI agents

The first line is timestamped 18:48:49 on 2 April 2026. It is one sentence long, and the last clause is the one that matters: “Created lsn-roadmap knowledge database.”

Eighteen minutes later there is a second entry. An audit of everything I ran: 161 web routes, 119 containers, 45 namespaces, 192 AI tools across six servers, 28 places where a credential lived. “Updated roadmap DB with 100+ new entries across all tables.”

Today that record holds 5,771 entries: 2,721 tasks, 2,701 observations, 253 mistakes, 61 runbooks, and 1,350 session summaries. It is read and written by me, by AI coding assistants, and by my own local AI agent. On 18 October 2026 I wrote one sentence about it that I still like: “new era begins”.

This is the story of how a habit of writing things down became the backbone of how I work with machines, and what it means to do that work in the open.

Contents · 10 sections

In one minute #

• My lab is one person, one machine, and a lot of AI help. Every AI session used to start from zero, and so, often, did I.
• In April 2026 I started a simple database of tasks, mistakes, runbooks and gotchas, and made a rule: read it before working, write to it after.
• By October it had about 5,500 entries, keyword search was failing, and the record had become the most valuable thing in the lab.
• On 18 October I moved it into a shared context database that humans and agents can search by meaning, retired the old database, and put even my own writing, the analyses of my 195 blog posts, inside it.
• What changed: AI stopped being a stranger at every session. It became a colleague with a memory, and the memory is open for inspection.

The before: every morning, a stranger #

If you have worked with an AI assistant on anything bigger than a single file, you know this moment. You open a new session. It is brilliant, fast, polite, and it knows nothing about you. Not which database holds the cluster state, not the mistake you made last Tuesday, not that the “temporary” folder is load-bearing. You explain it all again. Or you forget to, and it finds out the hard way.

My lab made this worse, because it is big for one person. Kubernetes, local models, a research library, honeypots, mail, single sign-on. The knowledge about it lived in my head, in scattered notes, in shell history, and in the transcripts of AI sessions that ended and took what they had learned with them.

So on 2 April I did the plainest thing possible. A SQLite file. Tables for tasks, observations, mistakes, runbooks and sessions. A small command-line tool to add and find entries. And a rule written into the instructions every assistant reads at start: read the roadmap before you work, write to it after.

The name “roadmap” was a bit aspirational. It was never a plan of the future. It was a record of the past, precise enough to plan from.

The growth: a habit becomes a dependency #

The entries were small and stubborn. “What happened. Root cause. Lesson. Prevention.” A mistake written as a lesson is a mistake you only pay for once.

In July I noticed that the record had no backup and no history. The thing every agent depended on was one file on one disk. On 7 July it went into a private Git repository, with a text dump next to the binary so changes could be read as diffs, and a job that committed and pushed every 30 minutes whenever the content had changed.

Then came 3 October, the day a test deleted my home folder, which I wrote about in the full post-mortem. The lesson I put at the end of that post was “commit or lose it: anything that matters is in Git, pushed to a server that is not my home folder, even the record of what was changed and why.” The roadmap is the last item in that sentence for a reason: it is the record of everything else.

The day after the wipe, 4 October, there is an entry in my own voice, recorded as an owner decision: the roadmap is mandatory for every AI. My local agent, which until then could only read it, got write access to the database folder, and nothing else around it.

By mid-October the record was around 5,500 entries, and its weakness had become obvious. Keyword search. My agents asked questions in plain words (“which log viewer do I use?”) and the entry said “Dozzle”. The answer was in the record. The record could not hand it over.

The trigger: one evening, two sessions, and a deadline I did not set #

On 17 October I chose an open-source context database, OpenViking, to sit on top of the record: a file tree for agents, short summaries of folders, search by meaning, and an MCP interface so agents use it as tools. Two AI sessions worked on it at the same time, one deploying and one wiring my local agent and testing, and every change needed a review from the other before it counted as done. Two machines checking each other’s homework, with me deciding.

That same day I also said something that is in the record word for word, after an assistant finished a feature and asked whether it should commit: “commit, push, update roadmap (every single time!)”. The next day I added, in effect: even with a capable AI assistant, I have to remind it that we have a roadmap. So the reminder became code. A hook now stops an assistant from ending its turn once if it has changed a repository without committing, or changed things without writing to the roadmap. Humans need habits. Machines need hooks.

The full import started at 19:10 UTC. My one GPU wrote summaries for about 1,300 nodes at roughly seven per minute. The record notes VRAM temperatures between 63 and 82 °C and the plan for the night: pause the OCR, let the study-guide workers keep running. At 03:08 on 18 October it finished: 610 of 610 files, zero errors, about five hours.

Then the test. One session held fifteen questions it had written in advance and kept sealed from the other, so that nobody could tune the data to the exam. The result: 9 out of 15 correct. The old keyword search: 8 out of 15. Barely better, after all that.

The misses were not missing facts. Every answer was stored. The import had packed several entries into each searchable chunk, so a single one-line fact was diluted by its neighbours, and plain-language questions did not share words with technical answers. The fix, by mid-morning: one fact per file, each with a short plain-language header naming its project, and a reranker that reads the top twenty candidates before choosing four. The score went to 12 out of 15, nine of them at rank one. I wrote about the mechanics in the shared memory post, including two secrets that slipped past my redaction on the first day and how they were caught.

The decision: retire the old one #

A trial is easy to keep forever. Two systems, both “kind of” the truth, and before long nobody knows which one to believe.

So on 18 October I made the call, and the record keeps my words: “the complete switch over means the retirement of the old sqlite roadmap also … new era begins.” The old database was frozen read-only as a historical archive. Every new entry since then goes straight into the new store, through a write path that refuses any text containing a secret.

And then I asked for something more, in capitals: “I WANT EVERY THING FROM ROADMAP TO VIKING, ESPECIALLY THOSE BLOG POSTS AND PERSONAL INFO”. The final export included an identity folder: analyses of 195 posts from my personal blog, 19 aspects of a person profile, 22 worldview topics. Each file starts with the same line: the owner’s identity, their own words, feelings and evaluations, kept as they asked.

I also raised the trust of my local agent that day. It may read, write and use the roadmap, and add skills. Destructive actions are allowed only if they pass a second factor on my phone.

The technical heart: what a shared memory needs #

If you want to build something like this, for a team of humans, a team of agents, or both, here is what the record taught me, in order of importance.

1. One fact per entry, with its context attached #

Every file in my record looks roughly like this:

> Mistake · Project: <project> -- <one-line description of the project>

# Mistake #233 (2026-10-17)
**What happened:** ...
**Root cause:** ...
**Lesson:** ...
**Prevention:** ...

The header line is not decoration. It is what lets a vector search connect “my log viewer” to an entry that only says “Dozzle”, because the project description is embedded together with the fact.

2. Permissions per reader, not per system #

My assistant sessions can manage the roadmap. My local agent can write to it. Content the agent writes is treated as untrusted data by the other sessions: read as information, never followed as instructions. A shared memory is also a shared attack surface.

3. A leak gate on every write #

Every entry is checked against the live values of every secret in the cluster before it is stored. Pattern matching alone is not enough; a bare token with no label in front of it matches no pattern.

4. History in Git, automatically #

A snapshot of the whole tree is mirrored to Git every 30 minutes. If an agent writes something wrong, the diff shows who and when.

5. Memory is history, not status #

“The service is healthy” was true when it was written. Questions about the present need a live check. My agents are instructed to treat the roadmap as the past and verify the present.

The after: working in the open record #

Here is the part that is less about software.

Everything I do with AI is now on a record I can open: what was asked, what was done, what broke, what the machine got wrong and what I got wrong. My stance, written down in October, is that AI “can and should be used to ease people’s life, to bring clarity in the mess”, and that I am happy to live in these days. The record is how I hold myself to that. Clarity is not a feeling; it is a file you can read.

The record is not flattering, and that is the point. On 24 October an assistant rebuilt a book of mine from a website copy instead of from its real source, and the mistake entry quotes me directly: “REDO IT IN PLAIN LATEX”, “I HAVE ALREADY THE ORIGINAL TEX FILE”. The lesson underneath it is now read by every session before it builds any file: search for the original source before you reconstruct anything from a copy. My frustration on that Friday became a rule that protects every build after it.

It also changed the work itself, in ways the record shows. Sessions now open with context instead of questions. My local agent searches the record with the same reranked tool the assistant sessions use. A lesson written down in summer is in front of every session that touches the same system in autumn. I am still the one who brings the questions and the judgement. But I am no longer the only memory in the room.

It cost things too. Hours of GPU time on summaries. A night of import. Real work on leak detection, because a memory that holds everything also holds things that must never leave. And a constant, small discipline: write it down, every single time, even when the fix took two minutes and the entry takes three.

There is a question this does not settle. A record this complete is also a portrait of me: how I work, what I check twice, where I was careless. I chose to keep it, including the personal writing. The argument for it is simple: a partner who only knows the tidy version of you is not much of a partner. The argument against it is just as simple: everything written down can be read by someone you did not plan for. That is why the leak gate and the permissions exist, and why I keep the trade-off in view instead of pretending it is gone.

What changed #

• Every AI session starts from the record, not from zero.
• Mistakes are paid for once: each one becomes a lesson other sessions read before they repeat it.
• Search by meaning, one fact per file, with a measured score instead of a feeling.
• One source of truth. The old database is an archive, not a second opinion.
• My own words live in the same place as my systems, by my choice.

Questions people ask me #

Why not just use the memory feature of an AI product? #

Because that memory belongs to the product, lives on someone else’s servers, and is shared with no other tool. My record is read by assistant sessions, by my local agent and by me, it is in Git, and it is searchable by anything I connect to it. Memory that one vendor controls is a convenience, not a foundation.

Is it safe to let AI agents write to a shared memory? #

Only with limits. Mine can write entries, but every write passes a secret scanner, content written by the agent is treated as untrusted data by other sessions, history is snapshotted to Git every 30 minutes, and destructive actions need a second factor from a human. Trust is raised step by step, on the record.

What should a small team start with? #

A plain database or even a folder of Markdown files, with four kinds of entry: tasks, observations, mistakes with lessons, and runbooks. One fact per entry. A rule that you read it before work and write to it after. Search by meaning can come later; the habit cannot.

Does writing everything down slow you down? #

Each entry costs a minute or three. A forgotten lesson costs hours, and a forgotten warning can cost a weekend. My record holds 253 mistakes, each written down with its lesson. Each one is something I, or a machine, did not have to learn twice.

Closing #

On 2 April the roadmap was one sentence. Today it is the closest thing my lab has to a conscience: it remembers what we promised to check, what we got wrong, and why we decided what we decided.

I used to think the hardest part of working with AI would be getting it to be smart. It turned out the hardest part was getting both of us to remember. Write it down. Then let the machine read it, and let anyone who wants to, see what you wrote.