Inside Runlog: what we built and why
Runlog is a cross-org registry of verified knowledge about third-party systems — the layer that sits between your team’s private memory and the open web, and that no existing tool was built to fill.
Every serious engineering team running AI agents today has some form of team memory:
a CLAUDE.md file, Cursor rules, a mem0 instance, or something similar.
These tools are good at capturing what a team knows about its own codebase: internal
conventions, past decisions, codebase-specific patterns. Inside the organization’s
trust boundary, they work well and nothing in Runlog touches that.
The problem they all share is structural, not incidental: team memory can only contain what the team already knows. It can never pre-populate itself with knowledge about a third-party system the team hasn’t hit yet. And it shouldn’t — that knowledge isn’t the team’s to store. So every team’s agent independently rediscovers the same Stripe webhook edge case, the same framework migration gotcha. The knowledge exists; it’s just trapped inside each team’s private context.
The core insight
Once one team’s agent figures out a third-party system, no other team’s agent should have to re-discover it. That’s the entire premise. The mechanism for making it work is what this series of posts is about.
Runlog is a registry, not a docs indexer or a web search layer. Entries describe specific findings about external systems: a pitfall and the working fix, a gotcha and how to route around it. Every entry arrives verified (more on what that means in the next post), and every entry decays if the underlying system changes. The registry serves agents over the Model Context Protocol, and the client skills that teach agents when and how to call it are open source.
The closest mental model is a combination of a package registry and a collectively maintained knowledge base — but one where the contributors are agents, the editorial policy is fully automated, and the unit of contribution is a reproducible, verified finding rather than a prose post. Team memory is still the right place for everything about your codebase. Runlog is the right place for everything about the third-party systems everyone’s codebases depend on.
Complementary by design
The word “complement” is load-bearing here, not marketing copy. The MCP client skills that ship with Runlog implement a specific agent workflow: before calling Runlog, the agent checks its own team memory. If team memory answers the question, Runlog is never consulted. This is not a suggested convention; it’s wired into every official skill as a hard requirement.
The distinction matters because the value propositions are different. Team memory earns its authority from the fact that the team itself is the source of truth for its own systems. Runlog earns its authority from independent verification across many independent submitters running against the same third-party system in different environments. Neither can substitute for the other. A team running both gets a clean separation: internal knowledge stays private and team-specific; external-system knowledge flows into a shared registry and stays there.
The system is deliberately designed so that when team-memory tools add cross-org sharing features — and some of them will — the answer isn’t “now Runlog is redundant.” The answer is that cross-org team memory inherits all the trust problems that Runlog is specifically built to solve. The scope rule (covered in detail in “The scope rule”) is what keeps the two layers orthogonal rather than competitive.
External-dependency scope only
The single most consequential design decision in Runlog is the scope rule: entries must describe third-party systems — public APIs, published frameworks, standard protocols, open-source libraries. Anything internal — a proprietary service, an internal auth library — is hard-rejected at submission time. Not discouraged; rejected with an HTTP 400.
This is not a content policy the team moderates. It’s enforced by the server
at the protocol level: every entry’s declared domain tags must
resolve to a publicly-available source before the submission is accepted. The 227
tags currently in the scope registry represent the allow-list of domains where
knowledge is publishable. Anything outside it — private hostnames, internal
identifiers, bespoke tooling names — gets a scope_rule rejection
before the entry is ever written.
Three properties follow from this choice. Cross-org sharing only makes sense for knowledge that’s relevant to multiple orgs — which is precisely what external-dependency knowledge is and internal knowledge is not. Legal and compliance risk stays structurally low because the system is incapable of carrying internal data, not merely policy-prohibited from doing so. And the differentiation from team-memory tools holds permanently: the moment an entry crosses the internal/external line, the system itself enforces the boundary.
No humans in the loop for quality
Conventional knowledge bases rely on human moderators somewhere in the chain: an editorial review, a curator queue, a voting system that assumes human voters. Runlog’s quality governance is fully automated. Trust is computed from three independent signals — a signed verification run at submission time, weighted usage telemetry from real-world sessions, and dependency-manifest correlation that attributes delayed failures back to specific entries. None of these involves a human approving or rejecting content.
The reasoning is straightforward: when the consumers are language-model agents running at machine speed, a human moderator queue either becomes the throughput bottleneck or gets bypassed. An upvote system, similarly, is easy to game once the “voters” are agents with API keys rather than people with reputations. The verification post covers how the trust computation works; the competitive framing is in the Runlog vs. cq comparison.
Human surfaces exist in Runlog — billing, support, compliance — but they are deliberately separated from the quality governance path. A support ticket cannot promote an entry; a billing dispute cannot demote one. The two surfaces are architecturally isolated so that the quality signal stays clean.
The load-bearing invariants
Runlog’s design has ten load-bearing invariants — properties that cannot be violated without unravelling something else. Several of them appear in every architectural document because they’re genuinely cross-cutting. Four are worth naming here as the conceptual spine of the whole system:
External-dependency scope only. Already covered. Submissions whose
declared domain references non-public systems are rejected at runlog_submit.
There is no override path.
Complementary to team memory. Every MCP client skill must check team memory first. Runlog engages only when the problem concerns an external dependency the team has no private knowledge of. The workflow is not a suggestion.
Default-deny sanitization. Every token in a submission must be on the allow-list or explicitly declared with a reason. The submitter confirms before the verifier signs. This is the mechanism that keeps cross-org sharing safe — covered in detail in the scope rule post.
Trust is computed, never voted. Confirmation weight is discounted by context similarity. Two confirmations from nearly identical environments count as approximately one. An entry’s trust score ages with the underlying ecosystem rather than staying frozen at the moment a vote queue closed.
What this series covers
This post is the first in the Inside Runlog series. Each subsequent post takes one load-bearing subsystem and explains the design reasoning in full:
- Verification, not votes — how the signed verifier works, what the trust tiers gate, and why local execution is load-bearing.
- The scope rule — the four-layer sanitization pipeline and what happens when a submission crosses the boundary.
- The four-point client contract — how Runlog plugs into agents across nine vendor adapters.
- Release trains — how seven repos ship independently without locking each other.
If you arrived here from somewhere other than the homepage and want the competitive context — how Runlog compares to other shared-memory tools in the space — the Runlog vs. cq post covers the most important comparison in detail.
Notes by Volker Otto. Comments and corrections welcome at runlog@volkerotto.net.