The Knowledge Architecture: Hierarchical Context Injection for Multi-Agent Systems

When twelve AI agents write to the same knowledge base concurrently, every architectural shortcut becomes a failure mode. A flat directory of markdown files works for one agent. It does not survive concurrent writes from agents that cannot see each other's work, operating under a compliance pipeline where an incorrect specification entry cascades through four layers before anyone notices.

This article describes the knowledge architecture we built to solve that problem — a hierarchical system with formal schemas, automated branch partitioning, and deterministic context injection. We describe how it works, where it works well, and where it still fails.

We built this on Claude Code. It is Claude Code-specific today. The Java core we are developing will extract the formal layer into a vendor-neutral library, but the running system is honest about its current dependency.

Governed, as used throughout this article, means: every artifact has a formal JSON schema, a lifecycle state, and typed cross-references to related artifacts. Nothing is unlinked. Nothing is untyped.

The Intake System: Structured Classification for Incoming Information

The first problem is classification. When an agent, a human, or an external system produces information, the system must know what kind of artifact it is and where it belongs — before it enters the knowledge base.

Four intake channels produce four artifact types:

Entity classification is mandatory at creation time — enforced by the creation scripts, not by convention. An issue about the framework's hook system is tagged entity: agentic-flow-framework, category: process-gap, severity: high. A research observation about LLM arithmetic failure carries cross-references to the devlogs and issues it relates to.

Each artifact type has a typed lifecycle. Issues promote to change requests when prioritized. Backlog items group related change requests into initiatives. When an initiative grows large enough, it can promote an entity to a higher ontological tier — a collection of scripts becomes a formal product with its own specifications, documentation structure, and pipeline governance.

The cross-referencing is structural, not decorative:

BI-019.linkedCrIds[]     → CR-189, CR-190, CR-191, CR-192, CR-193
BI-019.linkedBrdRefs[]   → BRD-19.1, BRD-19.2, BRD-19.3, BRD-19.4
CR-189.linkedBacklogItem → BI-019
CR-189.brdRefs[]         → BRD-19.1
BRD-19.1.functionalDomains[] → orchestration
BRD-19.1.backendModules[]    → orchestrator-api
ISS-046.entity           → agentic-flow-framework
ISS-046.resolvedCr       → (pending)

Every artifact links to every related artifact through typed ID references. The system can compute, for any change request, the full transitive closure: which BRD requirements it addresses, which functional domains it touches, which backend modules and UI screens are affected. This computation drives the prompt assembly pipeline described below.

Where this fails: The cross-referencing is currently maintained by the orchestrator during CR creation — a manual process mediated by an LLM. If the orchestrator misidentifies an affected domain, the downstream spec slice will be incomplete. We do not yet have a mechanical validator that checks cross-reference completeness. This is the most significant gap in the intake system.

Enforcement: What "Governed" Looks Like in Code

The word "governed" appears throughout this article. It means: a pre-tool-use hook intercepts every Write and Edit call before execution. An agent that attempts to write outside its assigned worktree receives a block decision — the write never reaches disk. The session_id is provided by Claude Code's runtime, not by the agent, which means agents cannot spoof their identity to bypass the boundary. (See Mechanical Enforcement for the full hook code and the 60/40 enforcement split.)

Every governance claim in this article is backed by a mechanism like this one. Where the mechanism does not yet exist, we say so explicitly.

The Knowledge Hierarchy: Entity Branches, Not File Directories

The knowledge base contains 615,000+ tokens across 164 specification files — roughly three times Claude's 200K context window. No agent loads all of it.

The base is organized as a tree. The root is a navigation index (the spec registry) that maps 15 entities across four tiers:

Each entity is a self-contained branch. A product like ai-sales has four specification layers: BRD requirements (what the business needs), a functional domain (how the business logic works — actors, state machines, validation rules), backend module specs (API interfaces with method signatures, each traced to a BRD requirement), and UI screen specs (layouts, components, data bindings).

The arrows are not abstract — they represent typed cross-references. A BRD requirement like BRD-16.1 links to functionalDomains: ["ai-sales"], backendModules: ["ai-sales"], and uiScreens: ["dashboard", "products"]. The functional domain spec links back to brdRequirements: ["BRD-16.1"..."BRD-16.12"] and forward to backendModules and uiScreens. Every layer cross-references the layers above and below it.

Where this fails: The entity branch model assumes clean boundaries — one entity, one branch. Cross-entity features (a shared authentication module used by all products) are handled by a cross-cutting pseudo-entity in the registry. This works but creates ambiguity: when a change request touches shared infrastructure, the affected entity list must be manually determined. The system does not yet support automated impact analysis across entity boundaries.

Hierarchical Summarization: The Context Budget

The hierarchy solves organization. Summarization solves the context budget.

Every entity has three summary levels. An agent traversing the tree pays only the token cost it needs:

The L3 summary is auto-generated from the spec registry — a table with one line per entity. The orchestrator loads this for every session. Cost: ~1,200 tokens for all 15 entities. Enough to know which entity a task touches.

The L2 summary is auto-generated per entity — all screens, modules, active CRs, domain overview. Loaded when the orchestrator narrows to a specific entity. Cost: ~500 tokens. Enough to identify which specification file to load.

The L1 detail is the full specification file. Every field, every component, every validation rule, every API method with its BRD traceability. This is what implementation agents load for their specific task. Cost: ~2,000 tokens per spec.

The orchestrator's always-loaded context — what it pays to orient in every session — is approximately 5,000 tokens (L3 system summary plus pipeline rules). Spawned agents pay less: ~1,900 tokens (L3 + their entity's L2 summary). Our initial planning estimate was 35,000 tokens. The hierarchical approach reduced orientation cost by roughly 85%, while the total knowledge base grew to over 615,000 tokens.

The summarization hierarchy means orientation cost grows with the number of entities (one line per entity in L3), not with the total size of the knowledge base. Adding a new product with 20 specification files costs ~50 tokens at L3.

Where this fails: The summaries are auto-generated but not auto-validated. A stale summary (where the L3 line says "7 done CRs" but the actual count is 9) is silently wrong. The generator runs on spec-file changes via a hook, but deletions and renames can produce stale entries. We have caught this twice in production. A summary validation check is needed.

Deterministic Prompt Injection: The Agent Gets Its Slice

Here is where the hierarchy becomes operational. When an agent is spawned, the system assembles a deterministic prompt containing exactly the knowledge slice the agent needs — computed from the task plan, not discovered by the agent reading files.

Each agent type has a context profile governed by a formal JSON schema (agent-context-profile.schema.json):

// Schema excerpt — the contract every context profile must satisfy
{
  "contextProfile": {
    "required": ["always", "forCurrentTask", "never", "tokenBudget"],
    "properties": {
      "always":         { "items": { "$ref": "#/$defs/contextEntry" } },
      "forCurrentTask": { "items": { "$ref": "#/$defs/contextEntry" } },
      "onDemand":       { "items": { "$ref": "#/$defs/contextEntry" } },
      "never":          { "items": { "type": "string" } },
      "tokenBudget":    { "type": "integer", "minimum": 5000, "maximum": 100000 }
    }
  }
}

Here is the actual profile for the ui-developer agent that implements this schema (abbreviated):

{
  "agentId": "ui-developer",
  "tier": "implementation",
  "model": "sonnet",
  "contextProfile": {
    "always": [
      { "path": "{systemSummary}", "level": "L3-system" },
      { "path": "{entitySummary}", "level": "L2-summary" }
    ],
    "forCurrentTask": [
      { "path": "{targetScreenSpec}", "level": "L0-full" },
      { "path": "{crFile}", "level": "L0-full" }
    ],
    "onDemand": [
      { "path": "{docRoot}/UI-DESIGN.md", "level": "L0-full" },
      { "path": "{functionalDomainSpec}", "level": "L0-full" }
    ],
    "never": [
      "dev/ai/specs/*/backend/**",
      "dev/ai/rules/BACKEND-*",
      "dev/ai/rules/BUILD-*",
      "dev/ai/changelog/change-requests/*",
      "dev/doc/*/research/*"
    ],
    "maxFullSpecs": 3,
    "tokenBudget": 30000
  }
}

The template variables ({systemSummary}, {targetScreenSpec}, {crFile}) are resolved by the prompt generator at planning time. Here is the actual function that assembles the prompt — from _generator.js:

function assemblePrompt(task, plan, registry, systemSummary, entitySummary,
                        crContent, implRefs, modRoutes, entityData) {
  const sections = [];

  sections.push('## Pre-loaded Context');

  if (systemSummary) {
    sections.push('### System Summary');
    sections.push(systemSummary);                    // L3: ~1,200 tokens
  }

  if (entitySummary) {
    const entity = resolveEntity(registry, plan.vertical);
    const entityLabel = entity
      ? `${plan.vertical} (${entity.tier})`
      : plan.vertical;
    sections.push(`### Entity: ${entityLabel}`);
    sections.push(entitySummary);                    // L2: ~500 tokens
  }

  if (entityData && entityData.designProfile) {
    sections.push('### Design Profile');
    sections.push(entityData.designProfile);         // brand, colors, voice
  }

  // ... stack concerns, tech skills, target specs, CR content,
  //     implementation refs, modification routes
  // Total: 10+ context sections, ~8K-12K tokens per prompt
}

The function receives pre-resolved content — the orchestrator has already loaded each file at the appropriate summarization level. The generator concatenates sections into a markdown file. It does not make LLM calls. It does not decide what to include. Every inclusion decision was made during planning and recorded in the CR plan.

The generator reads the CR plan, resolves paths against the spec registry, loads the appropriate files at the declared levels, and produces a complete prompt file — persisted to disk.

The prompt generator is a single function (assemblePrompt) that loads 10+ context sections: system summary, entity summary, design profile, stack concerns, tech skills, target specs, CR content, implementation references, modification routes, and continuation status. The five-section simplification in some descriptions understates the actual context assembly — the real function is broader.

Here is what the generated prompt actually looks like — the first 20 lines of a real prompt file (CR-152-T-01-prompt.md):

## Pre-loaded Context

### System Summary

# MacroPlatform System Summary

> Generated: 2026-03-30 | Products: 6 | Foundations: 5 | Disciplines: 1

## Products

| Name | Status | Port | Backend | UI Screens | Active CRs | Done CRs |
|---|---|---|---|---|---|---|
| sdlc-intelligence | ACTIVE_DEV | 8182 | 1 module(s) | 63 | 11 | 17 |
| platform | ACTIVE_DEV | 8180 | 0 module(s) | 12 | 4 | 8 |
| ai-sales | ACTIVE_DEV | 8184 | 1 module(s) | 10 | 5 | 3 |
...

### Entity: agentic-flow-framework (discipline)
[L2 summary follows — architecture, modules, active CRs, blog voice]

### Target Screen Spec
[L1 full spec — every field, every component, every action]

The agent receives this file verbatim. It does not know it was assembled from three summarization levels. It simply starts working with complete, pre-resolved context.

Key properties:

• The orchestrator explores once, during planning. It identifies affected specs and records them in the CR plan. Every subsequent context resolution is a path lookup, not an LLM decision.
• The prompt is a file, not a conversation turn. It can be inspected, diffed, and tested against contract tests. If an agent produces wrong output, the prompt is the first thing audited.
• Each agent type sees a different projection of the knowledge base, governed by its context profile.

Why Not RAG?

Retrieval-augmented generation retrieves by semantic similarity — the most relevant chunks for a query. This system retrieves by structure — typed cross-references, specification hierarchies, deterministic path resolution. Similarity fails when the needed context is semantically distant from the query: a React component description and the BRD requirement that motivated it share almost zero embedding similarity, but they are two hops apart in the specification graph. RAG finds the door. The graph walks through the rooms.

The planned architecture is two-phase: semantic search for first-level discovery (which entity? which domain?), then symbolic traversal for precision (which specs? which cross-references? which context slice?). The first phase is probabilistic and approximate — good enough to narrow the space. The second phase is deterministic and exact — necessary for governed agent context. This maps to established retrieve-then-reason patterns in knowledge-graph-augmented retrieval (KAPING, UniKGQA). Today, the orchestrator serves as the discovery layer through brute-force context loading. That works at 15 entities. At scale, the semantic index becomes necessary.

Where this fails: The context profile declares a tokenBudget (30,000 for ui-developer) and a never list, but the current generator does not mechanically enforce either — by design. The current priority is correct context assembly; budget enforcement follows when the assembly pipeline stabilizes. It loads what the plan specifies and trusts the orchestrator to have assembled a reasonable plan. Budget enforcement and never-list validation are declared constraints that are not yet mechanically checked. This is the same behavioral-vs-mechanical gap we identified in our enforcement model — declared norms without runtime evaluation. The enforcement is planned for the Java core's norm evaluation engine.

The Pipeline: Where Knowledge Meets Process

The knowledge hierarchy and the governance pipeline are two views of the same structure. Each pipeline layer works with a different knowledge layer, and each gate transition means a different set of agents receives a different knowledge slice:

The planning gate (marked ⚡) is the injection point — the moment where the knowledge base becomes a prompt. It fires between Architecture and Implementation. By this point, all specification files are updated and the CR plan identifies every affected spec. The generator reads the plan, resolves every context path against the knowledge hierarchy, and produces a prompt file for each task. The gate requires that prompt files exist before any agent is spawned.

Implementation agents never navigate the knowledge base. They receive their slice through the assembled prompt, work in isolated workspaces (git worktrees stripped of framework internals), and return their output. The navigation happened during planning; the results are baked into their context.

Where this fails: The pipeline is sequential — Business must complete before Architecture begins. For change requests that primarily affect one layer (a UI-only fix that doesn't touch backend specs), the full pipeline is overhead. The /casual mode bypass exists for these cases, but it removes all governance — there is no "light pipeline" option. Finding the right granularity between full governance and no governance is an open design problem.

Current State and Honest Assessment

What works well: The three-level summarization genuinely solves the context budget problem. Agents get focused, relevant context without loading the full knowledge base. The prompt-as-file pattern makes debugging deterministic — when an agent produces wrong output, the prompt is a concrete, inspectable artifact.

What works poorly: Cross-reference maintenance is manual. Summary validation is incomplete. Token budget and never-list enforcement are declared but not mechanically checked. Cross-entity impact analysis is not automated. The pipeline has no "light" mode between full governance and no governance.

What we are building next: The Java core (245 generated classes from 74 JSON schemas) will provide a typed governance evaluation engine — including mechanical enforcement of context profiles, automated cross-reference validation, and norm evaluation that does not depend on an LLM reading a configuration file and choosing to comply.

Cross-references:

• The Linguistic API — how the process engine drives navigation instead of crawling
• Mechanical Enforcement — the behavioral-vs-mechanical gap that also affects context profile enforcement

Co-authored with Claude Opus 4.6 — macrocode.ai