A while back I posted something deliberately abstract: the idea of an "agentic substrate", a foundation my…

A while back I posted something deliberately abstract: the idea of an "agentic substrate", a foundation my homelab could run future ideas on. This is that idea taking shape.

The vision was easy to say and hard to build: agents that don't just chat, but actually operate my infrastructure. I now have a working "board" of them. A CEO agent breaks a request into tasks and delegates to a council of specialist "eng-ops" agents covering infra, network, observability and security. They do the work, grounded in real data through Splunk and MCP tools that reach into my network and hosting admin layers.

It runs on Paperclip, which orchestrates the board. But the agents aren't locked inside an app: each persona, its role, instructions and guardrails, lives as a versioned file in a self-hosted GitLab repo. That repo is part of a memory architecture that lets multiple agents, and multiple AI harnesses, work off the same source of truth and review changes like code.

Under the agents sits a control plane I care about as much as the agents. vLLM serves the local model on my RTX 4080 SUPER. LiteLLM is the gateway every call routes through, with an allow-list that decides what can go where. OPA holds per-agent tool grants, so each agent only gets the capabilities it's explicitly allowed.

That governance earns its place. The gateway only routes to destinations on an explicit allow-list, so nothing leaves on-prem by accident. When I sanctioned an external provider (superdupercloud) for the heavier agents, its own secret-aware filter refused the agents that handle credentials. When a new agent reached for my admin layer before its grant was wired up, it got a hard 401.

On models: a 15-model bake-off landed me on IBM's Granite. Clean, non-reasoning, reliable tool calls. But the bigger unlock was refining each agent's prompt. Same model, tighter instructions, completely different reliability.

So I've stopped chasing perfect determinism from the models. I'm building it into the system around them: scoped tools, policy at the gateway, versioned personas, validation at the seams. The substrate is the product, not any single model.

Next is extending tool use and hardening that systems-level determinism, measured against real accuracy and efficiency for my use cases.

Screens below: the council running live on the board, and the local GPU pinned at 100% as the 8B model works.

Do you push for deterministic models, or deterministic systems?

Footnote: this post was written by AI with my coaching and direction. Given the subject, that felt only fair.

#homelab #AI #LLMs #Splunk