> ## Documentation Index
> Fetch the complete documentation index at: https://docs.augustin.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Self capable arc design

# The Self-Capable Arc — a Broad Code-Eval Capability + a Safe Write-Wall Takedown

**Status:** PLAN / design doc for review. No code changed, nothing deployed.
**Author:** architecture pass (Opus 4.8), 2026-07-06.
**Scope:** design a broad, general-purpose "execute + evaluate arbitrary code" capability for the homelab Mastra agent, and the safe removal of the write-wall that currently keeps the agent out of its own source (`images/`). The last step of the arc is letting the agent commit code changes — but only behind a real, tested verification loop.

> **Extension — concurrent workstreams + conflict resolution:** once the agent self-edits AND self-deploys (all 9 images incl. its own brain, turn 109) with MULTIPLE workstreams running at once, their commits collide. The **branch-per-workstream isolation + git-does-the-mechanical-merge + LLM-resolver-for-intent** design lives in [`concurrent-workstreams-design.md`](./concurrent-workstreams-design.md). It builds directly on this doc's `verify()` Layer-1 primitive (re-verifying every merge before deploy) and slots into Phases 4–5 below as a new **Phase 6**.

***

## 1. Goal + principles

**Goal.** Give the main agent a *broad* capability to safely modify and verify code — its own AND any other code — and, behind that verification, let it commit code changes to the repo. Two distinct powers fall out of this:

1. **Evaluate arbitrary code** (broad; no repo write needed): run + test any code in an isolated sandbox and get a real pass/fail. This works for a fresh Rust CLI in a scratch dir, a new service, someone else's repo — anything, not just the agent's own image repo.
2. **Commit a verified change to a repo path** (needs a widened `WRITE_ROOTS`): once a change has passed the verification loop, land it in `images/**` so CI rebuilds and Flux reconciles.

**Augustin's design principles** (hard requirements; tensions called out in §6):

* **P1 — Broad modules, not narrow ones.** "A few broad tools rather than lots of general-specific ones." Large, general-purpose capabilities that can be used for anything.
* **P2 — Code-eval must be general.** "If we're giving it the power to evaluate code, it should do that for ANY code, not just its own image repo." The capability is "run + evaluate arbitrary code in an isolated sandbox," full stop.
* **P3 — The main agent stays shell-free.** The main agent must never wield a host `execute_command` / command-interpreter tool. The established reconciliation (already in the codebase, REWRITE-NOTES turn 105): running arbitrary code inside an *isolated* throwaway pod, driven from *within* a workflow/tool and never attached to an Agent, does NOT violate shell-free — because attaching a sandbox to an Agent is precisely what auto-injects `execute_command`. This plan preserves that: broad execution power, isolated execution, main-agent tool surface never gains a raw host shell.
* **P4 — Do more with less code.** Capability-per-line is the north star. Don't build a grand framework.

***

## 2. Investigation findings (ground truth)

### 2.1 Mastra's current relevant features

Pinned version, from `images/mastra/package.json`: **`@mastra/core: ^1.47.0`** (caret; any `1.x ≥ 1.47.0` resolves). REWRITE-NOTES and code comments say `1.49.0` — that's the resolved lockfile version, consistent with the `^1.47.0` range. `@mastra/docker` and `@mastra/agentcore` are **not** dependencies.

Sources: `https://mastra.ai/llms.txt` and the doc pages it indexes. All version markers below are the docs' own "Added in `@mastra/core@x.y.z`" lines.

| Feature                                                                                       | What it actually does                                                                                                                                                                                                                                                                                                                                                                                                                             | Maturity                                                                                                     | Fit                                                                                                                                                                                                                                    |
| --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Code Mode** (`docs/agents/code-mode.md`, `reference/tools/create-code-mode.md`)             | The model authors one TypeScript function per query that orchestrates your existing tools (exposed to the generated code as `external_<toolId>`). That orchestration code runs *in a Workspace sandbox*; each `external_*` call bridges back to the real tool on the host. Adds ONE tool, `execute_typescript` (not a raw `execute_command`). Requires a sandbox; `LocalSandbox` "runs the function as a host node process with host privileges." | **BETA** ("breaking changes may occur without a major bump"). Added in `1.38.0` (available under `^1.47.0`). | Solves a *tool-fan-out* problem this single-user agent doesn't have. Its generated code runs sandboxed, but the point of the feature is orchestrating **the agent's own tools**, not "evaluate arbitrary code." Not the shape we want. |
| **Workspace family** (`docs/workspace/overview.md`): Filesystem, Sandbox, LSP, Search, Skills | A `Workspace` bundles five capability areas. Attached via the `workspace` config key on a `Mastra` instance or an `Agent`. **Attaching a Sandbox auto-injects `execute_command`** (+ `get_process_output` / `kill_process` if the sandbox supports background processes); a Filesystem injects `read_file`/`write_file`/`list_directory`/`grep`; LSP injects `mastra_workspace_lsp_inspect`.                                                      | Added `1.1.0`; no beta label. Process manager added `1.7.0`.                                                 | The Sandbox *interface* is exactly what we already implement (see §2.2). The auto-injection-on-attach behavior is the thing P3 forbids at the agent — so we use the interface but never attach.                                        |
| **`WorkspaceSandbox` interface** (`reference/workspace/sandbox.md`)                           | Required: `start()`, `executeCommand(command, args?, options?) → CommandResult`, `destroy()`, `getInfo()`. Optional: `stop?()`, `getInstructions?()`, `processes` (a `SandboxProcessManager`; omit → no background-process tools). File I/O is NOT on this interface — it lives on `WorkspaceFilesystem`.                                                                                                                                         | Added `1.1.0`.                                                                                               | **This is the interface `K8sExecSandbox` already implements.** We're conformant today.                                                                                                                                                 |
| **Built-in sandbox implementations**                                                          | `LocalSandbox` (host `child_process`, `isolation: 'none'` by default — no OS sandbox), `AgentCoreRuntimeSandbox` (AWS Bedrock AgentCore; ships in `@mastra/agentcore`, one-shot only, no background processes, no persistent state, no filesystem mounts), plus cloud/OCI: `E2BSandbox`, `ModalSandbox`, `DaytonaSandbox`, `BlaxelSandbox`, `RailwaySandbox`, `AppleContainerSandbox`, `VercelSandbox`.                                           | `LocalSandbox` `1.1.0`; others per provider.                                                                 | Every one either runs on the host (LocalSandbox — rejected for prod) or leaves the homelab for a cloud (E2B/Modal/etc. — new dep + API key + egress). None is homelab-native.                                                          |
| **Background Tasks** (`docs/long-running-agents/background-tasks.md`)                         | Lets an *agent* dispatch a slow tool call without blocking the agentic loop; built on the same `dountil` under the hood.                                                                                                                                                                                                                                                                                                                          | Added `1.29.0`.                                                                                              | Agent-tool-loop-coupled, not a detached workflow executor. Already evaluated + rejected for the harness (HARNESS.md §2).                                                                                                               |

**Is there a Mastra built-in for "run + evaluate ARBITRARY code in an isolated sandbox, driven from a workflow step (not attached to an agent)"?** No single *named/branded* feature. But the primitives fully support it: you can instantiate a `Workspace`/sandbox standalone and call `sandbox.executeCommand(...)` directly from any code (including a workflow step) without attaching it to an agent. A **custom `WorkspaceSandbox` called directly from a workflow step is a supported, first-class pattern** — which is exactly what we already do. The one honest gap: the docs never show a worked "sandbox inside a workflow step" recipe; we're composing primitives, not using a labeled feature.

### 2.2 What we already own — `K8sExecSandbox` (this is most of the answer)

`images/mastra/src/mastra/durable/executor.ts` is a **working, isolated, arbitrary-code-execution sandbox** that already implements Mastra's `WorkspaceSandbox` interface. It is not aspirational — it is shipped and live on `main` (REWRITE-NOTES turns 107–108: "the durable-build harness is now genuinely LOAD-BEARING").

What it does (mechanism):

* `start()` — creates a bare **Pod** (not a Job) in the `agent` namespace, pinned to i5 (`kind: compute`), image `DURABLE_EXECUTOR_IMAGE` (default `node:22`; has node/npm/tar/git, pnpm via corepack). `sleep infinity`, waits for Ready.
* `writeFiles()` — tars files and streams them over `exec … tar -xmf -` stdin into `/work`. No shell; fixed argv.
* `executeCommand(command, args, opts)` — execs the argv in the pod, captures stdout/stderr/exit. Fixed argv, no string concatenation.
* `readFile()` / `listFiles()` — `cat` / `find` a file back out.
* `destroy()` — deletes the pod (best-effort; `activeDeadlineSeconds` self-reaps a leak).

**Isolation (verified in `cluster/apps/agent/mastra/rbac.yaml`):**

* The pod runs as `durable-build-executor` SA, which has **zero role bindings** and `automountServiceAccountToken: false` (the SA manifest also pins it off) — generated code has **no k8s API access at all**, cannot touch the control plane, cannot read a secret.
* The pod is cap-dropped (`drop: ALL`), `runAsNonRoot` (uid 1000), `seccompProfile: RuntimeDefault`, resource-capped (2 CPU / 2Gi RAM / 2Gi emptyDir), `restartPolicy: Never`, `terminationGracePeriodSeconds: 0`, `activeDeadlineSeconds` hard wall (default 3h).
* mastra itself only holds the namespace-scoped `mastra-durable-build` Role (create/get/list/delete pods, pods/status, pods/exec create+get, pods/log — `agent` ns only). No cross-ns, no secrets, no nodes.

**Shell-free held (verified):** the sandbox is NEVER attached to an Agent — it is driven only from workflow steps. The main agent's tool surface (`main.ts`) sees only `run_durable_build` / `durable_status`; it never gets a command string into the pod.

**The catch — it is web-app-specific today.** The generic *primitive* (create → inject → exec → read → teardown) is fully general, but the *caller* around it (the durable-build harness) is hardwired for web apps:

* `workflow.ts` step `generate-and-evaluate` always runs `evaluateLiveApp(ws, kind, selectors)`, which **serves the app on :3000 and drives browserless**.
* The **hard gate at `workflow.ts:315`**: `passed = verdict.passed && live.started && live.consoleErrors.length === 0`. `live.started` requires an HTTP server to have come up. **A Rust CLI or a typecheck-only change can never set `started: true`, so it can never pass** — the harness is web-only in practice.
* Evaluation evidence is DOM/console/selectors — meaningless for non-web code.

So: **we own a general sandbox wearing a web-app-shaped harness.** The realization to act on is that the sandbox is the reusable asset; the web-app judging is one policy on top of it.

### 2.3 The write-wall reality

* `images/mastra/src/mastra/tools/git-commit.ts:19` — `const WRITE_ROOTS = ["cluster/", "apps-src/"]`. Anything outside these roots is refused. This walls the agent out of `images/` (its own source + all custom-image source).
* Write path: `lib/github.ts` `commitFile` / `commitFiles` — GitHub Contents/Data API with a fine-grained PAT scoped to only this repo, committing straight to `main`. No SSH, no local clone, no node shell. Blast radius is one repo.
* `git-commit` runs a **structural gate** before push (`validate()`): parses YAML for `cluster/`/`*.yaml`, rejects unencrypted `*.sops.yaml`. Web assets under `apps-src/` skip it. There is **no code-level gate** today (there was never code in the write roots).
* `self-edit.ts` is a *separate* narrow write path: it writes the agent's own prompt/config to the PVC (live in seconds) and mirrors to `images/mastra/runtime/{prompts,config}` in git. It proves the agent can already version-control changes to itself — but only its prompt/config, never its code. REWRITE-NOTES turn 105: "self-modification is DOUBLY blocked: can't read its own code, can't write it."
* **`images/` is bigger than assumed.** It holds **9** custom images, not 4: `mastra`, `console`, `feed-scraper`, `cluster-backup`, `riverbot`, `senses`, `kiosk`, `stt`, `tts`. Any `WRITE_ROOTS` widening to "all of `images/`" covers all nine.
* Self-deploy path today: edit `images/<name>/` → CI (`.github/workflows/images.yml`, a matrix over all nine, builds on push to `images/**`) → new `ghcr.io/amarcin/<name>` image → bump the `image:` tag in `cluster/apps/<...>/deployment.yaml` → Flux reconciles. The tag bump lives in `cluster/` — **already inside `WRITE_ROOTS`**, so the agent can already trigger its own redeploy; it just can't change the *code* that gets built.
* mastra's own deployment is `replicas: 1` + `strategy: { type: Recreate }` (libsql single-writer). **Consequence:** a bad mastra image doesn't roll gracefully — Recreate kills the old pod, then the new one crashloops with nothing serving. The agent editing its own running code can ship a crashlooping image and cannot recover a pod it just killed. This is the core danger the verification loop must defend against.

### 2.4 Addressing the prior SKIP verdicts (REWRITE-NOTES turn 102/103, verbatim)

The prior design doc recorded these SKIPs:

* **"Code mode: SKIP — reintroduces a host exec surface (violates shell-free) and solves a fan-out problem this single-user agent doesn't have."**
* **"Mastra Workspaces family: keep Skills + BM25; SKIP filesystem / sandbox / LSP; Search-vector = SKIP."**

**Does that reasoning still hold now that the goal is general code-eval + safe self-modification?**

* **Code mode: STILL SKIP, and the reasoning is unchanged.** Code mode as a Mastra *agent feature* means attaching `execute_typescript` to the main agent — an exec surface on the agent, exactly what P3 forbids. And its purpose (orchestrating the agent's own tools in generated code) is a fan-out problem we still don't have. We are NOT reversing this. Note the distinction that matters: our capability runs arbitrary code in an *isolated pod driven from a workflow*, which is the turn-105 carve-out — categorically different from bolting a code-execution tool onto the agent.
* **"Workspace sandbox = SKIP": ALREADY REVERSED, and correctly.** Turn 102 said "SKIP … sandbox"; turn 105 then built `K8sExecSandbox` — but reconciled it by NOT attaching a Mastra Workspace sandbox to the agent. The turn-102 SKIP was about *attaching Mastra's sandbox to the agent* (which auto-injects `execute_command`). The thing we built and now propose to generalize is a *workflow-internal* sandbox. These look contradictory on the surface; the turn-105 reconciliation ("shell-free binds the main agent's tool surface, not a workflow step running generated code in isolation") is exactly what separates them. This plan is consistent with the reversal, not a new one.
* **Workspace Filesystem / LSP: STILL SKIP.** We don't need Mastra's `WorkspaceFilesystem` (we have `readFile`/`listFiles`/tar-inject on the sandbox) or LSP (typecheck via `tsc` in the pod gives the same signal without a language-server dependency). Adding them buys nothing for this goal and costs surface area (P4).

**Net: no prior SKIP is being reversed by this plan.** Code mode stays skipped. The sandbox "reversal" already happened at turn 105 and this plan builds on it. We are generalizing what we own, not adopting a new Mastra feature.

***

## 3. Options for the broad code-eval capability

### Option A — Generalize the sandbox we already own

Extract the general "create isolated pod → inject files → run commands → read results → teardown" loop from under the web-app harness into a **broad, kind-agnostic "execute + evaluate arbitrary code" capability**. Web-app-build becomes ONE caller (it keeps the browserless live-app judge as a *web-specific evaluation policy*); non-web code (Rust CLI, mastra typecheck+test, a fresh service) is judged by **running its own build/test commands and reading exit codes + output** — no HTTP server required.

* **Pros:** Reuses a shipped, isolation-hardened, RBAC-verified, Mastra-conformant asset — the hardest 80% (the isolated executor + the exec plumbing + the teardown discipline) is already built and load-bearing. It's the most homelab-native (k3s on i5, no cloud, no key, no egress). It directly satisfies P2 (any code) and P3 (workflow-internal, never agent-attached). It's the highest capability-per-line (P4): we're removing a hard gate and adding a kind-generic evaluation path, not writing a new engine. It honors P1 by producing a *few broad* verbs, not many narrow ones.
* **Cons:** The current harness is opinionated around the planner→generator→evaluator, spec.md/findings.md, browserless flow. Generalizing it means carefully separating "run arbitrary code and report" from "build a whole app with a planner + rubric." If done sloppily it becomes a leaky mega-workflow. The single long-lived pod is a single point of failure (HARNESS.md §8), 2Gi RAM may be tight for a real `node_modules` build, and the node/next serve path is less proven than the static one.

### Option B — Adopt a Mastra built-in sandbox

Swap in `E2BSandbox` / `ModalSandbox` / `DaytonaSandbox` (cloud) or `AgentCoreRuntimeSandbox` (AWS), or `LocalSandbox` with `isolation: 'bwrap'`.

* **Pros:** Less code we maintain for the executor itself; the provider handles pod/VM lifecycle. E2B/Modal are purpose-built for "run untrusted code."
* **Cons:** Every cloud option **leaves the homelab** — a new external dependency, an API key, and egress off the fleet for every eval. That contradicts the whole self-hosted posture and was explicitly rejected for the harness (executor.ts header). `AgentCoreRuntimeSandbox` is one-shot only (no background processes, no persistent state) — a poor fit for a multi-command build, and needs `@mastra/agentcore` + AWS creds. `LocalSandbox` runs on the *mastra pod itself* — the exact isolation failure P3/the mandate calls out (generated code as the agent's own process on i5). None is homelab-native; all are strictly worse than what we already run. **We'd be adding a dependency to replace a working, more-isolated component.** Fails P4 hard.

### Option C — Hybrid: our sandbox as the default, a pluggable backend for overflow

Keep `K8sExecSandbox` as the prod backend but formalize the `WorkspaceBackend` seam (already present in `workspace.ts` via `setWorkspaceFactory`) so an E2B/Modal backend is a one-line swap for a future heavy workload our 2Gi pod can't hold.

* **Pros:** All of Option A's benefits, plus a documented escape hatch if a build genuinely outgrows the homelab pod.
* **Cons:** The seam already exists and is dev/test-only today; "formalizing" it for prod overflow is speculative work for a capacity problem we don't have yet (P4). Risks over-engineering.

***

## 4. RECOMMENDATION

**Option A: generalize the sandbox we already own. Do NOT adopt a new Mastra feature.**

The honest answer is that we already own the load-bearing capability. `K8sExecSandbox` is an isolated, tokenless, Mastra-conformant, homelab-native arbitrary-code executor that is shipped and proven. Every Mastra built-in alternative is either less isolated (LocalSandbox on the mastra pod), off-fleet (E2B/Modal/Daytona/AgentCore), or crippled (AgentCore one-shot). The only thing standing between "we have a web-app builder" and "we have a broad code-eval capability" is that the *harness around the sandbox* hard-codes web-app judging. Fix that, and web-app-build becomes one caller of a general capability. Adopting a Mastra sandbox would mean adding a dependency to replace a better component — the opposite of P4. Keep the `setWorkspaceFactory` seam as-is (it already exists for dev/test); do NOT build Option C's prod-overflow backend until a real workload needs it.

### 4.1 Shape of the broad capability

Introduce **one broad workflow + one broad tool**, sitting beside (not replacing) the existing durable-build. The design keeps the existing harness working and adds the general primitive underneath.

**The broad primitive: an "evaluate code" workflow.** A kind-agnostic workflow that:

1. Takes: a set of files (or a git ref / repo path to fetch), an optional setup step, and a **verification command list** (e.g. `["pnpm install", "pnpm typecheck", "pnpm test"]` for a TS package; `["cargo build", "cargo test"]` for Rust; `["ruff check", "pytest"]` for Python). The caller declares what "verified" means as commands + expected exit 0.
2. Spins up a `K8sExecSandbox` (the existing executor, unchanged), injects the files, runs each verification command in order via `sandbox.executeCommand(argv)`, captures exit codes + stdout/stderr tails.
3. Returns a structured **VerificationReport**: per-command exit code + output tail, an overall `passed` (all commands exit 0), and the final file snapshot. **No HTTP-server requirement.** The web-app live-app judge becomes an *optional evaluation policy* the caller can request, not a mandatory gate.

**Web-app-build becomes one caller.** The existing planner→generator→evaluator harness keeps its browserless live-app judge — but expressed as *one evaluation policy* (`kind: web-app` → run `evaluateLiveApp` and apply the console-error gate) layered on the general verify step, rather than as an unconditional gate in the workflow. Refactor the `workflow.ts:315` hard gate so `live.started`/`consoleErrors` only apply when the caller asked for the web-app policy. For a non-web kind, "passed" comes from the verification-command exit codes instead.

**Non-web code, concretely:**

* *A fresh Rust CLI in a scratch dir:* inject the crate, verification commands `["cargo build --release", "cargo test"]`. `passed` = both exit 0. No server, no browser.
* *A mastra typecheck+test change:* inject the changed `images/mastra/**` files (plus enough of the package to build — see §5 phasing / open questions), verification `["pnpm install --frozen-lockfile", "pnpm typecheck", "pnpm test"]`. This is the exact loop that gates a self-modification (§5).
* *Someone else's repo:* fetch by git ref into the pod, run its declared build/test. Purely evaluate; nothing gets committed.

**How it stays shell-free at the main agent (P3):** identical to today. The workflow instantiates and drives the sandbox from *inside its steps*; the sandbox is never attached to an Agent, so `execute_command` is never injected. The main agent gains at most **one broad tool** — `evaluate_code` (kick a verification run; poll status) — mirroring `run_durable_build`/`durable_status`. That tool takes files + a command list and returns a report; the model never gets a shell handle into the pod. The verification commands are fixed argv chosen by the *caller of the workflow*, executed inside the isolated pod, exactly as the generator's `run` action works today.

**On P1 (broad, few tools):** the main-agent tool surface grows by ONE broad tool (`evaluate_code`, plus its status reader), not a family of `run_rust` / `run_python` / `typecheck_mastra` tools. The kind-specific behavior lives in the *command list the caller passes* and in optional evaluation policies inside the workflow — not in the tool count. This is the "large module usable for anything" shape.

### 4.2 Why one workflow, not two separate ones

Durable-build (planner→generator→evaluator, builds an app from a prompt) and evaluate-code (verify a given set of files) are genuinely different jobs. But they share the sandbox and the "run commands, collect a report" core. The clean decomposition:

* **Layer 0 (unchanged):** `K8sExecSandbox` — the isolated executor.
* **Layer 1 (new, broad):** a `verify(files, commands, policy?)` capability — inject, run commands, optional policy (web-app live check), return a report. This is the general "execute + evaluate arbitrary code" module.
* **Layer 2 (existing, refactored to sit on Layer 1):** durable-build — the planner/generator/evaluator loop, whose evaluation step now calls Layer 1 with the web-app policy.

Two callers (durable-build; the new `evaluate_code` tool), one broad primitive underneath. That's the P1/P4 sweet spot: one new broad module, the app-builder becomes a consumer of it.

***

## 5. The write-wall takedown, coupled to the verification loop

The write-wall comes down in a way that makes **an unverified code change structurally impossible to commit.** The coupling is the point: `WRITE_ROOTS` widening is worthless (dangerous, even) without a mandatory verification gate in front of it.

### 5.1 What gets verified, where, before a code commit lands

A **code commit** (a write whose path is under `images/**`) must pass Layer-1 verification *before* `commitFile` is called. Concretely, the widened write path for code:

1. The agent proposes a change to `images/<name>/**` (one or more files).
2. The write tool assembles the **buildable unit** — the changed files plus enough of the package to build/test (for mastra: the `images/mastra/` package; §7 open question on how much context to inject).
3. It runs Layer-1 `verify()` in an isolated pod with the package's **declared verification commands** — for mastra that is at minimum `pnpm install --frozen-lockfile && pnpm typecheck && pnpm test` (the `test`/`typecheck` scripts already exist in `package.json`). For other images, the per-image command list (e.g. a Dockerfile lint + the image's own tests) — declared once per image in a small config, not inferred.
4. **Only if `verify()` passes** does the tool call `commitFile`/`commitFiles` to land the change in `images/**`. A failing verification returns the report to the agent and commits nothing.

This is offline validation *before* the commit — the same discipline `git-commit`'s YAML gate already applies to manifests, extended to code via the sandbox. The verification runs in the isolated pod (P3 held), never on the mastra process.

### 5.2 The `WRITE_ROOTS` change

* Widen `WRITE_ROOTS` from `["cluster/", "apps-src/"]` to include **all of `images/`** (covers all nine custom images). Proposed: `["cluster/", "apps-src/", "images/"]`.
* **But gate by path class**, not one flat allow. Split the write path in `git-commit.ts` (or a sibling tool) into:
  * *Manifest/asset writes* (`cluster/`, `apps-src/`) — unchanged: structural YAML/SOPS gate, then commit. No code verification (they aren't code).
  * *Code writes* (`images/**`) — **must go through Layer-1 verification first** (§5.1). A raw `commitFile` to `images/**` that bypasses verification is refused.
* Keep `self-edit.ts` as-is for prompt/config — it's a fast, non-code runtime path and shouldn't be forced through a container-build verification.

Distinguishing the two powers (per P2 + the prompt): **"evaluate arbitrary code"** (Layer 1, `evaluate_code` tool) needs NO write and NO `WRITE_ROOTS` change — it runs anywhere on any code. **"commit a verified change to a repo path"** (the widened code-write path) needs `WRITE_ROOTS` to include `images/` AND the verification gate. Both are in scope; they're separate tools with separate blast radii.

### 5.3 Self-deploy safety

Self-deploy stays **through git → CI → Flux — never the agent's own `kubectl apply`** of its own new image (AGENTS.md rule). The agent already can bump its own `image:` tag (it's in `cluster/`). The dangers and backstops:

* **Crashloop risk (the sharp edge):** mastra is `replicas: 1` + `Recreate`. A bad mastra image, once the tag is bumped and Flux reconciles, kills the old pod and crashloops the new one — the agent cannot recover a pod it just killed. **Backstops:**
  1. **Verification gate (§5.1)** — `pnpm typecheck && pnpm test` in the sandbox catches the large class of "it doesn't even build/compile" and "tests fail" *before* the code is committed, so a broken build never reaches CI.
  2. **The commit is revertable** — every code change is a git commit on `main`. If a crashloop slips past verification (e.g. a runtime-only failure tests didn't catch), the revert path is: revert the `image:` tag bump in `cluster/apps/agent/mastra/deployment.yaml` to the last-good SHA → Flux rolls back → the previous known-good image runs. Because the *previous image is already built and in GHCR*, the rollback is a pure tag change, fast.
  3. **Human-in-the-loop for the agent's OWN image (recommendation, open decision §7):** a self-modification of `images/mastra` that requires a redeploy of the agent that is doing the modifying is uniquely risky — the agent can brick itself mid-operation. Consider requiring that the *tag bump for mastra specifically* (not the code commit, not other images) either (a) needs an explicit `confirm` gate, or (b) is left for Augustin to trigger, while other images (`riverbot`, `feed-scraper`, etc.) can self-deploy fully. The code commit + verification can be fully autonomous; the "redeploy the brain" step is where a human backstop earns its keep.
* **CI as a second gate:** CI rebuilds the image on push to `images/**`. If the Dockerfile build itself fails, no new image is produced and the old tag keeps running — a natural backstop. (Verification in §5.1 runs the package's tests, not the full Docker build; the Docker build is CI's job and is the belt to verification's suspenders.)
* **Offline manifest validation** (AGENTS.md) still applies to any `cluster/` change the agent makes — one bad manifest wedges all of Flux.

***

## 6. Where the principles create tension (called out honestly)

* **P1 (broad) vs P4 (less code):** a truly broad "evaluate any code" module wants to handle many languages/build systems. The temptation is a big framework that knows about npm/cargo/pip/go. The resolution that keeps both: **the module doesn't know languages — the caller passes the command list.** `verify(files, ["cargo test"])` and `verify(files, ["pnpm test"])` are the same module. Broad *because* it's dumb about specifics. If we start baking per-language logic into the module, we've violated P4; push that into caller-supplied config.
* **P2 (general) vs P3 (isolation):** general code-eval means running genuinely arbitrary, untrusted code. P3's isolation (zero-token pod, no cluster API, cap-dropped, resource-capped, deadline-bounded) is what makes "run anything" safe. These reinforce rather than fight — but only as long as the pod stays zero-privilege. The risk is scope creep: someone later wants the eval pod to reach a real database or the cluster API "just for this build." That must be refused — the moment the eval pod gains privilege, P2 (run anything) becomes a liability. Keep the executor SA at zero bindings, always.
* **P3 (shell-free) vs the goal (self-modification):** self-modification *feels* like it needs a shell. It doesn't. The write is a GitHub API call (`commitFile`), the verification is workflow-internal sandbox exec, the deploy is a git tag bump + Flux. At no point does the main agent get `execute_command`. The turn-105 carve-out is what makes this legal, and this plan stays strictly inside it.
* **Broad tool vs blast radius:** one broad `evaluate_code` tool that runs arbitrary commands is powerful. The blast radius is bounded entirely by the isolated pod, not by the tool's breadth — which is why the isolation (not tool-narrowing) is the safety mechanism. This is the correct P1-compatible way to be safe: broad tool, tight sandbox.

***

## 7. Risks, open questions, and decisions needed from Augustin

**Open questions (need a decision before building):**

1. **How much context to inject for a mastra self-verification?** A `pnpm install --frozen-lockfile && pnpm typecheck && pnpm test` needs the whole `images/mastra/` package (not just the changed file) plus the lockfile in the pod, and 2Gi RAM may be tight for `node_modules` + tsc + vitest (HARNESS.md §8). Options: (a) inject the full package each time (simple, heavy); (b) fetch the package from git at HEAD into the pod and overlay only the changed files (lighter injection, needs git-in-pod fetch, which the pod supports). **Decision needed:** which, and whether to bump the eval pod's RAM for code-verification runs.
2. **Autonomy level for redeploying the agent's own image.** Fully autonomous self-deploy of `images/mastra` (verification passes → commit → tag bump → Flux → agent restarts on the new image) vs. requiring a `confirm` / human trigger for the *mastra-specific* tag bump (§5.3). Other images could be fully autonomous either way. **This is the sharpest decision** — it's the difference between "the agent can brick its own brain" and "the agent proposes, Augustin ships the brain."
3. **Where verification commands per image are declared.** A small per-image config (e.g. `images/<name>/verify.json` listing the command list) vs. inferring from `package.json`/`Cargo.toml`. Explicit config is more predictable (and self-editable), inference is less setup. Recommend explicit config (predictable, versioned, honors "the caller declares what verified means").
4. **Does `evaluate_code` need to fetch remote repos?** P2 says "someone else's repo." Fetching an arbitrary git URL into the pod is easy (pod has git), but it means the eval pod makes outbound network calls. Confirm that's acceptable (the pod is isolated from the *cluster*, but it does have egress for `pnpm install`/`cargo fetch` today). **Decision:** is arbitrary outbound fetch in the eval pod OK, or should remote-repo eval be gated?

**Risks:**

* **Verification is not a proof of correctness.** `typecheck + test` catches compile errors and test regressions, not every runtime failure. A self-modification with passing tests can still crashloop (missing env var, bad startup ordering). The revert path (§5.3) is the mitigation; verification lowers but doesn't eliminate the crashloop risk. Be honest that "verified" means "builds + tests pass," not "cannot break."
* **Single long-lived pod SPOF + sizing** (HARNESS.md §8) — inherited from the existing sandbox; a code-verification run that OOMs the pod fails the verify (safe — it refuses to commit), but a flaky OOM could block legitimate self-improvement. Size the pod for the mastra build.
* **The node/next serve path is less proven than static** (HARNESS.md §6, §8) — only relevant to the web-app policy caller, not to non-web verification.
* **Refactoring the `workflow.ts:315` gate** risks regressing the working web-app harness. Keep the web-app policy behavior byte-identical; only change *when* it applies.

***

## 8. Rough build sequence (phased, delegable)

**Phase 1 — Extract Layer 1 (the broad verify primitive).** New module (e.g. `durable/verify.ts`): `verify(files, commands, policy?)` that spins up `K8sExecSandbox` (unchanged), injects, runs the command list, returns a `VerificationReport`. Pure extraction of the run-commands-collect-report core already present in `workflow.ts`/`workspace.ts`; no new isolation work. Unit-test with the existing local backend (`setWorkspaceFactory`), like `harness-e2e.test.ts`.

**Phase 2 — Refactor durable-build to sit on Layer 1.** Make `evaluateLiveApp` + the `:315` hard gate a *web-app policy* invoked by Layer 1 only when the caller asks; non-web kinds pass/fail on command exit codes. Keep the web-app path behavior-identical (regression-guard with the existing e2e test). No change to the executor or RBAC.

**Phase 3 — The broad `evaluate_code` tool.** One broad main-agent tool (+ a status reader) over Layer 1: takes files/ref + command list, returns the report. Mirrors `run_durable_build`/`durable_status` wiring in `main.ts` + `index.ts`. Satisfies P2 (evaluate any code) with zero `WRITE_ROOTS` change. This is a complete, shippable capability on its own — the agent can now verify arbitrary code before Augustin decides on the write-wall.

**Phase 4 — The verified code-write path.** Split `git-commit.ts` write handling by path class: widen `WRITE_ROOTS` to include `images/`, but route `images/**` writes through a mandatory Layer-1 verification (per-image command list from §7 Q3) before `commitFile`. Manifest/asset writes unchanged. Resolve open questions §7 Q1/Q3 before this phase.

**Phase 5 — Self-deploy safety wiring.** Implement the §5.3 decision for the mastra-specific tag bump (confirm gate or human trigger, per §7 Q2). Document the revert-to-last-good-SHA runbook. Other images can self-deploy per the chosen autonomy level.

Phases 1–3 deliver the broad code-eval capability (P1/P2/P3/P4 satisfied) with no write-wall change and are independently shippable. Phases 4–5 take the wall down safely, gated on Augustin's §7 decisions.

**Phase 6 — Concurrent workstreams + conflict resolution** (separate doc: [`concurrent-workstreams-design.md`](./concurrent-workstreams-design.md)). Once multiple self-editing workstreams run at once, their branches must merge safely: branch-per-workstream isolation (a one-parameter widening of `lib/github.ts`, folds into Phase 4), a merge-coordinator thread driving a durable merge+**re-verify**(this doc's `verify()`)+**deploy-once** workflow, and a conversational LLM resolver (a spawned federation thread, on beta Signals behind `lib/autonomy.ts`) that fires only for a true git conflict or a clean-merge-that-fails-verify. The re-verify coupling is why a git-clean merge of two `images/mastra` edits can't ship a broken brain. See that doc for the full flow, the self-image special case, beta-primitive risk, and its own §10 open decisions.

***

## 9. One-paragraph summary

We already own the hard part: `K8sExecSandbox` is an isolated, tokenless, homelab-native, Mastra-conformant arbitrary-code executor, shipped and load-bearing — wearing a web-app-shaped harness. The right move is to **generalize what we own, not adopt a Mastra built-in** (every built-in is less isolated, off-fleet, or crippled). Extract a broad `verify(files, commands, policy?)` primitive; web-app-build becomes one caller; add a single broad `evaluate_code` tool (P1) that runs any code (P2) in the isolated pod driven from a workflow (P3) with minimal new code (P4). Then take the write-wall down by widening `WRITE_ROOTS` to `images/` **only behind** a mandatory sandbox verification (`typecheck + test`) that runs before any code commit, with self-deploy staying through git→CI→Flux and the Recreate rollout's revert-to-last-good-SHA as the crashloop backstop. Code mode stays SKIPPED (it's an agent exec surface — P3 violation); the sandbox "reversal" already happened at turn 105 and this plan builds on it. The one decision that genuinely needs Augustin: whether the agent may autonomously redeploy its *own brain* (`images/mastra`) after verification, or whether that specific tag bump stays human-triggered.