The Self-Capable Arc — a Broad Code-Eval Capability + a Safe Write-Wall Takedown
Status: PLAN / design doc for review. No code changed, nothing deployed. Author: architecture pass (Opus 4.8), 2026-07-06. Scope: design a broad, general-purpose “execute + evaluate arbitrary code” capability for the homelab Mastra agent, and the safe removal of the write-wall that currently keeps the agent out of its own source (images/). The last step of the arc is letting the agent commit code changes — but only behind a real, tested verification loop.
Extension — concurrent workstreams + conflict resolution: once the agent self-edits AND self-deploys (all 9 images incl. its own brain, turn 109) with MULTIPLE workstreams running at once, their commits collide. The branch-per-workstream isolation + git-does-the-mechanical-merge + LLM-resolver-for-intent design lives inconcurrent-workstreams-design.md. It builds directly on this doc’sverify()Layer-1 primitive (re-verifying every merge before deploy) and slots into Phases 4–5 below as a new Phase 6.
1. Goal + principles
Goal. Give the main agent a broad capability to safely modify and verify code — its own AND any other code — and, behind that verification, let it commit code changes to the repo. Two distinct powers fall out of this:- Evaluate arbitrary code (broad; no repo write needed): run + test any code in an isolated sandbox and get a real pass/fail. This works for a fresh Rust CLI in a scratch dir, a new service, someone else’s repo — anything, not just the agent’s own image repo.
- Commit a verified change to a repo path (needs a widened
WRITE_ROOTS): once a change has passed the verification loop, land it inimages/**so CI rebuilds and Flux reconciles.
- P1 — Broad modules, not narrow ones. “A few broad tools rather than lots of general-specific ones.” Large, general-purpose capabilities that can be used for anything.
- P2 — Code-eval must be general. “If we’re giving it the power to evaluate code, it should do that for ANY code, not just its own image repo.” The capability is “run + evaluate arbitrary code in an isolated sandbox,” full stop.
- P3 — The main agent stays shell-free. The main agent must never wield a host
execute_command/ command-interpreter tool. The established reconciliation (already in the codebase, REWRITE-NOTES turn 105): running arbitrary code inside an isolated throwaway pod, driven from within a workflow/tool and never attached to an Agent, does NOT violate shell-free — because attaching a sandbox to an Agent is precisely what auto-injectsexecute_command. This plan preserves that: broad execution power, isolated execution, main-agent tool surface never gains a raw host shell. - P4 — Do more with less code. Capability-per-line is the north star. Don’t build a grand framework.
2. Investigation findings (ground truth)
2.1 Mastra’s current relevant features
Pinned version, fromimages/mastra/package.json: @mastra/core: ^1.47.0 (caret; any 1.x ≥ 1.47.0 resolves). REWRITE-NOTES and code comments say 1.49.0 — that’s the resolved lockfile version, consistent with the ^1.47.0 range. @mastra/docker and @mastra/agentcore are not dependencies.
Sources: https://mastra.ai/llms.txt and the doc pages it indexes. All version markers below are the docs’ own “Added in @mastra/core@x.y.z” lines.
| Feature | What it actually does | Maturity | Fit |
|---|---|---|---|
Code Mode (docs/agents/code-mode.md, reference/tools/create-code-mode.md) | The model authors one TypeScript function per query that orchestrates your existing tools (exposed to the generated code as external_<toolId>). That orchestration code runs in a Workspace sandbox; each external_* call bridges back to the real tool on the host. Adds ONE tool, execute_typescript (not a raw execute_command). Requires a sandbox; LocalSandbox “runs the function as a host node process with host privileges.” | BETA (“breaking changes may occur without a major bump”). Added in 1.38.0 (available under ^1.47.0). | Solves a tool-fan-out problem this single-user agent doesn’t have. Its generated code runs sandboxed, but the point of the feature is orchestrating the agent’s own tools, not “evaluate arbitrary code.” Not the shape we want. |
Workspace family (docs/workspace/overview.md): Filesystem, Sandbox, LSP, Search, Skills | A Workspace bundles five capability areas. Attached via the workspace config key on a Mastra instance or an Agent. Attaching a Sandbox auto-injects execute_command (+ get_process_output / kill_process if the sandbox supports background processes); a Filesystem injects read_file/write_file/list_directory/grep; LSP injects mastra_workspace_lsp_inspect. | Added 1.1.0; no beta label. Process manager added 1.7.0. | The Sandbox interface is exactly what we already implement (see §2.2). The auto-injection-on-attach behavior is the thing P3 forbids at the agent — so we use the interface but never attach. |
WorkspaceSandbox interface (reference/workspace/sandbox.md) | Required: start(), executeCommand(command, args?, options?) → CommandResult, destroy(), getInfo(). Optional: stop?(), getInstructions?(), processes (a SandboxProcessManager; omit → no background-process tools). File I/O is NOT on this interface — it lives on WorkspaceFilesystem. | Added 1.1.0. | This is the interface K8sExecSandbox already implements. We’re conformant today. |
| Built-in sandbox implementations | LocalSandbox (host child_process, isolation: 'none' by default — no OS sandbox), AgentCoreRuntimeSandbox (AWS Bedrock AgentCore; ships in @mastra/agentcore, one-shot only, no background processes, no persistent state, no filesystem mounts), plus cloud/OCI: E2BSandbox, ModalSandbox, DaytonaSandbox, BlaxelSandbox, RailwaySandbox, AppleContainerSandbox, VercelSandbox. | LocalSandbox 1.1.0; others per provider. | Every one either runs on the host (LocalSandbox — rejected for prod) or leaves the homelab for a cloud (E2B/Modal/etc. — new dep + API key + egress). None is homelab-native. |
Background Tasks (docs/long-running-agents/background-tasks.md) | Lets an agent dispatch a slow tool call without blocking the agentic loop; built on the same dountil under the hood. | Added 1.29.0. | Agent-tool-loop-coupled, not a detached workflow executor. Already evaluated + rejected for the harness (HARNESS.md §2). |
Workspace/sandbox standalone and call sandbox.executeCommand(...) directly from any code (including a workflow step) without attaching it to an agent. A custom WorkspaceSandbox called directly from a workflow step is a supported, first-class pattern — which is exactly what we already do. The one honest gap: the docs never show a worked “sandbox inside a workflow step” recipe; we’re composing primitives, not using a labeled feature.
2.2 What we already own — K8sExecSandbox (this is most of the answer)
images/mastra/src/mastra/durable/executor.ts is a working, isolated, arbitrary-code-execution sandbox that already implements Mastra’s WorkspaceSandbox interface. It is not aspirational — it is shipped and live on main (REWRITE-NOTES turns 107–108: “the durable-build harness is now genuinely LOAD-BEARING”).
What it does (mechanism):
start()— creates a bare Pod (not a Job) in theagentnamespace, pinned to i5 (kind: compute), imageDURABLE_EXECUTOR_IMAGE(defaultnode:22; has node/npm/tar/git, pnpm via corepack).sleep infinity, waits for Ready.writeFiles()— tars files and streams them overexec … tar -xmf -stdin into/work. No shell; fixed argv.executeCommand(command, args, opts)— execs the argv in the pod, captures stdout/stderr/exit. Fixed argv, no string concatenation.readFile()/listFiles()—cat/finda file back out.destroy()— deletes the pod (best-effort;activeDeadlineSecondsself-reaps a leak).
cluster/apps/agent/mastra/rbac.yaml):
- The pod runs as
durable-build-executorSA, which has zero role bindings andautomountServiceAccountToken: false(the SA manifest also pins it off) — generated code has no k8s API access at all, cannot touch the control plane, cannot read a secret. - The pod is cap-dropped (
drop: ALL),runAsNonRoot(uid 1000),seccompProfile: RuntimeDefault, resource-capped (2 CPU / 2Gi RAM / 2Gi emptyDir),restartPolicy: Never,terminationGracePeriodSeconds: 0,activeDeadlineSecondshard wall (default 3h). - mastra itself only holds the namespace-scoped
mastra-durable-buildRole (create/get/list/delete pods, pods/status, pods/exec create+get, pods/log —agentns only). No cross-ns, no secrets, no nodes.
main.ts) sees only run_durable_build / durable_status; it never gets a command string into the pod.
The catch — it is web-app-specific today. The generic primitive (create → inject → exec → read → teardown) is fully general, but the caller around it (the durable-build harness) is hardwired for web apps:
workflow.tsstepgenerate-and-evaluatealways runsevaluateLiveApp(ws, kind, selectors), which serves the app on :3000 and drives browserless.- The hard gate at
workflow.ts:315:passed = verdict.passed && live.started && live.consoleErrors.length === 0.live.startedrequires an HTTP server to have come up. A Rust CLI or a typecheck-only change can never setstarted: true, so it can never pass — the harness is web-only in practice. - Evaluation evidence is DOM/console/selectors — meaningless for non-web code.
2.3 The write-wall reality
images/mastra/src/mastra/tools/git-commit.ts:19—const WRITE_ROOTS = ["cluster/", "apps-src/"]. Anything outside these roots is refused. This walls the agent out ofimages/(its own source + all custom-image source).- Write path:
lib/github.tscommitFile/commitFiles— GitHub Contents/Data API with a fine-grained PAT scoped to only this repo, committing straight tomain. No SSH, no local clone, no node shell. Blast radius is one repo. git-commitruns a structural gate before push (validate()): parses YAML forcluster//*.yaml, rejects unencrypted*.sops.yaml. Web assets underapps-src/skip it. There is no code-level gate today (there was never code in the write roots).self-edit.tsis a separate narrow write path: it writes the agent’s own prompt/config to the PVC (live in seconds) and mirrors toimages/mastra/runtime/{prompts,config}in git. It proves the agent can already version-control changes to itself — but only its prompt/config, never its code. REWRITE-NOTES turn 105: “self-modification is DOUBLY blocked: can’t read its own code, can’t write it.”images/is bigger than assumed. It holds 9 custom images, not 4:mastra,console,feed-scraper,cluster-backup,riverbot,senses,kiosk,stt,tts. AnyWRITE_ROOTSwidening to “all ofimages/” covers all nine.- Self-deploy path today: edit
images/<name>/→ CI (.github/workflows/images.yml, a matrix over all nine, builds on push toimages/**) → newghcr.io/amarcin/<name>image → bump theimage:tag incluster/apps/<...>/deployment.yaml→ Flux reconciles. The tag bump lives incluster/— already insideWRITE_ROOTS, so the agent can already trigger its own redeploy; it just can’t change the code that gets built. - mastra’s own deployment is
replicas: 1+strategy: { type: Recreate }(libsql single-writer). Consequence: a bad mastra image doesn’t roll gracefully — Recreate kills the old pod, then the new one crashloops with nothing serving. The agent editing its own running code can ship a crashlooping image and cannot recover a pod it just killed. This is the core danger the verification loop must defend against.
2.4 Addressing the prior SKIP verdicts (REWRITE-NOTES turn 102/103, verbatim)
The prior design doc recorded these SKIPs:- “Code mode: SKIP — reintroduces a host exec surface (violates shell-free) and solves a fan-out problem this single-user agent doesn’t have.”
- “Mastra Workspaces family: keep Skills + BM25; SKIP filesystem / sandbox / LSP; Search-vector = SKIP.”
- Code mode: STILL SKIP, and the reasoning is unchanged. Code mode as a Mastra agent feature means attaching
execute_typescriptto the main agent — an exec surface on the agent, exactly what P3 forbids. And its purpose (orchestrating the agent’s own tools in generated code) is a fan-out problem we still don’t have. We are NOT reversing this. Note the distinction that matters: our capability runs arbitrary code in an isolated pod driven from a workflow, which is the turn-105 carve-out — categorically different from bolting a code-execution tool onto the agent. - “Workspace sandbox = SKIP”: ALREADY REVERSED, and correctly. Turn 102 said “SKIP … sandbox”; turn 105 then built
K8sExecSandbox— but reconciled it by NOT attaching a Mastra Workspace sandbox to the agent. The turn-102 SKIP was about attaching Mastra’s sandbox to the agent (which auto-injectsexecute_command). The thing we built and now propose to generalize is a workflow-internal sandbox. These look contradictory on the surface; the turn-105 reconciliation (“shell-free binds the main agent’s tool surface, not a workflow step running generated code in isolation”) is exactly what separates them. This plan is consistent with the reversal, not a new one. - Workspace Filesystem / LSP: STILL SKIP. We don’t need Mastra’s
WorkspaceFilesystem(we havereadFile/listFiles/tar-inject on the sandbox) or LSP (typecheck viatscin the pod gives the same signal without a language-server dependency). Adding them buys nothing for this goal and costs surface area (P4).
3. Options for the broad code-eval capability
Option A — Generalize the sandbox we already own
Extract the general “create isolated pod → inject files → run commands → read results → teardown” loop from under the web-app harness into a broad, kind-agnostic “execute + evaluate arbitrary code” capability. Web-app-build becomes ONE caller (it keeps the browserless live-app judge as a web-specific evaluation policy); non-web code (Rust CLI, mastra typecheck+test, a fresh service) is judged by running its own build/test commands and reading exit codes + output — no HTTP server required.- Pros: Reuses a shipped, isolation-hardened, RBAC-verified, Mastra-conformant asset — the hardest 80% (the isolated executor + the exec plumbing + the teardown discipline) is already built and load-bearing. It’s the most homelab-native (k3s on i5, no cloud, no key, no egress). It directly satisfies P2 (any code) and P3 (workflow-internal, never agent-attached). It’s the highest capability-per-line (P4): we’re removing a hard gate and adding a kind-generic evaluation path, not writing a new engine. It honors P1 by producing a few broad verbs, not many narrow ones.
- Cons: The current harness is opinionated around the planner→generator→evaluator, spec.md/findings.md, browserless flow. Generalizing it means carefully separating “run arbitrary code and report” from “build a whole app with a planner + rubric.” If done sloppily it becomes a leaky mega-workflow. The single long-lived pod is a single point of failure (HARNESS.md §8), 2Gi RAM may be tight for a real
node_modulesbuild, and the node/next serve path is less proven than the static one.
Option B — Adopt a Mastra built-in sandbox
Swap inE2BSandbox / ModalSandbox / DaytonaSandbox (cloud) or AgentCoreRuntimeSandbox (AWS), or LocalSandbox with isolation: 'bwrap'.
- Pros: Less code we maintain for the executor itself; the provider handles pod/VM lifecycle. E2B/Modal are purpose-built for “run untrusted code.”
- Cons: Every cloud option leaves the homelab — a new external dependency, an API key, and egress off the fleet for every eval. That contradicts the whole self-hosted posture and was explicitly rejected for the harness (executor.ts header).
AgentCoreRuntimeSandboxis one-shot only (no background processes, no persistent state) — a poor fit for a multi-command build, and needs@mastra/agentcore+ AWS creds.LocalSandboxruns on the mastra pod itself — the exact isolation failure P3/the mandate calls out (generated code as the agent’s own process on i5). None is homelab-native; all are strictly worse than what we already run. We’d be adding a dependency to replace a working, more-isolated component. Fails P4 hard.
Option C — Hybrid: our sandbox as the default, a pluggable backend for overflow
KeepK8sExecSandbox as the prod backend but formalize the WorkspaceBackend seam (already present in workspace.ts via setWorkspaceFactory) so an E2B/Modal backend is a one-line swap for a future heavy workload our 2Gi pod can’t hold.
- Pros: All of Option A’s benefits, plus a documented escape hatch if a build genuinely outgrows the homelab pod.
- Cons: The seam already exists and is dev/test-only today; “formalizing” it for prod overflow is speculative work for a capacity problem we don’t have yet (P4). Risks over-engineering.
4. RECOMMENDATION
Option A: generalize the sandbox we already own. Do NOT adopt a new Mastra feature. The honest answer is that we already own the load-bearing capability.K8sExecSandbox is an isolated, tokenless, Mastra-conformant, homelab-native arbitrary-code executor that is shipped and proven. Every Mastra built-in alternative is either less isolated (LocalSandbox on the mastra pod), off-fleet (E2B/Modal/Daytona/AgentCore), or crippled (AgentCore one-shot). The only thing standing between “we have a web-app builder” and “we have a broad code-eval capability” is that the harness around the sandbox hard-codes web-app judging. Fix that, and web-app-build becomes one caller of a general capability. Adopting a Mastra sandbox would mean adding a dependency to replace a better component — the opposite of P4. Keep the setWorkspaceFactory seam as-is (it already exists for dev/test); do NOT build Option C’s prod-overflow backend until a real workload needs it.
4.1 Shape of the broad capability
Introduce one broad workflow + one broad tool, sitting beside (not replacing) the existing durable-build. The design keeps the existing harness working and adds the general primitive underneath. The broad primitive: an “evaluate code” workflow. A kind-agnostic workflow that:- Takes: a set of files (or a git ref / repo path to fetch), an optional setup step, and a verification command list (e.g.
["pnpm install", "pnpm typecheck", "pnpm test"]for a TS package;["cargo build", "cargo test"]for Rust;["ruff check", "pytest"]for Python). The caller declares what “verified” means as commands + expected exit 0. - Spins up a
K8sExecSandbox(the existing executor, unchanged), injects the files, runs each verification command in order viasandbox.executeCommand(argv), captures exit codes + stdout/stderr tails. - Returns a structured VerificationReport: per-command exit code + output tail, an overall
passed(all commands exit 0), and the final file snapshot. No HTTP-server requirement. The web-app live-app judge becomes an optional evaluation policy the caller can request, not a mandatory gate.
kind: web-app → run evaluateLiveApp and apply the console-error gate) layered on the general verify step, rather than as an unconditional gate in the workflow. Refactor the workflow.ts:315 hard gate so live.started/consoleErrors only apply when the caller asked for the web-app policy. For a non-web kind, “passed” comes from the verification-command exit codes instead.
Non-web code, concretely:
- A fresh Rust CLI in a scratch dir: inject the crate, verification commands
["cargo build --release", "cargo test"].passed= both exit 0. No server, no browser. - A mastra typecheck+test change: inject the changed
images/mastra/**files (plus enough of the package to build — see §5 phasing / open questions), verification["pnpm install --frozen-lockfile", "pnpm typecheck", "pnpm test"]. This is the exact loop that gates a self-modification (§5). - Someone else’s repo: fetch by git ref into the pod, run its declared build/test. Purely evaluate; nothing gets committed.
execute_command is never injected. The main agent gains at most one broad tool — evaluate_code (kick a verification run; poll status) — mirroring run_durable_build/durable_status. That tool takes files + a command list and returns a report; the model never gets a shell handle into the pod. The verification commands are fixed argv chosen by the caller of the workflow, executed inside the isolated pod, exactly as the generator’s run action works today.
On P1 (broad, few tools): the main-agent tool surface grows by ONE broad tool (evaluate_code, plus its status reader), not a family of run_rust / run_python / typecheck_mastra tools. The kind-specific behavior lives in the command list the caller passes and in optional evaluation policies inside the workflow — not in the tool count. This is the “large module usable for anything” shape.
4.2 Why one workflow, not two separate ones
Durable-build (planner→generator→evaluator, builds an app from a prompt) and evaluate-code (verify a given set of files) are genuinely different jobs. But they share the sandbox and the “run commands, collect a report” core. The clean decomposition:- Layer 0 (unchanged):
K8sExecSandbox— the isolated executor. - Layer 1 (new, broad): a
verify(files, commands, policy?)capability — inject, run commands, optional policy (web-app live check), return a report. This is the general “execute + evaluate arbitrary code” module. - Layer 2 (existing, refactored to sit on Layer 1): durable-build — the planner/generator/evaluator loop, whose evaluation step now calls Layer 1 with the web-app policy.
evaluate_code tool), one broad primitive underneath. That’s the P1/P4 sweet spot: one new broad module, the app-builder becomes a consumer of it.
5. The write-wall takedown, coupled to the verification loop
The write-wall comes down in a way that makes an unverified code change structurally impossible to commit. The coupling is the point:WRITE_ROOTS widening is worthless (dangerous, even) without a mandatory verification gate in front of it.
5.1 What gets verified, where, before a code commit lands
A code commit (a write whose path is underimages/**) must pass Layer-1 verification before commitFile is called. Concretely, the widened write path for code:
- The agent proposes a change to
images/<name>/**(one or more files). - The write tool assembles the buildable unit — the changed files plus enough of the package to build/test (for mastra: the
images/mastra/package; §7 open question on how much context to inject). - It runs Layer-1
verify()in an isolated pod with the package’s declared verification commands — for mastra that is at minimumpnpm install --frozen-lockfile && pnpm typecheck && pnpm test(thetest/typecheckscripts already exist inpackage.json). For other images, the per-image command list (e.g. a Dockerfile lint + the image’s own tests) — declared once per image in a small config, not inferred. - Only if
verify()passes does the tool callcommitFile/commitFilesto land the change inimages/**. A failing verification returns the report to the agent and commits nothing.
git-commit’s YAML gate already applies to manifests, extended to code via the sandbox. The verification runs in the isolated pod (P3 held), never on the mastra process.
5.2 The WRITE_ROOTS change
- Widen
WRITE_ROOTSfrom["cluster/", "apps-src/"]to include all ofimages/(covers all nine custom images). Proposed:["cluster/", "apps-src/", "images/"]. - But gate by path class, not one flat allow. Split the write path in
git-commit.ts(or a sibling tool) into:- Manifest/asset writes (
cluster/,apps-src/) — unchanged: structural YAML/SOPS gate, then commit. No code verification (they aren’t code). - Code writes (
images/**) — must go through Layer-1 verification first (§5.1). A rawcommitFiletoimages/**that bypasses verification is refused.
- Manifest/asset writes (
- Keep
self-edit.tsas-is for prompt/config — it’s a fast, non-code runtime path and shouldn’t be forced through a container-build verification.
evaluate_code tool) needs NO write and NO WRITE_ROOTS change — it runs anywhere on any code. “commit a verified change to a repo path” (the widened code-write path) needs WRITE_ROOTS to include images/ AND the verification gate. Both are in scope; they’re separate tools with separate blast radii.
5.3 Self-deploy safety
Self-deploy stays through git → CI → Flux — never the agent’s ownkubectl apply of its own new image (AGENTS.md rule). The agent already can bump its own image: tag (it’s in cluster/). The dangers and backstops:
- Crashloop risk (the sharp edge): mastra is
replicas: 1+Recreate. A bad mastra image, once the tag is bumped and Flux reconciles, kills the old pod and crashloops the new one — the agent cannot recover a pod it just killed. Backstops:- Verification gate (§5.1) —
pnpm typecheck && pnpm testin the sandbox catches the large class of “it doesn’t even build/compile” and “tests fail” before the code is committed, so a broken build never reaches CI. - The commit is revertable — every code change is a git commit on
main. If a crashloop slips past verification (e.g. a runtime-only failure tests didn’t catch), the revert path is: revert theimage:tag bump incluster/apps/agent/mastra/deployment.yamlto the last-good SHA → Flux rolls back → the previous known-good image runs. Because the previous image is already built and in GHCR, the rollback is a pure tag change, fast. - Human-in-the-loop for the agent’s OWN image (recommendation, open decision §7): a self-modification of
images/mastrathat requires a redeploy of the agent that is doing the modifying is uniquely risky — the agent can brick itself mid-operation. Consider requiring that the tag bump for mastra specifically (not the code commit, not other images) either (a) needs an explicitconfirmgate, or (b) is left for Augustin to trigger, while other images (riverbot,feed-scraper, etc.) can self-deploy fully. The code commit + verification can be fully autonomous; the “redeploy the brain” step is where a human backstop earns its keep.
- Verification gate (§5.1) —
- CI as a second gate: CI rebuilds the image on push to
images/**. If the Dockerfile build itself fails, no new image is produced and the old tag keeps running — a natural backstop. (Verification in §5.1 runs the package’s tests, not the full Docker build; the Docker build is CI’s job and is the belt to verification’s suspenders.) - Offline manifest validation (AGENTS.md) still applies to any
cluster/change the agent makes — one bad manifest wedges all of Flux.
6. Where the principles create tension (called out honestly)
- P1 (broad) vs P4 (less code): a truly broad “evaluate any code” module wants to handle many languages/build systems. The temptation is a big framework that knows about npm/cargo/pip/go. The resolution that keeps both: the module doesn’t know languages — the caller passes the command list.
verify(files, ["cargo test"])andverify(files, ["pnpm test"])are the same module. Broad because it’s dumb about specifics. If we start baking per-language logic into the module, we’ve violated P4; push that into caller-supplied config. - P2 (general) vs P3 (isolation): general code-eval means running genuinely arbitrary, untrusted code. P3’s isolation (zero-token pod, no cluster API, cap-dropped, resource-capped, deadline-bounded) is what makes “run anything” safe. These reinforce rather than fight — but only as long as the pod stays zero-privilege. The risk is scope creep: someone later wants the eval pod to reach a real database or the cluster API “just for this build.” That must be refused — the moment the eval pod gains privilege, P2 (run anything) becomes a liability. Keep the executor SA at zero bindings, always.
- P3 (shell-free) vs the goal (self-modification): self-modification feels like it needs a shell. It doesn’t. The write is a GitHub API call (
commitFile), the verification is workflow-internal sandbox exec, the deploy is a git tag bump + Flux. At no point does the main agent getexecute_command. The turn-105 carve-out is what makes this legal, and this plan stays strictly inside it. - Broad tool vs blast radius: one broad
evaluate_codetool that runs arbitrary commands is powerful. The blast radius is bounded entirely by the isolated pod, not by the tool’s breadth — which is why the isolation (not tool-narrowing) is the safety mechanism. This is the correct P1-compatible way to be safe: broad tool, tight sandbox.
7. Risks, open questions, and decisions needed from Augustin
Open questions (need a decision before building):- How much context to inject for a mastra self-verification? A
pnpm install --frozen-lockfile && pnpm typecheck && pnpm testneeds the wholeimages/mastra/package (not just the changed file) plus the lockfile in the pod, and 2Gi RAM may be tight fornode_modules+ tsc + vitest (HARNESS.md §8). Options: (a) inject the full package each time (simple, heavy); (b) fetch the package from git at HEAD into the pod and overlay only the changed files (lighter injection, needs git-in-pod fetch, which the pod supports). Decision needed: which, and whether to bump the eval pod’s RAM for code-verification runs. - Autonomy level for redeploying the agent’s own image. Fully autonomous self-deploy of
images/mastra(verification passes → commit → tag bump → Flux → agent restarts on the new image) vs. requiring aconfirm/ human trigger for the mastra-specific tag bump (§5.3). Other images could be fully autonomous either way. This is the sharpest decision — it’s the difference between “the agent can brick its own brain” and “the agent proposes, Augustin ships the brain.” - Where verification commands per image are declared. A small per-image config (e.g.
images/<name>/verify.jsonlisting the command list) vs. inferring frompackage.json/Cargo.toml. Explicit config is more predictable (and self-editable), inference is less setup. Recommend explicit config (predictable, versioned, honors “the caller declares what verified means”). - Does
evaluate_codeneed to fetch remote repos? P2 says “someone else’s repo.” Fetching an arbitrary git URL into the pod is easy (pod has git), but it means the eval pod makes outbound network calls. Confirm that’s acceptable (the pod is isolated from the cluster, but it does have egress forpnpm install/cargo fetchtoday). Decision: is arbitrary outbound fetch in the eval pod OK, or should remote-repo eval be gated?
- Verification is not a proof of correctness.
typecheck + testcatches compile errors and test regressions, not every runtime failure. A self-modification with passing tests can still crashloop (missing env var, bad startup ordering). The revert path (§5.3) is the mitigation; verification lowers but doesn’t eliminate the crashloop risk. Be honest that “verified” means “builds + tests pass,” not “cannot break.” - Single long-lived pod SPOF + sizing (HARNESS.md §8) — inherited from the existing sandbox; a code-verification run that OOMs the pod fails the verify (safe — it refuses to commit), but a flaky OOM could block legitimate self-improvement. Size the pod for the mastra build.
- The node/next serve path is less proven than static (HARNESS.md §6, §8) — only relevant to the web-app policy caller, not to non-web verification.
- Refactoring the
workflow.ts:315gate risks regressing the working web-app harness. Keep the web-app policy behavior byte-identical; only change when it applies.
8. Rough build sequence (phased, delegable)
Phase 1 — Extract Layer 1 (the broad verify primitive). New module (e.g.durable/verify.ts): verify(files, commands, policy?) that spins up K8sExecSandbox (unchanged), injects, runs the command list, returns a VerificationReport. Pure extraction of the run-commands-collect-report core already present in workflow.ts/workspace.ts; no new isolation work. Unit-test with the existing local backend (setWorkspaceFactory), like harness-e2e.test.ts.
Phase 2 — Refactor durable-build to sit on Layer 1. Make evaluateLiveApp + the :315 hard gate a web-app policy invoked by Layer 1 only when the caller asks; non-web kinds pass/fail on command exit codes. Keep the web-app path behavior-identical (regression-guard with the existing e2e test). No change to the executor or RBAC.
Phase 3 — The broad evaluate_code tool. One broad main-agent tool (+ a status reader) over Layer 1: takes files/ref + command list, returns the report. Mirrors run_durable_build/durable_status wiring in main.ts + index.ts. Satisfies P2 (evaluate any code) with zero WRITE_ROOTS change. This is a complete, shippable capability on its own — the agent can now verify arbitrary code before Augustin decides on the write-wall.
Phase 4 — The verified code-write path. Split git-commit.ts write handling by path class: widen WRITE_ROOTS to include images/, but route images/** writes through a mandatory Layer-1 verification (per-image command list from §7 Q3) before commitFile. Manifest/asset writes unchanged. Resolve open questions §7 Q1/Q3 before this phase.
Phase 5 — Self-deploy safety wiring. Implement the §5.3 decision for the mastra-specific tag bump (confirm gate or human trigger, per §7 Q2). Document the revert-to-last-good-SHA runbook. Other images can self-deploy per the chosen autonomy level.
Phases 1–3 deliver the broad code-eval capability (P1/P2/P3/P4 satisfied) with no write-wall change and are independently shippable. Phases 4–5 take the wall down safely, gated on Augustin’s §7 decisions.
Phase 6 — Concurrent workstreams + conflict resolution (separate doc: concurrent-workstreams-design.md). Once multiple self-editing workstreams run at once, their branches must merge safely: branch-per-workstream isolation (a one-parameter widening of lib/github.ts, folds into Phase 4), a merge-coordinator thread driving a durable merge+re-verify(this doc’s verify())+deploy-once workflow, and a conversational LLM resolver (a spawned federation thread, on beta Signals behind lib/autonomy.ts) that fires only for a true git conflict or a clean-merge-that-fails-verify. The re-verify coupling is why a git-clean merge of two images/mastra edits can’t ship a broken brain. See that doc for the full flow, the self-image special case, beta-primitive risk, and its own §10 open decisions.
9. One-paragraph summary
We already own the hard part:K8sExecSandbox is an isolated, tokenless, homelab-native, Mastra-conformant arbitrary-code executor, shipped and load-bearing — wearing a web-app-shaped harness. The right move is to generalize what we own, not adopt a Mastra built-in (every built-in is less isolated, off-fleet, or crippled). Extract a broad verify(files, commands, policy?) primitive; web-app-build becomes one caller; add a single broad evaluate_code tool (P1) that runs any code (P2) in the isolated pod driven from a workflow (P3) with minimal new code (P4). Then take the write-wall down by widening WRITE_ROOTS to images/ only behind a mandatory sandbox verification (typecheck + test) that runs before any code commit, with self-deploy staying through git→CI→Flux and the Recreate rollout’s revert-to-last-good-SHA as the crashloop backstop. Code mode stays SKIPPED (it’s an agent exec surface — P3 violation); the sandbox “reversal” already happened at turn 105 and this plan builds on it. The one decision that genuinely needs Augustin: whether the agent may autonomously redeploy its own brain (images/mastra) after verification, or whether that specific tag bump stays human-triggered.