> ## Documentation Index > Fetch the complete documentation index at: https://docs.augustin.ai/llms.txt > Use this file to discover all available pages before exploring further. # Kiosk SPA design — what the i5 screen shows The runtime question is settled: **keep Chrome+cage, and make the kiosk boot ONE URL that never changes** — a self-navigating single-page app that re-renders itself over a live feed, so the browser never navigates again and the relaunch-to-navigate hack becomes dead code (`docs/kiosk-display-runtime-eval.md`, Option C; `docs/kiosk-voice-visual-architecture.md`, §5). This doc is the other half: **WHAT that one page shows** — its modes, how it switches between them client-side, where its data comes from, how it's served, and a phased plan to build it. It is a design, not an implementation; nothing here is built yet. The one hard recommendation: ship a v1 that boots to a **clock/ambient dashboard**, killing today's `about:blank` idle screen, before touching voice or agent-push. This builds on two decided facts and one just-shipped fix: * **Runtime = Chrome+cage, unchanged.** The self-navigating SPA is the whole point of keeping it. * **Content pipeline = `deploy-web-app`.** The agent already ships static apps to `apps.augustin.ai/` in one atomic git commit (ConfigMap → Caddy). No new tooling. * **Touch now works** (this session's Part-1 fix): the kiosk is a real touchscreen again, and the compositor hides the cursor on touch / shows it on mouse. So the SPA is **touch-first**. *** ## 0. Ground truth (what actually exists today) Read out of `cluster/apps/**` + `images/**` this session: | Piece | Reality | Consequence for the SPA | | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **kiosk runtime** | patched cage + Chrome on i5, boots `IDLE_URL` (`about:blank` today), output-only (no Service/ingress). Chrome runs *inside the cluster network*. | The SPA is loaded BY the kiosk browser and runs on i5 inside the cluster — it can hit `*.svc.cluster.local` directly. Boot URL is a one-line env change (`IDLE_URL`). | | **`deploy-web-app`** | `images/mastra/src/mastra/tools/deploy-web-app.ts`: takes `{name, files:[{name,content}]}`, flat files, **must include `index.html`**, writes `apps-src//` + wires a ConfigMap + Caddy mount in ONE commit. **≤1 MiB per app (ConfigMap limit). NO build step** — CDN or inline libs only. | The SPA must be a **single `index.html`** (plus maybe a couple flat assets) under 1 MiB, no bundler. This is the dominant framework constraint (see §3). | | **Caddy / apps.augustin.ai** | `cluster/apps/webapps/`: Caddy serves `/srv/`, SPA fallback `try_files … /index.html`, and sets **`X-Frame-Options: DENY`** on every app. | **Apps on apps.augustin.ai cannot be iframed.** This kills the naive "SPA iframes the pushed app" idea for AGENT\_PUSH (see §2). | | **Home Assistant** | `home-assistant.peripherals.svc.cluster.local:8123`, API/WS only, no Lovelace, **no token wired in git yet**. | Dashboard + light control data source. Needs a long-lived token as a k8s secret before the SPA can call it. | | **mosquitto** | `mosquitto.peripherals.svc.cluster.local:1883`, **raw TCP only — NO WebSocket listener** (verified: no `listener 9001` / `protocol websockets` in its ConfigMap). `allow_anonymous true`. | **A browser SPA cannot speak MQTT** (browsers do MQTT only over WebSocket). This is the single biggest plumbing gap — the feed transport must be solved (see §4). | | **senses** | `senses.peripherals.svc.cluster.local:8080` MJPEG from i5's camera. | Presence/motion source + a possible ambient camera tile. | | **console (other agent)** | `images/console/`: **Next.js 15 + React 19 + Tailwind v4 + assistant-ui**, dark **OKLCH "Cortex"** theme (near-black surfaces, green primary `oklch(0.723 0.219 149.579)`, violet secondary). A full server app. NOT yet deployed. | Styling-share question (see §3): the kiosk SPA can share the *tokens*, not the toolchain. | The takeaway that shapes everything below: the SPA is a **single static HTML file, served from a ConfigMap, running in a browser that lives inside the cluster network**. That is simultaneously very powerful (direct in-cluster HTTP to HA/senses/agent) and very constrained (no build step, ≤1 MiB, no iframe of sibling apps, no raw MQTT). *** ## 1. The modes and how ONE page switches between them From the voice+visual arch doc, the SPA is a **client-side state machine** with five modes. The whole design principle: **mode transitions are DOM/CSS changes in one long-lived page, never navigations.** The page loads once, opens its feed, and swaps what it renders. | Mode | What's on screen | Entered by | | ------------------ | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- | | **IDLE** | Fade to black (or a very dim clock). Backlight low. | Inactivity timer; explicit sleep event. | | **AMBIENT** | Smart-home dashboard: clock, weather, light/climate/media tiles from HA. The default "awake" screen. | Wake: touch (now works), camera-motion, or a pushed wake event. | | **VOICE\_ACTIVE** | Voice visualizer (orb/waveform) over a dimmed dashboard + live transcript. | Wake-word / tap-to-talk (voice pipeline is another agent's design). | | **AGENT\_PUSH** | Arbitrary content the agent pushed ("show me X"). | `kiosk/show {url}` from the agent (remote path). | | **LOCAL\_CONTROL** | A focused device control (light group, thermostat) with big touch targets. | User taps a tile / drills in from AMBIENT. | **The switching mechanism (client-side):** a single top-level state variable (`mode`) drives a `
` element; each mode is a component/section shown or hidden by CSS, and transitions are CSS `opacity`/`transform` cross-fades (300–500 ms). The feed (§4) pushes `{mode, payload}` messages; the SPA reduces them into state. So "fade to black → wake → cross-fade to dashboard → cross-fade to voice orb" is a sequence of state changes in one page, exactly what the relaunch model could never do. The IDLE fade is literally a CSS transition on the persistent DOM — a real cross-fade, not a page reload. **"Show me X" — the hard one, because iframes are blocked.** The founding intent is "show me images of X → pop up on the screen." Three ways to realize AGENT\_PUSH, given Caddy sends `X-Frame-Options: DENY` so the SPA **cannot iframe** a sibling `apps.augustin.ai/`: 1. **Render-in-a-pane (recommended default).** The agent doesn't push a *URL to iframe*; it pushes **structured content** the SPA knows how to render — `{mode:"agent_push", kind:"gallery", items:[{img,caption}]}` or `{kind:"markdown", body:"…"}` or `{kind:"iframe_external", url:"…"}`. The SPA has a small set of built-in renderers (image gallery, markdown/text card, a big headline, a chart). This covers the overwhelmingly common "show me images/a summary/a number" cases without any cross-origin iframe. It's also the most touch/glanceable-friendly. 2. **Hard navigate (escape hatch).** For a genuinely arbitrary web app the agent built with `deploy-web-app`, fall back to the *existing* relaunch/`kiosk/show {url}` path — the SPA (or the navigator) points Chrome at `apps.augustin.ai/` directly. This is a real navigation (accepts the cold-start flash) reserved for the rare "I built you a whole interactive app" case. The self-nav SPA stays the default boot page; this is the deliberate exception. 3. **Iframe an allowlisted internal origin.** If a pushed app is served from an origin that does *not* set `X-Frame-Options: DENY` (e.g. a dedicated content host, or Caddy relaxed to `SAMEORIGIN`/`frame-ancestors self` for a `/embed/*` prefix), the SPA *can* iframe it. Requires a Caddy config change — flagged as an option, not the v1 path. **Recommendation:** default to **#1 (render-in-a-pane with structured payloads)** — it's touch-native, needs no iframe, and keeps the persistent-page property. Keep **#2** as the escape hatch for full apps. Treat **#3** as a future nicety if arbitrary internal apps need embedding. The agent's push payload gains a `kind` discriminator; the SPA owns the renderers. *** ## 2. Data sources and control flow Because the SPA runs in a browser *inside the cluster*, it talks to services directly over HTTP/WS — the same reach the agent's `call-service` has. * **Smart-home dashboard ← Home Assistant.** Two options: * **HA WebSocket API** (`ws://home-assistant.peripherals.svc.cluster.local:8123/api/websocket`) — authenticate with a long-lived token, `subscribe_events` for `state_changed`, get **push** updates. Best for a live dashboard (no polling). * **HA REST API** (`GET /api/states`) — simpler, poll every few seconds. Fine for v1. * Recommend: **REST-poll for v1** (dead simple, single file), **upgrade to WS** once the dashboard is real. Either needs a **long-lived HA token** minted and stored as a k8s secret; the SPA reads it… — and here's a wrinkle: a static SPA has no server to hide a token behind. The token would be embedded in the page (in-cluster, output-only device, no inbound port, so the blast radius is contained, but it IS a token in client JS). **Cleaner: proxy HA through the agent** — the SPA calls the agent (which holds the token via `call-service`) instead of HA directly. Trade-off: adds the agent to the hot path for dashboard reads. For v1 the in-page token is acceptable given the device is inbound-portless and in-cluster; revisit if it bothers. * **Light / device controls → HA.** Per the arch doc's split: * **Touch → HA directly** (REST/WS from the browser): a tap on a light tile calls `POST /api/services/light/turn_on`. Sub-100 ms, no LLM. Right for direct manipulation. * **Voice → agent → `call-service` → HA**: "turn the lights warm" goes through the voice loop so the agent interprets intent. Same HA endpoint, two callers, chosen by modality. * **Voice viz ← the voice pipeline** (another agent's design): the SPA connects to the voice pod's WebSocket for amplitude/partial-transcript/speaking events and drives the visualizer from that signal. Adopt the *visual* components (openai-realtime-blocks / agent-starter-react MIT widgets), re-driven from our WS. Out of scope to design here; the SPA just needs a `voice` mode section and a WS client stub. * **Presence / wake ← touch (now fixed) / camera-motion (senses) / feed events.** Touch is a DOM event in-page (wakes IDLE→AMBIENT instantly, no round-trip). Motion comes from a senses-fed detector publishing a wake event onto the feed. The feed can also carry an explicit wake from the agent. *** ## 3. Framework choice — and should it share the console's styling? **The dominant constraint is `deploy-web-app`: single `index.html`, ≤1 MiB, no build step.** That rules out the console's stack outright — you cannot ship Next.js/React-19/Tailwind-v4/ assistant-ui through a ConfigMap with no bundler. So the honest options are: | Option | Fit for ≤1 MiB / no-build | Pros | Cons | | ------------------------------------------------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Plain HTML + vanilla JS + CSS** | Perfect (one file, zero deps) | Zero toolchain, tiny, fast, matches the existing `apps-src/mileage-tracker` pattern. Trivial for the agent to regenerate/patch. | You hand-write the state machine + DOM diffing. Fine for \~5 modes; gets painful if the UI grows a lot. | | **Lit / Preact / htm via CDN (ESM import)** | Good (import from esm.sh, no build) | Component model + reactive rendering with *no build step* (import maps / ESM). \~5–15 KB. Keeps it a single file. | Slight CDN dependency; still lightweight. **Best balance.** | | **React (CDN UMD) + htm** | OK (no JSX build, use `htm`) | Familiar model; large ecosystem. | Heavier; no JSX without a build; awkward vs Lit/Preact for this size. | | **Same stack as the console (Next/assistant-ui)** | **Impossible via `deploy-web-app`** | Visual consistency with the console. | Needs a real build + its own container image + manifest — a *different* deployment path entirely (a `console`-style pod, not a ConfigMap app). Massive overkill for a glanceable touch screen. | **Recommendation: build the SPA as a single `index.html` using Preact + htm (or Lit) via ESM CDN import**, with inline CSS. It satisfies `deploy-web-app` natively, gives a real component/reactive model for the mode machine, stays well under 1 MiB, and the agent can regenerate it. Vanilla JS is the acceptable fallback if even the CDN dependency is unwanted. **Should it share the console's styling? Partially — share the tokens, not the toolchain, and diverge on layout.** Arguments: * **For sharing:** a consistent "Cortex" look across the desk console and the wall kiosk is nice; the console already defined a considered OKLCH dark palette (near-black surfaces, green primary, violet secondary). Copying those **CSS custom-property values** into the SPA's inline `:root{}` is cheap and gives visual kinship for free. * **Against full sharing:** the two surfaces have genuinely different jobs. The **console** is a lean-in, mouse+keyboard, dense chat/observability app (assistant-ui components, small text, thin scrollbars). The **kiosk** is a lean-back, *glanceable, touch-first, across-the-room* display: huge type, big tap targets (≥48 px), no hover-dependent UI, high-contrast tiles, and a dark ambient aesthetic that fades to black. Those layout/interaction needs are different enough that reusing the console's *components* would fight the medium. **Verdict:** adopt the console's **color tokens** (the OKLCH `@theme` values) for brand kinship, but author kiosk-specific **layout and components** tuned for touch/glanceability. Same palette, different clothes. This keeps them recognizably one family without forcing a touch wall-display into a desktop chat app's component set. *** ## 4. How it's served + deployed — and the feed transport (the real gap) **Serving:** ship the SPA via **`deploy-web-app` as `terminal`** → `apps.augustin.ai/terminal`. Then set the kiosk's `IDLE_URL` env to `https://apps.augustin.ai/terminal` (one line in `cluster/apps/peripherals/kiosk/deployment.yaml`). The kiosk boots straight into it and never navigates again. Updating the UI = the agent re-runs `deploy-web-app terminal` (one commit, Flux reconciles, \~1 min). Zero new agent tooling — exactly the "don't invent a pipeline" principle. A dedicated pod/image (console-style) is the alternative *only if* the SPA outgrows the 1 MiB / no-build ceiling; it should not for a glanceable dashboard. **The feed transport is the one piece of missing plumbing, and the design must pick a lane:** The SPA needs a live push feed to switch modes. mosquitto today is **raw TCP 1883 only — a browser cannot connect to it.** Options: 1. **Add MQTT-over-WebSocket to mosquitto** — add `listener 9001` + `protocol websockets` to the mosquitto ConfigMap and expose 9001 on its Service; the SPA uses MQTT.js over WS. Keeps the existing `kiosk/*` topic design intact; the agent keeps publishing to MQTT as planned. **This is the cleanest fit** for the arch doc's MQTT-centric mode machine. One small manifest change. 2. **Use HA's WebSocket as the bus** — the SPA is already connecting to HA for the dashboard; HA can carry custom events too, and the agent already reaches HA via `call-service`. Fewer new listeners, but overloads HA with kiosk-control semantics and couples the feed to HA being up. 3. **A tiny WS relay in the SPA's own backend** — not available, since `deploy-web-app` apps are static (no server). Rejected. **Recommendation: option 1 — add a WebSocket listener to mosquitto.** It's a \~3-line ConfigMap addition + one Service port, it preserves the whole `kiosk/mode`/`kiosk/show`/`kiosk/wake` topic design from the arch doc unchanged, and it makes the SPA a first-class MQTT client. This is the critical dependency to land before the mode machine can work end-to-end. (Note: it's a peripherals-namespace central change — coordinate; it's the mosquitto owner's manifest, not the kiosk's.) *** ## 5. Touch-first UX (the screen is a touchscreen again) Now that touch works and the cursor hides on touch / shows on mouse (Part-1 fix), design for fingers, not a pointer: * **Big tap targets** — tiles and buttons ≥ 48×48 px (ideally larger for across-the-room use), generous spacing so fat fingers don't misfire. * **No hover-dependence** — every affordance must be visible/reachable without hover. No hover tooltips carrying essential info; no hover-to-reveal controls. Use explicit expand/tap. * **Glanceable hierarchy** — huge clock/primary info readable across the room; secondary info smaller but high-contrast. Dark theme, minimal chrome, content-forward. * **Direct manipulation** — light sliders, toggle tiles respond instantly (HA REST/WS, no LLM in the loop for touch). Optimistic UI (flip the tile immediately, reconcile on the state event). * **The cursor** — hidden during touch by the compositor patch, so a finger UI shows no stray arrow; a mouse (rare, for debugging) reveals it. The SPA can *additionally* set `cursor:none` and toggle on `PointerEvent.pointerType` as defense-in-depth, but the compositor already handles it page-independently, so this is optional belt-and-suspenders. * **Idle → fade to black** — a full-screen CSS opacity transition on inactivity; touch anywhere wakes. No jarring reloads. *** ## 6. Phased plan (v1 first — kill about:blank) Ordered to deliver value early and defer the hardest parts, mirroring the arch doc's phasing but scoped to the SPA content: * **v1 — Boot to an ambient clock/dashboard (kills `about:blank`).** Single `index.html` (`terminal`) with just **IDLE + AMBIENT**: a big clock + weather + a few HA tiles (REST-poll), fade-to-black on idle, tap-to-wake. `deploy-web-app terminal`, flip `IDLE_URL` to it. This alone replaces the blank idle screen with something worth looking at, and proves the persistent-page + HA-data path. **No feed/MQTT needed yet** (touch wake is a local DOM event; HA is polled). Smallest useful thing. * **v2 — LOCAL\_CONTROL + live HA + the feed.** Add device-control tiles (touch → HA directly), upgrade HA reads to the WebSocket subscription for live state, and stand up the **MQTT-over-WS feed** (§4) so the SPA can receive mode events. Now the walk-up local path is complete. * **v3 — AGENT\_PUSH.** Wire the agent's `kiosk/show {url|payload, priority, ttl}` into the feed; implement the render-in-a-pane renderers (gallery/markdown/headline) + the hard-navigate escape hatch; add the interrupt-vs-toast arbitration from the arch doc. Delivers the remote "show me X" path unified with the walk-up path. * **v4 — VOICE\_ACTIVE.** Add the voice visualizer mode + transcript, driven from the voice pod's WS (that pipeline is another agent's design). Last because it depends on the voice pod and is the highest-risk piece. Each phase is a `deploy-web-app terminal` re-ship — one commit, no new infra except the mosquitto WS listener at v2. *** ## 7. Riskiest unknown (flagged) **The browser feed transport, and specifically the HA-token-in-a-static-page tension.** The mode machine is only as good as the live feed, and there is no browser-reachable bus today — mosquitto is TCP-only, and `deploy-web-app` apps are static (no server to broker a WS or hide a token). v1 dodges this (poll HA, local touch wake), but v2+ *requires* solving it. The clean answer (add MQTT-over-WS to mosquitto) is a central manifest change in someone else's service, and the HA long-lived token either lives in client JS (contained but not pretty) or forces every dashboard read through the agent (adds latency + couples the dashboard to the agent). Neither is hard, but both are decisions that touch other components — so the SPA's mode machine can't be fully proven until the feed + token story is nailed down. Everything visual (modes, tiles, transitions, touch UX) is low-risk single-file work; the *plumbing to feed it live* is where the unknowns and cross-service coordination sit. Prove v1 (which needs none of it) first, then land the mosquitto WS listener before committing to v2's live feed. *** ## Sources / inputs * `docs/kiosk-display-runtime-eval.md` — runtime decision (keep Chrome, self-nav SPA). * `docs/kiosk-voice-visual-architecture.md` — the TERMINAL/CONTENT/AGENT split, modes, arbitration, voice. * `images/mastra/src/mastra/tools/deploy-web-app.ts` + `runtime/skills/deploy-web-app/SKILL.md` — content pipeline (single index.html, ≤1 MiB ConfigMap, no build, one commit). * `images/mastra/src/mastra/tools/call-service.ts` — agent's in-cluster HTTP reach to HA (raw `home-assistant…:8123`). * `cluster/apps/webapps/` (Caddyfile ConfigMap, deployment, ingress) — apps.augustin.ai serving + the `X-Frame-Options: DENY` iframe constraint. * `cluster/apps/peripherals/{home-assistant,mosquitto,senses}/` — data sources; mosquitto has NO WebSocket listener (verified). * `images/console/` (package.json, app/globals.css) — the console's Next/assistant-ui stack + OKLCH "Cortex" tokens (styling-share input). * `apps-src/mileage-tracker/index.html` — reference single-file SPA pattern (inline CSS design system, CDN libs) to model the `terminal` app on. * Part-1 fix (this session): touch enumerates in the compositor; patched cage hides cursor on touch / shows on mouse.