Skip to main content
Observability is k8s-native now: a Grafana/Loki/Alloy stack for logs plus Beszel for host metrics. The old Docker-era tools (Homepage, Dozzle, Uptime Kuma) are retired.

​
Tools

  • Grafana — dashboards at grafana.augustin.ai (cluster/infra/monitoring/, HTTPRoute + SealedSecret-managed admin secret).
  • Loki — log store, Longhorn-backed PVC.
  • Alloy — log collector (DaemonSet + RBAC), ships pod logs cluster-wide into Loki.
  • Beszel — self-hosted system metrics at beszel.augustin.ai (cluster/infra/beszel/). Hub Deployment + an agent DaemonSet on every node for CPU/mem/disk/SMART. Longhorn-backed data PVC.

​
Reading Status

Day-to-day, prefer kubectl directly:
  • kubectl get pods -A — is everything running.
  • kubectl logs, kubectl describe — per-pod detail.
  • flux get kustomizations — GitOps reconcile health (a single bad object stalls all of Flux).

​
Alerting

  • The ntfy topic alerts is for human escalations only (e.g. an agent needs a decision). Flux reconcile and CI failures do not notify — whoever pushes owns verifying CI green and flux get kustomizations Ready.

​
Gaps

  • No continuous external prober. Nothing automatically alerts when a public URL goes down; a small scheduled health-check → ntfy is the intended replacement.
  • Beszel can also push per-user Shoutrrr alerts, but host/URL alerting is not unified.