> ## Documentation Index
> Fetch the complete documentation index at: https://docs.augustin.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Reading pipeline

# Reading Pipeline

Chaptarr + CWA replace the old Bookshelf + Boundless setup.

> **Where it runs:** All three services — Chaptarr, CWA, Audiobookshelf — run on **i3 (`debian-1`)** as **Swarm services** on the `swarm` overlay. The earlier "runs on Pentium / inside the gluetun namespace" description is obsolete; the Swarm migration moved them to i3 and put them on the overlay network. Traefik's Swarm provider routes them via `deploy.labels` on `*.augustin.ai` — not the old `traefik/dynamic/pentium-services.yaml` file.

> **Status (updated 2026-06-07):** Working. Chaptarr was upgraded 0.9.439 → 0.9.580, which fixed the import-matching failures (`NO_MATCH_HOLY_GRAIL`). Requests are driven by the **Hardcover → Chaptarr bridge** (`chaptarr/hardcover-bridge.py`, cron every 30 min) since Chaptarr's native import list only adds catalog-*new* items. Reading is via Readest (OPDS off CWA) + KOReader; Readest also pushes progress/highlights back to Hardcover, closing the loop.

## Pipeline (intended)

```
Hardcover library (Want to Read, Currently Reading)
  → hardcover-bridge.py (cron 30min) monitors+searches in Chaptarr   [replaces the native import list, which only adds catalog-new items]
    → Prowlarr searches 8 indexers (1337x, BitSearch, Internet Archive, Knaben, LimeTorrents, Nyaa.si, The Pirate Bay, TorrentGalaxy)
      → qBittorrent downloads via Gluetun VPN
        → Ebooks: /media/books/{Author}/{Title}/        [imports cleanly on Chaptarr 0.9.580]
          → Host cron copies new ebooks to CWA ingest folder
            → CWA auto-ingests into Calibre library
            → CWA auto-converts to epub
            → CWA enriches metadata from Hardcover
            → CWA serves OPDS catalog for KOReader
            → CWA syncs reading progress via KOSync
        → Audiobooks: /media/audiobooks/{Author}/{Title}/
          → Audiobookshelf picks them up
```

## Components

| Service        | Image                                         | Node | Replaces               | Purpose                                 |
| -------------- | --------------------------------------------- | ---- | ---------------------- | --------------------------------------- |
| Chaptarr       | `robertlordhood/chaptarr:latest` (v0.9.580)   | i3   | 2× Bookshelf (Readarr) | Unified ebook + audiobook acquisition   |
| CWA            | `crocodilestick/calibre-web-automated:latest` | i3   | Boundless              | Calibre library, OPDS, KOSync, metadata |
| Audiobookshelf | `ghcr.io/advplyr/audiobookshelf:latest`       | i3   | (unchanged)            | Audiobook streaming                     |
| Prowlarr       | `lscr.io/linuxserver/prowlarr:latest`         | i3   | —                      | Indexer manager (also a Swarm service)  |

### How Chaptarr reaches the download stack

Chaptarr is on the `swarm` overlay, **not** inside gluetun's network namespace, so it talks to the compose-only download stack over the overlay by service-DNS name:

* **Download client:** `gluetun:8085` (qBittorrent, published on the gluetun container)
* **Indexers:** `prowlarr:9696`

## Chaptarr → CWA Ingest

Ebooks reach CWA via one host cron (the dumb `/media/torrents` bypass was retired 2026-06-07 once Chaptarr's matching became reliable):

1. **In-container custom script** — `chaptarr/chaptarr-cwa-ingest.sh`, mounted at `/config/scripts/cwa-ingest.sh`, wired as a Chaptarr "On Import" custom script. **It does not work** — its log only ever records `Test` events; the real import event never fires (Chaptarr notification bug). Dead weight, left in place harmlessly.
2. **Host cron `downloads/cwa-ingest-cron.sh`** — every minute. Polls `/mnt/main/media/books` (where Chaptarr's clean import lands files) for new ebooks and copies them to CWA's ingest folder. This is the live handoff.

> **Retired:** `downloads/torrent-to-cwa.sh` scraped raw `/media/torrents` (including whole series packs) straight into CWA, bypassing Chaptarr's matching. It papered over import failures but polluted the library (e.g. one "Throne of Glass" request dumped 12 Sarah J. Maas books). Removed once Chaptarr 0.9.580 imported reliably; the clean `/media/books` path is the only ingest now.

## CWA Features Enabled

* **OPDS** — `calibre.augustin.ai/opds` (authenticated)
* **KOReader sync** — reading progress across devices
* **Kobo sync** — KOReader Kobo protocol
* **Hardcover sync** — metadata enrichment from Hardcover API
* **Hardcover auto-fetch** — periodic metadata pulls
* **Auto-convert** — target format: epub
* **Kindle EPUB fixer** — fixes common epub issues
* **Duplicate auto-resolution** — strategy: keep newest

## Chaptarr Notes

* Metadata server: `api2.chaptarr.com` (set in Settings → Development; `api.chaptarr.com` has SSL issues)
* Hardcover JWT token stored in Chaptarr's `Config` table (`hardcoverapitoken`, `hardcoverenabled=True`); valid into 2027. Shared conceptually with CWA for metadata.
* API: `http://chaptarr:8789/api/v1` on the overlay; `X-Api-Key` in `chaptarr/config/config.xml`. Note `curl` is **not** in the container — use `wget -qO-` for in-container API calls.

## Known Issues

Verified 2026-06-04. These are why new books stopped flowing around 2026-03-14.

1. **Hardcover import list returns nothing.** Chaptarr's `ImportListSyncService` logs `No list items to process` every cycle. The import list is configured (`Want to Read` + `Currently Reading`, auto-add + search on, user `augustin`), but the **per-list `apiToken` field is empty** (DB-confirmed) even though the global Hardcover token is valid. Nothing new is being pulled in, so there's nothing new to grab. 27 books sit in wanted/missing.
2. **`NO_MATCH_HOLY_GRAIL` import failures.** Completed downloads fail to import in Chaptarr — three are stuck in the queue with `importFailed` / `Rejected: NO_MATCH_HOLY_GRAIL (authorId=…)`. The downloaded files are orphaned under `/media/torrents` and never reach `/media/books` (so they never reach CWA either). Examples stuck: *To Sell Is Human*, *Stillness Is the Key*, *Dungeon Crawler Carl 03*.
3. **Indexer throttling.** TorrentGalaxy returns `429` and self-disables for \~1 day; Knaben hits its API request limit; 1337x has RSS-sync gaps. Transient but recurring — narrows the indexer pool during searches.
4. **Triple/dead ingest** (see above) — the in-container custom script never fires; the two host crons duplicate each other. Both cron logs went silent on 2026-03-14, consistent with nothing new importing into `/media/books` since then (not necessarily a cron failure).
5. **Library counts.** Chaptarr DB: 2,370 book metadata rows, 43 monitored, 17 with files. CWA Calibre library: newest *added* timestamp is 2026-02-22.

## Reading & cross-device sync (the honest map)

Readest was torn down 2026-06-07 — its self-host wasn't worth it for the one thing it uniquely did (cross-device highlights). The current reality, **source-verified against the CWA code**, not assumed:

**CWA's web reader and KOReader/KOSync are two separate progress stores.** They do *not* cross-sync your exact reading position.

* Web reader: position in browser `localStorage` + (manual bookmark only) a CFI in the `bookmark` table, keyed by Calibre book-id (`cps/static/js/reading/epub-progress.js`, `cps/web.py:179`).
* KOSync: a separate `kosync_progress` table keyed by the file's partial-MD5 content hash, storing KOReader's xpointer (`cps/progress_syncing/protocols/kosync.py`).
* Only crossover: a one-way, percent-only fallback **Kindle → browser** (a KOSync push writes a percent into `KoboReadingState`; the web reader jumps to it *only* if there's no prior localStorage/bookmark). Browser → Kindle does not exist.

**What syncs exact position:** KOReader ↔ KOReader through CWA's KOSync. Same epub from CWA OPDS → identical content hash → the real xpointer round-trips.

| Device         | Reader                                            | Exact position synced w/ Kindle?                                              |
| -------------- | ------------------------------------------------- | ----------------------------------------------------------------------------- |
| Kindle         | KOReader                                          | ✅ CWA KOSync                                                                  |
| Phone          | KOReader (Android/iOS)                            | ✅ CWA KOSync                                                                  |
| Linux laptop   | KOReader AppImage (`koreader-v*-x86_64.AppImage`) | ✅ CWA KOSync                                                                  |
| **Mac laptop** | CWA web reader (`calibre.augustin.ai`)            | ❌ separate island — KOReader has no current macOS build (v2026.03 ships none) |

**Net:** with a Mac + Kindle you get *either* exact cross-device position (live entirely in KOReader, which excludes the Mac) *or* a browser reader on the Mac (CWA web reader, an island) — not both, self-hosted. The product that did Mac-browser + Kindle + highlights together was hosted Readest.com (their cloud, not self-host). Practical setup: Kindle + phone on KOReader synced via CWA KOSync; Mac web reader is casual/occasional. Highlights stay per-device; Hardcover is one-way export only.

### KOSync setup (per KOReader device)

KOReader's built-in **Progress sync** plugin speaks the KOSync protocol — no extra plugin needed for progress:

* Tools → **Progress sync** → **Custom sync server** → `https://calibre.augustin.ai/kosync`
* Register / Login with CWA credentials
* Turn on **automatically keep documents in sync**; set document matching to **binary** (content hash) so the same file matches across devices.

## KOReader → Hardcover Progress Sync