Reach's Wiki Gets a Verified Backup — and Railway's IaC Nearly Upgraded Four Live Databases on the Way

Why Care?

Reach University's wiki went live three days ago and immediately started holding real work. Until today its database had no backup — Reach's R2 bucket covered Twenty only. Now Outline dumps weekly alongside it, on the same Friday cron.

More importantly: we did not take the deployment's word for it. A green cron service proves nothing, because a cron service doesn't run when you deploy it — it builds, reports SUCCESS, and waits for Friday, with empty logs and an empty bucket. The run was forced, watched, and the resulting archive downloaded from R2 and opened.

What's New?

• pg-dump-outline in the reach-edu project — railwayapp-templates/postgres-s3-backups, cron 0 5 * * 5, SINGLE_SHOT_MODE, restart NEVER, writing to backups/outline-postgres/ in the existing reach-edu R2 bucket.

• Its DB URL is a Railway reference variable, not a hardcoded connection string. pg-dump-twenty duplicates the postgres password inline; this one resolves it from outline-postgres at runtime.

• Proof, not assumption: forced run logged pg_dump (PostgreSQL) 16.11 and "Backup archive file is valid", 37.82 kB uploaded. The object was pulled back out of R2 and inspected — pg_dump directory format, 42 TABLE DATA entries, Outline's schema present (users, documents, collections, teams, revisions, attachments, stars, shares).

The near-miss worth reading

Creating that service needed Railway's IaC, because railway add -r <github repo> returns Unauthorized even with a valid session. So: railway config pull, add one service, railway config plan. The plan came back:

Plan: 1 to add, 7 to change, 0 to destroy
  + Create service pg-dump-outline
  ~ Update twenty-postgres   source.image (unset → "postgres:18")
  ~ Update outline-postgres  source.image (unset → "postgres:18")
  ~ Update postiz-postgres   source.image (unset → "postgres:18")
  ~ Update temporal-postgres source.image (unset → "postgres:18")
  ~ …and both redis → redis:8

Those run 16 / 17-alpine / redis 7 on live volumes. A Postgres major bump is not an upgrade — the data directory is incompatible and the server won't start. Applying would have taken down every database in Reach's project, CRM included.

The cause: railway config pull re-declares each postgres()/redis() with no image, and Railway resolves "no image" to latest at apply time. Pinning all seven to their real versions brought the plan to 1 to add, 0 to change, 0 to destroy — which is the only shape worth applying.

Written up as a reminder: context-v/reminders/Railway-IaC-Pull-Does-Not-Pin-Database-Images.md.

Gotchas

1. A cron service does not run on deploy. Green build, empty logs, empty bucket. Force a run by dropping cronSchedule, applying, watching it exit, then restoring the cron.

2. IaC deletes variables it doesn't declare — anything set via railway variable set after a pull vanishes on the next apply. Declare secrets as preserve(). The plan does say - Delete variable …, so read it.

3. railway config needs the railway npm package — not @railway/sdk, not railway-ts-sdk, both of which 404.

4. Check the pg client against the server. The template built postgresql16-client and outline-postgres is 16 — matched. When they don't match, pg_dump writes an empty archive and still exits zero (that's how twf's postiz backup failed on 2026-08-05).

Left open

restore-runbook.md for Reach's Outline. A backup proven to write is not yet a backup proven to restore, and that distinction is the whole point.