maintenance
This page says how the repository stays current. It tracks the Matrix specification, the proposals that Spindle serves ahead of the spec, and the clients and suites that test it. It also says what a maintainer does when one of them moves.
The principle throughout: a gate holds every claim to the code, and a job watches every upstream. Nothing here is a checklist that a person must remember. Each item is a script. It runs in CI or on a schedule, prints what it found, and fails or files an issue. A person still makes the judgement calls. These are whether to bump a pin, whether a new endpoint is in scope, and what to do when a proposal dies.
1. The gates, per pull request
ci.yml runs these on every push. Each one answers "does a generated
artefact still tell the truth about the code":
| Gate | Holds | Regenerate with |
|---|---|---|
coverage-dashboard.py --check | docs/dashboard.md: the endpoints the router serves, per milestone | just regen |
spec-drift.py --check | docs/spec-gaps.md: what the pinned spec defines that the router does not serve | just regen |
msc-ledger.py --check | docs/mscs.md, and that /versions' flags and the unstable/ routes are all owned by a served entry in contrib/msc/ledger.toml with a test naming the MSC | just regen after editing the ledger |
upstream-pins.py --check | that every hand-pinned upstream can still be read from its file | fix the regex or the file |
ruma-drift.py --check | vendor/ruma-state-res is its crates.io base plus exactly the hunks SPINDLE-PATCH.md lists | §8; ruma-drift.py --hunks prints the rows |
openapi-check.py | every response the scripted client gets validates against the pinned spec's schema | scripts/openapi-allowlist.txt for an explained divergence |
readme-numbers.py --check | the README's headline counts are the gated ones | just regen |
config-drift.py | the example config names every setting | edit spindle.example.toml |
untested-routes.py | (report) routes nothing names in a test, rig or evidence page | write the test |
tests/surface.rs | /versions advertises nothing unbuilt | build it, then advertise it |
Run just lint, just test and just regen before you push. Together they
reproduce all of it.
2. The ratchets, nightly
Three external suites run against a fresh build every night from
compliance.yml. Element Call's suite runs twice: against two Spindles,
and against one Spindle and upstream's Synapse (SPINDLE_PEER=synapse).
Each run has an allowlist that may only grow:
| Suite | Pin | Allowlist | Check |
|---|---|---|---|
| Complement | scripts/complement.sh | complement/allowlist.txt | scripts/complement-check.py |
| matrix-rust-sdk integration tests | contrib/rust-sdk/run.sh | contrib/rust-sdk/allowlist.txt | scripts/rust-sdk-check.py |
| Element Call Playwright | contrib/element-call/run.sh | contrib/element-call/allowlist.txt | scripts/element-call-check.py |
| Element Call Playwright, Synapse peer | contrib/element-call/run.sh | contrib/element-call/allowlist-synapse-peer.txt | scripts/element-call-check.py |
The job prints a test that passes but is not on the list as a candidate. A reviewed commit promotes it, and the commit names the run it passed in. Promotion is never automatic, so people do not learn to ignore the gate because of one flaky run. A listed test that fails is a regression and fails the job. The procedure:
- Read the job's tail: the
candidateandREGRESSEDlines. - For a regression, fix the code first. Do not edit the allowlist to make the test pass. (A test that upstream changed under the same name is the one exception, and the commit says so.)
- For candidates, add them under a header naming the run id and the Spindle commit, one pull request per run.
The maintainers triage the remaining failures in each suite on one issue per
suite (#112 for the SDK suite). Each is a real gap or a real defect; the
sdk_findings tests in crates/spindle-server/tests/ are the unit-sized
reproductions of what the suite found.
3. The weekly upkeep report
upkeep.yml runs on Mondays and edits one issue with the label upkeep.
Locally, just upkeep prints the same four reports:
Pins behind upstream (upstream-pins.py --upstream). Every upstream
this repository pins by hand, read from the file that uses it, against the
newest tag or the branch head. Dependabot covers Cargo, the actions, the
Docker bases and the Go module; this covers the rest.
MSCs against the proposals (msc-ledger.py --upstream). Each ledger
entry's pull request on matrix-spec-proposals: open, merged or closed,
with its labels. Two findings matter. A served MSC that has landed
upstream while the ledger records no stable version means the stable
spelling exists and clients will start sending it — see §5. A planned MSC
closed unmerged is one to stop planning for.
What a spec pin bump would bring (spec-drift.py against matrix-spec
main). The operations that differ between the pinned spec and today's
main, so the size of the next release's work is known before the bump.
The vendored state resolver against upstream (ruma-drift.py --upstream). vendor/ruma-state-res is a patched copy of a release. The
patch has no upstream path (ADR 0006). Dependabot cannot see this copy, so
no other job tells us about a ruma fix to state resolution.
The report compares the src/ of the newest crates.io release with the
vendored base, hunk by hunk. It marks the files that Spindle's patch also changes.
A released change is drift: the report says so and the script exits 1.
Rebase per §8. The report also compares ruma's main, as a warning only:
main moves daily, and an unreleased change can change again before
release.
The report does not start an action. A person decides; §4 tells how to bump a pin.
4. Bumping a pin
Every pin bump is its own pull request, and the pull request's job is to show what the bump changed. What to expect from each:
| Pin | What moves with it | What to read before merging |
|---|---|---|
SPEC_PIN in scripts/openapi-check.py | docs/spec-gaps.md is rewritten; openapi-check validates against new schemas | The diff of spec-gaps.md is the release's new work. File an issue per group that is in scope. A schema failure is a real divergence: fix it or allowlist it with a reason. |
RUST_SDK_REV | The nightly suite runs the new tests | Run compliance.yml by dispatch on the branch and read the tail: new candidates are promoted with the bump, regressions block it. |
COMPLEMENT_REV | Same, for Complement | Same; the protected subset runs on the pull request, the whole suite on main. |
ELEMENT_TAG, ELEMENT_CALL_REV | The browser and call suites | The E2E jobs on the pull request. A new spec Element Call requires appears as a failing spec, and the fix goes in before or with the bump. |
SYNAPSE_VERSION and the other benchmark versions | The next sitting's field | Nothing gates it; the sitting's sidecar records the versions, and the page keeps groups apart, so an older sitting is never compared against a newer field. |
Two pins are deliberately not automated: rust-toolchain.toml (the MSRV
moves on its own decision, #193) and ruma (ADR-0002: the room-version,
redaction and authorization rules are taken from ruma and a bump is read as
a protocol change, not a dependency update). The weekly report covers the
vendored ruma-state-res, and a person rebases it by hand, §8.
5. Absorbing a spec release
A Matrix spec release (v1.N) reaches this repository as a SPEC_PIN bump,
and the bump's spec-gaps.md diff lists what it added under ### Not served, added in v1.N. For each group:
- Decide scope. A homeserver serves what it serves; the page's job is to be honest about it. Third-party identity lookups and MSISDN tokens, for instance, are listed and not planned. File an issue for what is in scope, and say why the rest is not in the issue that tracks the release.
- Serve it. A route arrives with the test that would have failed
without it (CONTRIBUTING).
just regenmoves it fromspec-gaps.mdtodashboard.md; the OpenAPI check validates its responses on the next run. - Claim it.
crates/spindle-server/src/surface.rslists the spec versions/versionsadvertises, each with the routes that make the claim honest;tests/surface.rsfails if one is missing. Advertising v1.N is the last step, after everything the version requires is served — a client plans against/versions, and a longer list buys nothing but failures further from the cause. - Retire the unstable spelling of anything the release stabilised (§6).
6. Adding, stabilising and retiring an MSC
Spindle serves proposals ahead of the spec when a client it targets needs them: Element X's sliding sync, MatrixRTC's delayed and sticky events. Each one goes through the same life:
- Add. Serve it under the
unstable/org.matrix.mscNNNNprefix the proposal names, advertise the flag insurface::UNSTABLE_FEATURESonly if the endpoint answers, and add the entry tocontrib/msc/ledger.tomlwithstatus = "served"and the test that proves it.msc-ledger.py --checkrefuses a flag or a route the ledger does not own, and a served entry with no test naming the MSC. - Stabilise. When the weekly report says the proposal merged, the
stable spelling exists. Serve both — the stable path and the unstable
one, with the same handler, as
auth_metadatadoes — record the spec version in the entry'sstablefield, and keep the flag until the clients that check it have moved. A proposal that has merged but that no spec release names yet getsmerged = trueinstead: there is no stable spelling to adopt, and the report stops asking until there is. - Retire. The unstable route and flag go once no pinned client sends
them; the entry stays, with its
stableversion, so the page still says what was served and when.
A proposal that dies upstream is marked declined or superseded with a
note saying what replaced it, never deleted: the ledger is a record, not a
list of the present.
7. What is deliberately not automated
- Bumps. Every one above is a reviewed pull request. The suites are how a newer client tells this server what it now expects, and that is a thing to read, not to merge.
- Promotion. A passing test enters an allowlist by a commit naming the run, so the ratchet cannot learn to ignore a flake.
- Claims.
/versions, the dashboard, the MSC page and the spec-gaps page are generated from or checked against the router; none is typed. - Scope. The spec-gaps page lists everything unserved and gates on nothing but its own accuracy. Which of it to build is the roadmap's decision, made in an issue where the reasons are written down.
8. Rebase vendor/ruma-state-res
vendor/ruma-state-res is a release of ruma-state-res plus the hunks its
SPINDLE-PATCH.md lists, each with the reason it exists (ADR 0006 says why
the patch cannot go upstream). ruma-drift.py --check holds the tree to
that list on every pull request, so the patch cannot grow without a
reviewed edit to the table. When the upkeep report says a release changed
src/, the rebase is its own pull request:
-
Read the drift. The report's hunks are upstream's change to state resolution. Read each file that the report marks "Spindle patches this file" first. Upstream's change can make a hunk unnecessary. Drop that hunk; do not carry it.
-
Rebase in git, not by eye. Commit the old base. Commit the vendored tree on top of it, and the new release beside it. Then let
git rebasedo the three-way merge. From the repository root:repo=$PWD old=0.18.0 new=0.19.0 # the base, and the release the report nameswork=$(mktemp -d)for v in $old $new; docurl -sSfL -A spindle -o "$work/$v.crate" \"https://static.crates.io/crates/ruma-state-res/ruma-state-res-$v.crate"donesha256sum "$work"/*.crate # $old: SPINDLE-PATCH.md's; $new: crates.io's `checksum` for itgit init -q -b base "$work/tree" && cd "$work/tree"tar xzf "../$old.crate" --strip-components=1 && git add -A && git commit -qm "ruma-state-res $old"git switch -qc spindlegit rm -rq . && cp -a "$repo/vendor/ruma-state-res/." . && git add -A && git commit -qm "Spindle patch"git switch -q basegit rm -rq . && tar xzf "../$new.crate" --strip-components=1 && git add -A && git commit -qm "ruma-state-res $new"git switch -q spindle && git rebase base # a conflict is a hunk upstream touchedResolve each conflict by the intent of its table row, not by line. For a
Cargo.lockconflict, take upstream's file. The test build in step 5 makes it again. Then copy the result back:rm -rf "$repo/vendor/ruma-state-res" && mkdir "$repo/vendor/ruma-state-res" && git archive HEAD | tar x -C "$repo/vendor/ruma-state-res". -
Record the new base. In
SPINDLE-PATCH.md: the version, the crate checksum, and the upstream commit from the new crate's.cargo_vcs_info.json. The vendoredCargo.tomlcarries the version, which is where the script reads it from. -
Rewrite the table.
ruma-drift.py --hunksprints the rows for the new tree. It keeps the reason of each row that did not change. A row with an empty reason moved or is new: write its reason. Thenruma-drift.py --checkpasses and--upstreamreports no drift. -
Test. Run upstream's tests with the patch (
cargo test --manifest-path vendor/ruma-state-res/Cargo.toml). Then run Spindle's tests for state resolution, imported rejections and room recovery, the gates ADR 0006 names. Then runcargo update -p ruma-state-resto update the workspace lock. Put the drift section of the report in the pull request description. The reviewer then reads the upstream change next to the rebased patch.