(motir-core) Cut the THREE container supervisors over IN PRODUCTION — ADD the ids to `MOTIR_POSTGRES_JOB_IDS`, watch a real push coalesce, boot and index
Move the three container supervisors onto the Postgres engine in production, and read the result back from the ledger and the platform — the operator half of MOTIR-3417, which its merge cannot perform.
Why this is a task under the EPIC rather than a child of the story
Its first step needs the story's merge to be deployed, and a post-merge card parented inside the container it verifies either deadlocks the parent run or is cascaded done unverified. MOTIR-3463 and MOTIR-3475 sit here for exactly this reason and this card is their third sibling — the one the epic's partition was missing. 14 cron + 20 event + 3 supervisors is the whole fleet; two of those three families had an operator card and the supervisors did not.
The steps
-
Ship the DECLARATION first. ⚠️ Added 2026-08-27 — this card acquired a code half it did not have when it was written. MOTIR-3716 moved the lane census into shipped code, and
lib/jobs/engine/census.tslists all three supervisors underDELIBERATELY_ON_INNGEST. Adding the ids to the secret and nothing else produces the drift that file exists to catch, in theroutedNotDeclareddirection:dailyHealthCheck.tsthrowsJobLaneDriftErrorand dead-letters daily. So the three ids move toMIGRATED_TO_ENGINEin a pull request, and that pull request deploys before step 2 —census.tsstates the order itself, deploy-then-route. -
Confirm the deploy that carries the story is LIVE — not that the pull requests merged. Read the running release, not
main. -
ADD the three ids to
MOTIR_POSTGRES_JOB_IDS, preserving everything already in it:system.code-graph-indexsystem.code-graph-refreshsystem.ci-runner-boot
⚠️ ADD, never SET. By the time this runs the variable already carries the 20 event ids and the 14 scheduled ones. Read the current value first and append; a
fly secrets setwith a fresh list silently moves 34 jobs back to Inngest, and nothing would report it — every one of them would simply keep working, on the wrong engine, until somebody looked. This is why this card is sequenced after MOTIR-3475: both write the same single variable, and the two must not race. -
Make it LIVE, and verify that it is.
fly secrets settriggers a rolling machine update;--stagedefers it to the next deploy. Both theappand theworkerprocess groups must be running with the new value —appreads the switch when it emits,workerwhen it decides whether to run. Read it back from the platform (fly secrets list -a motir-core, and the machines' state), never from a local file. -
Watch a real push. Push to a connected repo's default branch — twice within the debounce window, which is what makes the coalescing observable — and confirm ONE
system.code-graph-refreshrun appears, boots its containers, and settlessucceededwith itsoutput.repoRef. -
Watch a real CI job.⚠️ DOWNGRADED 2026-08-27 to routed and switch-verified, never exercised.system.ci-runner-boothas 0job_runrows all time — it has never executed, so there is no window in which watching one becomes possible;system.code-graph-indexhas 15 all time and none since 2026-08-19. This is the same disposition MOTIR-3463 made for its five never-firing ids, and the lesson is already recorded (motir-ai#309). Prove the ids are ROUTED and that the switch reads them, record the counts, and do not wait for a trigger that has never fired. -
Check the fleet's spend did not move. The admission cap is untouched by this story, so the number of concurrent containers should look exactly as it did the week before. A coalescing bug shows up here first.
Rollback is removing the three ids and letting the machines roll. The switch defaults to Inngest for any id it does not name, and both lanes are still deployed — that is what makes this reversible and why it is worth doing as its own act rather than inside a deploy.
Scope boundary
ENDS at: the three ids live in production, with a real push proven from the ledger and the two dormant supervisors proven ROUTED.
Changes only the DECLARATION. ⚠️ Amended 2026-08-27: this card originally said "Changes NO code", and that stopped being true when MOTIR-3716 made the lane census shipped code. The one permitted edit is moving the three ids between the two lists in lib/jobs/engine/census.ts and the tests and docs that read them. Anything else that is wrong still STOPS this card and is filed as a bug; it does not patch.
Does NOT remove anything from Inngest. The SDK, the serve route and the dependency are MOTIR-3418's, and it is blocked_by this card precisely so the retirement cannot begin while a third of the fleet is still on the old lane.
Does NOT touch the fleet's spend controls — MOTIR_INDEX_MAX_IN_FLIGHT, codeGraphIndexAdmissionService, lib/ciFleet/limits.ts. Step 6 OBSERVES them; it does not adjust them.
Acceptance criteria
- The three ids are named in
MIGRATED_TO_ENGINEinlib/jobs/engine/census.tsonmain,DELIBERATELY_ON_INNGESTis EMPTY, and that commit is in the RUNNING release — checked in the image, not inmain. fly secrets list -a motir-coreshowsMOTIR_POSTGRES_JOB_IDSupdated, and the value read back from inside a machine contains every id it carried before plus these three, by name — acommagainst the recorded baseline, with nothing dropped. ⚠️ No literal count. The card originally said "all 37 job ids — the 20, the 14 and these 3"; the baseline was 38 on 2026-08-27 (MOTIR-3682, MOTIR-3688 and MOTIR-3709 landed after this card was written), making the target 41, and any figure written here will be stale again the next time a job lands. The set is the criterion; the number is not.reconcileLanes()reportsin_sync— the declaration and the live secret agree — read from asystem.daily-health-checkjob_runrow rather than computed by hand.- Both the
appandworkerprocess groups are running a release that carries the new value, read from the platform's own API rather than fromfly.tomlor a dashboard screenshot. - A real default-branch push produces exactly ONE
system.code-graph-refreshjob_run,succeeded, onlane = engine, carrying oneoutput.repoRef, and TWO pushes inside the debounce window still produce one. system.ci-runner-bootandsystem.code-graph-indexare proven ROUTED — present in the live secret and reported so by the switch — with their execution counts recorded on this card as measured, NOT observed running. Neither can be exercised by waiting.- Concurrent index containers over the following day are within the range they occupied before the flip, checked rather than assumed.
- The Inngest dashboard shows no new runs for these three function ids after the flip — the half that proves nothing is running on BOTH engines.
- Anything unexpected is a filed bug with its evidence, and the ids are rolled back rather than left half-flipped.
Context refs
lib/jobs/engine/cutover.ts—MOTIR_POSTGRES_JOB_IDS, and the default-to-Inngest safety property that makes rollback triviallib/jobs/engine/census.ts— the DECLARATION half (step 0) andreconcileLanes();docs/jobs.mdcarries the two-edits-plus-a-read-back procedurefly.toml—app = "motir-core", and theapp/workerprocess groups this value must reach/settings/workspace/jobs— the ledger surface these runs appear onlib/services/codeGraphIndexAdmissionService.ts·lib/ciFleet/limits.ts— the spend controls step 6 observes and must not adjust- MOTIR-3463 · MOTIR-3475 — the two sibling flips that write the same variable, and the pattern this follows
- MOTIR-3425 — the worker machines these runs execute on, already provisioned