Repository navigation
Multi-arch runner image (amd64 + arm64), built natively - #33
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each architecture builds and smoke-tests on its own runner (ubuntu-latest, ubuntu-24.04-arm). On main each pushes by digest, and a publish job tags the manifest list of both as :latest and :<date>-<sha>, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sodre
added a commit
that referenced
this pull request
Nov 14, 2026
…ration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sodre
added a commit
that referenced
this pull request
Nov 14, 2026
…epair (#18) (#32) ## Summary Spec #18. Every job names its full **shape**, and every **capacity** serves every shape. Two new Fargate capacities, `ci-fargate-spot` and `ci-fargate`, have no Docker daemon (ADR-0001). Caps become **vCPU budgets**, and **swaps** are repaired instead of prevented (ADR-0003). Terms are in `GLOSSARY.md`. ```diff label contract - self-hosted + one of ci-spot|ci-ondemand [+ linux, x64] + self-hosted + one capacity (ci-spot | ci-ondemand | ci-fargate-spot | ci-fargate) + + one size (ci-cpu-2 | ci-cpu-4 | ci-cpu-8 | ci-cpu-16) no default + + one arch (x64 | arm64) no default + [+ linux] ``` ```text webhook queued → contract → queue {label, vcpus, arch} in_progress → runner launched for another job? → re-queue that job, re-tag the runner (swap repair) launcher budget: sum the capacity's live vCPUs (instances + tasks not yet STOPPED) → defer if over (soft) ci-spot / ci-ondemand → EC2 Fleet: x64 m8i/m8a/m7i/m6i/m6a/m7a, arm64 m7g/m6g/m8g, .large–.4xlarge ci-fargate[-spot] → RunTask: ECR image with SOCI index, no task role, 50 GB, idle-exit entrypoint sweeper → instances and tasks over 2 h; registrations with no instance or task notifier → EC2 Spot warning, or ECS SpotInterruption → failed check run on the right commit ``` Infrastructure: - deploy through an S3 bucket (1 MB template limit); - an ECR repo with a push role trusted only for `main` (the repo's immutable OIDC subject), reusing the account's existing OIDC provider (ADR-0002); - an ECS cluster and two task definitions; - an arm64 launch template; - the launcher's role carries `ZAEPowerUserBoundary`, so it can pass the task execution role. The image work landed on `main` already (#33–#36): the multi-arch build, the SOCI/ECR push and the idle-exit wrapper. ## Evidence Each was proven live on the deployed integration branch. The evidence is on each ticket. | | Before | After | |---|---|---| | Sizes (#22) | every runner `.xlarge` | `ci-cpu-8` → `m6a.2xlarge`, `nproc` 8 | | arm64 (#24) | none | EC2 `m8g.xlarge` and Fargate both print `aarch64` | | Budgets (#25) | 6/2 runners | 24/8 vCPUs; `"deferred": "4+4 of 4 on-demand vCPUs"`, then launched | | Swaps (#21, #30) | a deferred job could be stranded forever | repaired live on EC2 (across repos) and Fargate (4 repairs, every job completed) | | Fargate (#29) | none | jobs on both capacities; no AWS credentials in a task; idle exit at 120 s and 900 s | | Start time (#28) | EC2 ~100 s queued→started | Fargate Spot 27–30 s | | Reclaims on Fargate (#30) | none | a synthetic `SpotInterruption` produced `ci-runner / j (4)` with the real job name; a repeat logs `already reported` | `make test`: 144 pass, including the aws-emulator ECS tests. The template is 74,337 of 1,048,576 bytes. ## Release note: breaking label change From this deploy on, **a job must carry a size label and an architecture label**. `runs-on: [self-hosted, ci-spot]` is now outside the contract, and the job waits for a runner that never comes. Use, for example, `[self-hosted, ci-spot, ci-cpu-4, x64]`, which is the old shape. Every repo in the `ci-runner` runner group was updated before the change went live (ci-runner-canary#2, zae-zcm#37, foresight#1194). `SpotCap` and `OnDemandCap` are replaced by `SpotVcpuCap` (24) and `OnDemandVcpuCap` (8), plus `FargateSpotVcpuCap` (48) and `FargateVcpuCap` (8). An override on the old names is dropped. ## Merge Danger **Door:** two-way for the code, one-way for the label contract in practice. The integration branch is already what the live stack runs, so this merge changes nothing live. Rolling back means redeploying `main`'s template, which brings back the defaults and drops the Fargate capacities and the new parameters. **Blast Radius:** every consumer's CI. A job missing a size or architecture label now hangs; the runbook's traps table covers it. Budgets are soft (the owner's choice): a burst can exceed them, and this was seen live at 36–44 of 24 Spot vCPUs. The launcher's role depends on the account's `ZAEPowerUserBoundary`, which lives outside this stack. Not proven live: a real Spot reclaim on Fargate, and a sweeper stop of a task older than 2 hours. #31, the human canary, stays open for those. Closes #18 Closes #19, closes #20, closes #21, closes #22, closes #23, closes #24, closes #25, closes #26, closes #27, closes #28, closes #29, closes #30 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Landed early from the integration branch (#32), because only
mainpublishes. After this merges,ghcr.io/zeroae/ci-runner:latestis a multi-arch manifest list.image/smoke.shnow checks thatuname -mand every binary the image adds (docker, buildx, compose, gh, node, aws) are built for the expected architecture.Evidence
:latestis a single linux/amd64 image.EXPECT_ARCH=x86_64on an arm64 machine fails every architecture check, andaarch64passes.main. I'll watch the first run after merge and check that:latestlists amd64 and arm64.Merge Danger
Door: two-way. Reverting brings back the single-arch build; the next weekly or
image/**build republishes:latest.Blast Radius: runners.
Every EC2 CI runner pulls
:latestat boot. An amd64 host pulls the amd64 entry of the manifest list, so it gets the same image as before. If the publish job fails,:latestsimply isn't updated.Part of #18, closes #23 on the integration side.
🤖 Generated with Claude Code