Repository navigation
Multi-arch container publishing #3875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Hans Bakker (hansmbakker)
wants to merge
5
commits into
Azure:main
Choose a base branch
from
hansmbakker:fix/multiarch-container-images
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
d2caee1
Multi-arch container publishing
hansmbakker 2f29f28
Merge branch 'main' into fix/multiarch-container-images
aaronburtle 4ad39b6
remove UseAppHost=false flag after feedback
hansmbakker 8144b2a
update .dockerignore to ignore all .env files
hansmbakker 00e0c13
Add container application smoke test
hansmbakker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| .git | ||
| **/bin | ||
| **/obj | ||
| **/out | ||
| **/TestResults | ||
| **/.env | ||
| **/.env.* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| name: Container build | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - Dockerfile | ||
| - .dockerignore | ||
| - .github/workflows/container-build.yml | ||
| - scripts/publish-container.sh | ||
| - samples/getting-started/azure-postgresql/library.postgresql.sql | ||
| - src/** | ||
| - schemas/** | ||
| - Nuget.config | ||
| - global.json | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - Dockerfile | ||
| - .dockerignore | ||
| - .github/workflows/container-build.yml | ||
| - scripts/publish-container.sh | ||
| - samples/getting-started/azure-postgresql/library.postgresql.sql | ||
| - src/** | ||
| - schemas/** | ||
| - Nuget.config | ||
| - global.json | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build: | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| target: [runtime, runtime-nonroot] | ||
| runner: [ubuntu-24.04, ubuntu-24.04-arm] | ||
| include: | ||
| - runner: ubuntu-24.04 | ||
| platform: linux/amd64 | ||
| arch: x64 | ||
| - runner: ubuntu-24.04-arm | ||
| platform: linux/arm64 | ||
| arch: arm64 | ||
| runs-on: ${{ matrix.runner }} | ||
| services: | ||
| postgres: | ||
| image: postgres:16 | ||
| env: | ||
| POSTGRES_USER: dab | ||
| POSTGRES_PASSWORD: dab | ||
| POSTGRES_DB: dab | ||
| ports: | ||
| - 5432:5432 | ||
| options: >- | ||
| --health-cmd "pg_isready -U dab -d dab" | ||
| --health-interval 5s | ||
| --health-timeout 5s | ||
| --health-retries 10 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: docker/setup-buildx-action@v3 | ||
| - name: Check publishing script syntax | ||
| run: bash -n scripts/publish-container.sh | ||
| - name: Build image | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| target: ${{ matrix.target }} | ||
| platforms: ${{ matrix.platform }} | ||
| load: true | ||
| tags: dab:container-test | ||
| - name: Check runtime, application architecture, and native launcher | ||
| env: | ||
| ARCH: ${{ matrix.arch }} | ||
| TARGET: ${{ matrix.target }} | ||
| run: | | ||
| docker run --rm --entrypoint sh dab:container-test -ec ' | ||
| dotnet --info | ||
| test -s /App/dab-config.json | ||
| grep -F "\"name\": \".NETCoreApp,Version=v10.0/linux-$1\"" \ | ||
| /App/Azure.DataApiBuilder.Service.deps.json | ||
| test -x /App/Azure.DataApiBuilder.Service | ||
| if launcher_output=$(ASPNETCORE_URLS=invalid /App/Azure.DataApiBuilder.Service 2>&1); then | ||
| echo "Expected the launcher to reject an invalid ASPNETCORE_URLS" >&2 | ||
| exit 1 | ||
| fi | ||
| if ! printf "%s\n" "$launcher_output" | grep -F "Invalid ASPNETCORE_URLS format"; then | ||
| printf "%s\n" "$launcher_output" >&2 | ||
| exit 1 | ||
| fi | ||
| if [ "$2" = runtime-nonroot ]; then | ||
| test "$(id -u)" = "$APP_UID" | ||
| touch /App/logs/container-test | ||
| else | ||
| test "$(id -u)" = 0 | ||
| fi | ||
| ' sh "$ARCH" "$TARGET" | ||
| - name: Check application startup and REST read | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| postgres_container="${{ job.services.postgres.id }}" | ||
| docker cp samples/getting-started/azure-postgresql/library.postgresql.sql \ | ||
| "$postgres_container:/tmp/library.sql" | ||
| docker exec "$postgres_container" psql -U dab -d dab -v ON_ERROR_STOP=1 -f /tmp/library.sql | ||
|
|
||
| config_file="$RUNNER_TEMP/dab-smoke-config.json" | ||
| cat > "$config_file" <<'EOF' | ||
| { | ||
| "data-source": { | ||
| "database-type": "postgresql", | ||
| "connection-string": "@env('DAB_CONNECTION_STRING')", | ||
| "options": {} | ||
| }, | ||
| "runtime": { | ||
| "rest": { "enabled": true, "path": "/api" }, | ||
| "graphql": { "enabled": false }, | ||
| "mcp": { "enabled": false }, | ||
| "host": { | ||
| "authentication": { "provider": "StaticWebApps" }, | ||
| "mode": "development" | ||
| } | ||
| }, | ||
| "entities": { | ||
| "Book": { | ||
| "source": { "object": "books", "type": "table" }, | ||
| "rest": { "enabled": true }, | ||
| "permissions": [ | ||
| { "role": "anonymous", "actions": [{ "action": "read" }] } | ||
| ] | ||
| } | ||
| } | ||
| } | ||
| EOF | ||
|
|
||
| docker run -d --name dab-smoke --network host \ | ||
| -e 'DAB_CONNECTION_STRING=Host=127.0.0.1;Port=5432;Database=dab;Username=dab;Password=dab;SSL Mode=Disable' \ | ||
| -v "$config_file:/App/dab-config.json:ro" \ | ||
| dab:container-test | ||
| trap 'docker rm -f dab-smoke >/dev/null 2>&1 || true' EXIT | ||
|
|
||
| for attempt in $(seq 1 30); do | ||
| if curl --fail --silent http://127.0.0.1:5000/health > "$RUNNER_TEMP/dab-health.json"; then | ||
| break | ||
| fi | ||
| if [ "$attempt" -eq 30 ]; then | ||
| docker logs dab-smoke | ||
| echo "DAB did not become healthy" >&2 | ||
| exit 1 | ||
| fi | ||
| sleep 2 | ||
| done | ||
| grep -F '"status": "Healthy"' "$RUNNER_TEMP/dab-health.json" | ||
|
|
||
| curl --fail --silent http://127.0.0.1:5000/api/Book \ | ||
| | grep -F 'Prelude to Foundation' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| # Building and publishing container images | ||
|
|
||
| The Dockerfile supports `linux/amd64` and `linux/arm64`. The SDK runs on the | ||
| builder's native architecture and cross-compiles for the requested platform; | ||
| the ASP.NET runtime base image is selected for that platform. Both the default | ||
| `runtime` target and the opt-in `runtime-nonroot` target support these platforms. | ||
| Both images include the native launcher at `/App/Azure.DataApiBuilder.Service`; | ||
| the default entrypoint remains `dotnet Azure.DataApiBuilder.Service.dll`. | ||
|
|
||
| ## Local build | ||
|
|
||
| Build and load a single platform into the local Docker image store: | ||
|
|
||
| ```sh | ||
| docker buildx build --platform linux/arm64 --target runtime --load -t dab:local . | ||
| ``` | ||
|
|
||
| Omit `--platform` to build for the local builder's default platform. Running an | ||
| image for another architecture requires emulation. Cross-building | ||
| `runtime-nonroot` also requires emulation (or native builder nodes for both | ||
| architectures) because its permissions setup executes in the target image. | ||
|
|
||
| ## Release publishing | ||
|
|
||
| Use a Buildx builder with multi-platform support, for example: | ||
|
|
||
| ```sh | ||
| docker buildx create --name dab-multiarch --driver docker-container --use | ||
| docker buildx inspect --bootstrap | ||
| ``` | ||
|
|
||
| Authenticate to the release pipeline's **writable registry**, then publish all | ||
| tags for a variant in one build. Replace the example registry and version with | ||
| the release pipeline's existing destination and version: | ||
|
|
||
| ```sh | ||
| docker login <registry> | ||
| bash scripts/publish-container.sh runtime \ | ||
| <registry>/data-api-builder:<version> \ | ||
| <registry>/data-api-builder:latest | ||
| bash scripts/publish-container.sh runtime-nonroot \ | ||
| <registry>/data-api-builder:<version>-nonroot \ | ||
| <registry>/data-api-builder:latest-nonroot | ||
| ``` | ||
|
|
||
| The script pushes both platform images and their shared image index for each | ||
| supplied tag. Do not push the architectures separately to the same tag: the | ||
| last push would replace the earlier architecture instead of publishing an | ||
| index. A multi-platform build uses `--push`, not `--load`. | ||
|
|
||
| The official MCR release pipeline is not defined in this repository. Its | ||
| maintainers must replace the existing single-platform build/push with this | ||
| script (or the equivalent `docker buildx build --platform | ||
| linux/amd64,linux/arm64 --push` command), preserving their existing registry | ||
| authentication, tag selection, and MCR promotion process. Every promoted tag | ||
| must retain the multi-platform index and both referenced platform images. | ||
| Changing the Dockerfile alone does not make existing MCR tags multi-platform. | ||
|
|
||
| Verify the resulting tag in the writable registry and again after MCR promotion: | ||
|
|
||
| ```sh | ||
| docker buildx imagetools inspect <registry>/data-api-builder:<version> | ||
| docker buildx imagetools inspect mcr.microsoft.com/azure-databases/data-api-builder:<version> | ||
| ``` | ||
|
|
||
| Both must list `linux/amd64` and `linux/arm64`. Provenance attestations may also | ||
| appear as `unknown/unknown`; these do not replace the required platform images. | ||
| The `Container build` workflow builds both variants on native x64 and ARM64 | ||
| runners and checks the application's runtime identifier, native launcher, and | ||
| non-root log access. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| #!/usr/bin/env bash | ||
| # Copyright (c) Microsoft Corporation. | ||
| # Licensed under the MIT License. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| if [[ $# -lt 2 || ( "$1" != runtime && "$1" != runtime-nonroot ) ]]; then | ||
| echo "Usage: $0 <runtime|runtime-nonroot> <image:tag> [<image:tag> ...]" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| target=$1 | ||
| shift | ||
| tags=() | ||
| for tag in "$@"; do | ||
| tags+=(--tag "$tag") | ||
| done | ||
|
|
||
| repo_root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) | ||
| docker buildx build \ | ||
| --file "$repo_root/Dockerfile" \ | ||
| --platform linux/amd64,linux/arm64 \ | ||
| --target "$target" \ | ||
| "${tags[@]}" \ | ||
| --push \ | ||
| "$repo_root" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.