Skip to content

Add ARM64 support to cross-version tests and release pipeline #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dhalperi merged 6 commits into from
Mar 18, 2026
+147 −51
Merged

Add ARM64 support to cross-version tests and release pipeline #160

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
775ce64
Initial plan
Copilot Mar 18, 2026
d738473
Add ARM (linux/arm64) multi-platform Docker image support
Copilot Mar 18, 2026
795735d
Add ARM build verification steps to precommit image_build job
Copilot Mar 18, 2026
103fa24
Use native ubuntu-24.04-arm runners for ARM64 image build and testing
Copilot Mar 18, 2026
2ddeb1c
Add pybf_cross_version_tests_arm64 job on native ubuntu-24.04-arm runner
Copilot Mar 18, 2026
779ff6b
Use matrix strategy to collapse duplicate amd64/arm64 jobs
Copilot Mar 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 25 additions & 6 deletions .github/workflows/reusable-integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,19 +108,24 @@ jobs:
pytest -k "not test_notebook_output" "${TEMP_DIR}/tests/integration"
popd
pybf_cross_version_tests:
runs-on: ubuntu-latest
runs-on: ${{ matrix.platform.runner }}
if: inputs.run_cross_version_tests
needs:
- get_pybf_versions
strategy:
matrix:
platform:
- runner: ubuntu-latest
bf_artifact: bf_image
- runner: ubuntu-24.04-arm
bf_artifact: bf_image_arm64
pybf_version: ${{ fromJson(needs.get_pybf_versions.outputs.versions) }}
steps:
- uses: actions/checkout@v6
- name: Download bf image
uses: actions/download-artifact@v8
with:
name: bf_image
name: ${{ matrix.platform.bf_artifact }}
- name: Download pybf tests
uses: actions/download-artifact@v8
with:
Expand Down Expand Up @@ -155,13 +160,20 @@ jobs:
- name: Stop bf
run: docker stop $(docker ps -q)
bf_image_test:
runs-on: ubuntu-latest
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- runner: ubuntu-latest
bf_artifact: bf_image
- runner: ubuntu-24.04-arm
bf_artifact: bf_image_arm64
steps:
- uses: actions/checkout@v6
- name: Download bf image
uses: actions/download-artifact@v8
with:
name: bf_image
name: ${{ matrix.bf_artifact }}
- name: Download pybf whl
uses: actions/download-artifact@v8
with:
Expand Down Expand Up @@ -200,13 +212,20 @@ jobs:
- name: Stop bf
run: docker stop $(docker ps -q)
allinone_image_test:
runs-on: ubuntu-latest
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- runner: ubuntu-latest
image_artifact: allinone_image
- runner: ubuntu-24.04-arm
image_artifact: allinone_image_arm64
steps:
- uses: actions/checkout@v6
- name: Download allinone image
uses: actions/download-artifact@v8
with:
name: allinone_image
name: ${{ matrix.image_artifact }}
- name: Download pybf whl
uses: actions/download-artifact@v8
with:
Expand Down
19 changes: 14 additions & 5 deletions .github/workflows/reusable-precommit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,11 +180,20 @@ jobs:
name: pybf_notebooks
path: pybatfish-notebooks.tgz
image_build:
runs-on: ubuntu-latest
runs-on: ${{ matrix.runner }}
needs:
- bf_build
- pybf_build
- bf_version
strategy:
matrix:
include:
- runner: ubuntu-latest
bf_artifact: bf_image
allinone_artifact: allinone_image
- runner: ubuntu-24.04-arm
bf_artifact: bf_image_arm64
allinone_artifact: allinone_image_arm64
steps:
- name: Checkout
uses: actions/checkout@v6
Expand Down Expand Up @@ -216,10 +225,10 @@ jobs:
org.batfish.batfish-version=${{ needs.bf_version.outputs.bf_version }}
outputs: type=docker,dest=/tmp/bf.tar
context: .
- name: Upload docker image
- name: Upload Batfish image
uses: actions/upload-artifact@v7
with:
name: bf_image
name: ${{ matrix.bf_artifact }}
path: /tmp/bf.tar
- name: Download pybf whl
uses: actions/download-artifact@v8
Expand Down Expand Up @@ -251,8 +260,8 @@ jobs:
org.batfish.batfish-version=${{ needs.bf_version.outputs.bf_version }}
outputs: type=docker,dest=/tmp/allinone.tar
context: .
- name: Upload docker image
- name: Upload allinone image
uses: actions/upload-artifact@v7
with:
name: allinone_image
name: ${{ matrix.allinone_artifact }}
path: /tmp/allinone.tar
142 changes: 104 additions & 38 deletions .github/workflows/reusable-upload.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,27 +56,64 @@ jobs:
if: inputs.BATFISH_GITHUB_BATFISH_REPO == 'batfish/batfish' && inputs.BATFISH_GITHUB_PYBATFISH_REPO == 'batfish/pybatfish'
runs-on: ubuntu-latest
steps:
- name: Download bf image
- name: Checkout
uses: actions/checkout@v6
- name: Download jar
uses: actions/download-artifact@v8
with:
name: bf_jar
- name: Download questions
uses: actions/download-artifact@v8
with:
name: bf_image
- name: Download allinone image
name: bf_questions
- name: Download pybf whl
uses: actions/download-artifact@v8
with:
name: pybf_whl
- name: Download pybf notebooks
uses: actions/download-artifact@v8
with:
name: allinone_image
name: pybf_notebooks
- name: Setup Batfish image assets
run: |
mkdir assets
tar xzf questions.tgz -C assets
cp allinone-bundle.jar assets
cp log4j2.yaml assets
- name: Setup Allinone image assets
run: |
cp pybatfish-*.whl assets
TEMP_DIR=$(mktemp -d)
tar xzf pybatfish-notebooks.tgz -C ${TEMP_DIR}
cp -r ${TEMP_DIR}/jupyter_notebooks/ assets/notebooks/
cp wrapper.sh assets
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: DockerHub Login
uses: docker/login-action@v4
with:
username: ${{ env.BATFISH_DOCKER_LOGIN_USERNAME }}
password: ${{ secrets.BATFISH_DOCKER_LOGIN_TOKEN }}
- name: Upload Batfish image
run: |
docker load < bf.tar
docker push batfish/batfish:${{ inputs.test_tag }}
- name: Upload Allinone image
run: |
docker load < allinone.tar
docker push batfish/allinone:${{ inputs.test_tag }}
- name: Build and push Batfish image (multi-platform)
uses: docker/build-push-action@v7
with:
push: true
file: batfish.dockerfile
platforms: linux/amd64,linux/arm64
build-args: ASSETS=./assets
tags: batfish/batfish:${{ inputs.test_tag }}
context: .
- name: Build and push Allinone image (multi-platform)
uses: docker/build-push-action@v7
with:
push: true
file: allinone.dockerfile
platforms: linux/amd64,linux/arm64
build-args: ASSETS=./assets
tags: batfish/allinone:${{ inputs.test_tag }}
context: .
dev_whl_upload:
if: inputs.BATFISH_GITHUB_BATFISH_REPO == 'batfish/batfish' && inputs.BATFISH_GITHUB_PYBATFISH_REPO == 'batfish/pybatfish'
runs-on: ubuntu-latest
Expand Down Expand Up @@ -118,41 +155,70 @@ jobs:
environment: Release Containers
runs-on: ubuntu-latest
steps:
- name: Download bf image
- name: Checkout
uses: actions/checkout@v6
- name: Download jar
uses: actions/download-artifact@v8
with:
name: bf_jar
- name: Download questions
uses: actions/download-artifact@v8
with:
name: bf_image
- name: Download allinone image
name: bf_questions
- name: Download pybf whl
uses: actions/download-artifact@v8
with:
name: pybf_whl
- name: Download pybf notebooks
uses: actions/download-artifact@v8
with:
name: allinone_image
name: pybf_notebooks
- name: Setup Batfish image assets
run: |
mkdir assets
tar xzf questions.tgz -C assets
cp allinone-bundle.jar assets
cp log4j2.yaml assets
- name: Setup Allinone image assets
run: |
cp pybatfish-*.whl assets
TEMP_DIR=$(mktemp -d)
tar xzf pybatfish-notebooks.tgz -C ${TEMP_DIR}
cp -r ${TEMP_DIR}/jupyter_notebooks/ assets/notebooks/
cp wrapper.sh assets
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: DockerHub Login
uses: docker/login-action@v4
with:
username: ${{ env.BATFISH_DOCKER_LOGIN_USERNAME }}
password: ${{ secrets.BATFISH_DOCKER_LOGIN_TOKEN }}
- name: Upload Batfish image
run: |
TEST_TAG="${{ inputs.test_tag }}"
RELEASE_TAG="${{ inputs.release_tag }}"
docker load < bf.tar
# Re-tag with release tag and latest tag
docker tag "batfish/batfish:${TEST_TAG}" "batfish/batfish:${RELEASE_TAG}"
docker tag "batfish/batfish:${TEST_TAG}" "batfish/batfish:latest"
echo placeholder for pushing "batfish/batfish:${RELEASE_TAG}" and "batfish/batfish:latest"
docker push "batfish/batfish:${RELEASE_TAG}"
docker push "batfish/batfish:latest"
- name: Upload Allinone image
run: |
TEST_TAG="${{ inputs.test_tag }}"
RELEASE_TAG="${{ inputs.release_tag }}"
docker load < allinone.tar
# Re-tag with release tag and latest tag
docker tag "batfish/allinone:${TEST_TAG}" "batfish/allinone:${RELEASE_TAG}"
docker tag "batfish/allinone:${TEST_TAG}" "batfish/allinone:latest"
echo placeholder for pushing "batfish/allinone:${RELEASE_TAG}" and "batfish/allinone:latest"
docker push "batfish/allinone:${RELEASE_TAG}"
docker push "batfish/allinone:latest"
- name: Build and push Batfish image (multi-platform)
uses: docker/build-push-action@v7
with:
push: true
file: batfish.dockerfile
platforms: linux/amd64,linux/arm64
build-args: ASSETS=./assets
tags: |
batfish/batfish:${{ inputs.test_tag }}
batfish/batfish:${{ inputs.release_tag }}
batfish/batfish:latest
context: .
- name: Build and push Allinone image (multi-platform)
uses: docker/build-push-action@v7
with:
push: true
file: allinone.dockerfile
platforms: linux/amd64,linux/arm64
build-args: ASSETS=./assets
tags: |
batfish/allinone:${{ inputs.test_tag }}
batfish/allinone:${{ inputs.release_tag }}
batfish/allinone:latest
context: .
prod_whl_upload:
if: inputs.queue_prod_release && inputs.BATFISH_GITHUB_BATFISH_REPO == 'batfish/batfish' && inputs.BATFISH_GITHUB_PYBATFISH_REPO == 'batfish/pybatfish'
environment: Release Pybatfish
Expand Down
3 changes: 2 additions & 1 deletion allinone.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ ARG ASSETS
RUN mkdir -p /data
RUN chmod a+rw /data
COPY ${ASSETS} ./
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64/
ARG TARGETARCH
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-${TARGETARCH}
ENV JAVA_LIBRARY_PATH=/usr/lib
ENV HOME=/data

Expand Down
3 changes: 2 additions & 1 deletion batfish.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ ARG ASSETS
RUN mkdir -p /data
RUN chmod a+rw /data
COPY ${ASSETS} ./
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64/
ARG TARGETARCH
ENV JAVA_HOME=/usr/lib/jvm/java-17-openjdk-${TARGETARCH}
ENV JAVA_LIBRARY_PATH=/usr/lib
ENV HOME=/data

Expand Down