Skip to content

Commit 2a38532

Browse files
crepererumcrepererum
authored
Merge pull request #225 from influxdata/crepererum/issue186
ci: workflow to pre-build WASM binaries
2 parents 38e195b + 7aa299e commit 2a38532

File tree

5 files changed

+159
-28
lines changed

5 files changed

+159
-28
lines changed

‎.github/actions/free-disk-space/action.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
name: Free Disk Space
3+
description: Delete a bunch of unnecessary data from the pre-built GitHub runner images.
4+
runs:
5+
using: "composite"
6+
steps:
7+
- name: Free Disk Space
8+
run: ${{ github.action_path }}/free.sh
9+
shell: bash

‎.github/actions/free-disk-space/free.sh‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
#!/usr/bin/env bash
2+
3+
set -euo pipefail
4+
5+
df -h
6+
7+
to_delete=(
8+
"/lib/firefox"
9+
"/lib/google-cloud-sdk"
10+
"/lib/jvm"
11+
"/lib/llvm-16"
12+
"/lib/llvm-17"
13+
"/opt/az"
14+
"/opt/google"
15+
"/opt/hostedtoolcache/CodeQL"
16+
"/opt/hostedtoolcache/PyPy"
17+
"/opt/hostedtoolcache/Ruby"
18+
"/opt/hostedtoolcache/go"
19+
"/opt/hostedtoolcache/node"
20+
"/opt/microsoft"
21+
"/opt/pipx"
22+
"/usr/share/swift"
23+
)
24+
for d in "${to_delete[@]}"; do
25+
echo "delete $d"
26+
sudo rm -rf "$d"
27+
done
28+
29+
df -h

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -15,34 +15,6 @@ jobs:
1515
check:
1616
runs-on: ubuntu-latest
1717
steps:
18-
- name: Free disk space
19-
run: |
20-
df -h
21-
22-
to_delete=(
23-
"/lib/firefox"
24-
"/lib/google-cloud-sdk"
25-
"/lib/jvm"
26-
"/lib/llvm-16"
27-
"/lib/llvm-17"
28-
"/opt/az"
29-
"/opt/google"
30-
"/opt/hostedtoolcache/CodeQL"
31-
"/opt/hostedtoolcache/PyPy"
32-
"/opt/hostedtoolcache/Ruby"
33-
"/opt/hostedtoolcache/go"
34-
"/opt/hostedtoolcache/node"
35-
"/opt/microsoft"
36-
"/opt/pipx"
37-
"/usr/share/swift"
38-
)
39-
for d in "${to_delete[@]}"; do
40-
echo "delete $d"
41-
sudo rm -rf "$d"
42-
done
43-
44-
df -h
45-
4618
- name: Checkout
4719
uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
4820
with:
@@ -51,6 +23,9 @@ jobs:
5123
- name: Fetch main branch
5224
run: git fetch --depth=1 origin main
5325

26+
- name: Free disk space
27+
uses: ./.github/actions/free-disk-space
28+
5429
- name: Install `yamllint`
5530
run: pip install --requirement=.github/workflows/requirements.txt
5631

‎.github/workflows/prebuild.yml‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
name: Prebuild
3+
4+
# yamllint disable-line rule:truthy
5+
on:
6+
workflow_dispatch:
7+
8+
permissions:
9+
attestations: write
10+
contents: write
11+
id-token: write
12+
13+
env:
14+
# Disable incremental compilation to avoid overhead.
15+
CARGO_INCREMENTAL: "0"
16+
17+
jobs:
18+
release:
19+
runs-on: ubuntu-latest
20+
steps:
21+
# do NOT load any caches here, we want a clean, freestanding build w/o stateful dependencies!
22+
23+
- name: Set Timestamp
24+
id: timestamp
25+
run: |
26+
ts_iso="$(date -u -Is)"
27+
ts_tag="$(echo "$ts_iso" | sed 's/+00:00//g' | sed 's/:/-/g')"
28+
29+
echo "iso=$ts_iso"
30+
echo "tag=$ts_tag"
31+
32+
echo TIMESTAMP_ISO="$ts_iso" >> "$GITHUB_OUTPUT"
33+
echo TIMESTAMP_TAG="$ts_tag" >> "$GITHUB_OUTPUT"
34+
35+
- name: Checkout
36+
uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
37+
with:
38+
fetch-depth: 1
39+
40+
- name: Fetch main branch
41+
run: git fetch --depth=1 origin main
42+
43+
- name: Free disk space
44+
uses: ./.github/actions/free-disk-space
45+
46+
- name: Install `just`
47+
uses: taiki-e/install-action@763e3324d4fd026c9bd284c504378585777a87d5 # v2
48+
with:
49+
tool: just
50+
51+
- name: Install stable toolchain
52+
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master
53+
with:
54+
toolchain: stable
55+
targets: wasm32-wasip2
56+
57+
- name: build "add one" example (debug)
58+
run: just guests::rust::build-add-one-debug
59+
60+
- name: build "add one" example (release)
61+
run: just guests::rust::build-add-one-release
62+
63+
- name: build python guest (debug)
64+
run: just guests::python::build-debug
65+
66+
- name: build python guest (release)
67+
run: just guests::python::build-release
68+
69+
# we need unique file names for the release
70+
- name: stage release files
71+
run: |
72+
mkdir out
73+
mv target/wasm32-wasip2/debug/examples/add_one.wasm out/example_add_one.debug.wasm
74+
mv target/wasm32-wasip2/release/examples/add_one.wasm out/example_add_one.release.wasm
75+
mv target/wasm32-wasip2/debug/datafusion_udf_wasm_python.wasm out/datafusion_udf_wasm_python.debug.wasm
76+
mv target/wasm32-wasip2/release/datafusion_udf_wasm_python.wasm out/datafusion_udf_wasm_python.release.wasm
77+
78+
- name: attestation
79+
id: attestation
80+
uses: actions/attest-build-provenance@977bb373ede98d70efdf65b84cb5f73e068dcc2a # v3
81+
with:
82+
subject-path: out/*.wasm
83+
84+
- name: publish release
85+
uses: softprops/action-gh-release@5be0e66d93ac7ed76da52eca8bb058f665c3a5fe # v2
86+
with:
87+
body: |
88+
WASM guest binaries.
89+
90+
**Commit:** [`${{ github.sha }}`](${{ github.server_url }}/${{ github.repository }}/tree/${{ github.sha }})
91+
**Build Timestamp:** `${{ steps.timestamp.outputs.TIMESTAMP_ISO }}`
92+
**Build Attestation:** <${{ steps.attestation.outputs.attestation-url }}>
93+
preserve_order: true
94+
files: out/*.wasm
95+
name: "WASM Binaries ${{ steps.timestamp.outputs.TIMESTAMP_ISO }}"
96+
tag_name: "wasm-binaries/${{ steps.timestamp.outputs.TIMESTAMP_TAG }}/${{ github.sha }}"
97+
fail_on_unmatched_files: true
98+
target_commitish: "${{ github.sha }}"
99+
make_latest: false

‎CONTRIBUTING.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,8 +105,27 @@ Current thread 0x012bd368 (most recent call first):
105105

106106
Then the [Python Standard Library] was not found or not bundled correctly. You may try to wipe `guests/python/download`. If that does not help, open a ticket.
107107

108+
## Pre-built WASM Binaries
109+
We offer pre-built WASM guest binaries to simplify integration into other software artifacts. This way you only need to depend on the host crates and don't need a WASI compilation toolchain. It also cuts build and test times, since you can include a release-optimized guest even during development and CI. Release-optimized guests are smaller and can be JIT-compiled and executed faster.
110+
111+
You find WASM builds published as releases named "WASM Binaries".
112+
113+
### Triggering A Build
114+
To trigger a build, you need write access to this repository. Then use the [GitHub CLI] tool and run:
115+
116+
```console
117+
$ gh workflow run Prebuild
118+
```
119+
120+
This will trigger the build on the `main` branch. If you need a different branch, use:
121+
122+
```console
123+
$ gh workflow run Prebuild --ref <BRANCH_NAME>
124+
```
125+
108126

109127
[cargo-deny]: https://embarkstudios.github.io/cargo-deny/
128+
[GitHub CLI]: https://cli.github.com/
110129
[insta]: https://insta.rs/
111130
[just]: https://github.com/casey/just
112131
[Python]: https://www.python.org/

0 commit comments

Comments
 (0)