Merge branch 'main' into jn/cancel-signal

Author: vtjnash

devdocs/create-new-release-pipeline.md

@@ -0,0 +1,49 @@
+1. Go to https://buildkite.com/julialang
+2. In the top-right-hand corner, click on the white "New Pipeline" button.
+3. You are now on a page that says "New Pipeline". Fill in the fields as follows:
+    - Name: `julia-release-1.9`
+    - Description: `https://github.com/JuliaLang/julia`
+    - Git Repository:
+        - Step i. Click on the dropdown that says "Any account", and click on `JuliaLang`.
+        - Step ii: Click on the text field that says "[email protected]:your/repo.git", and from the dropdown menu that appears, click on `JuliaLang/julia`.
+        - Step iii: Next to "Checkout using", click on the radio button for `HTTPS`. Once you do that, the URL should automatically change to `https://github.com/JuliaLang/julia.git`
+    - Auto-create webhooks: make sure that this checkbox IS checked.
+    - Teams: select the following two teams:
+        - `[ALL USERS] Base Julia CI (build and read)`
+        - `Base Julia CI (Slack notifications)`
+4. At the bottom of the page, click on the green "Create Pipeline" button.
+5. Go to https://buildkite.com/julialang/julia-release-1-dot-9 and click on the white "Edit steps" button.
+6. Delete ALL text in the text box.
+7. Into the text box, paste the exact contents from the folllowing file: https://github.com/JuliaCI/julia-buildkite/blob/main/pipelines/main/0_webui.yml
+8. Click on the white "Save Steps" button.
+9. Go to https://buildkite.com/julialang/julia-release-1-dot-9 and click on the white "Pipeline Settings" (or just "Settings") button in the top-right-hand corner.
+10. In the left-hand side, under "Pipeline Settings", click on "General". Then, scroll down to the bottom of the page. In the "Pipeline Management" section, click on the white "Make Pipeline Public" button. If you are asked for confirmation, click on the green "Make Pipeline Public" button to confirm your decision.
+11. Go to https://buildkite.com/julialang/julia-release-1-dot-9 and click on the white "Pipeline Settings" (or just "Settings") button in the top-right-hand corner.
+12. In the left-hand side, under "Pipeline Settings", click on "Builds". Then, in the "Build Skipping" section, do the following actions:
+    - Step i: Make sure that the "Skip Intermediate Builds" checkbox IS checked.
+    - Step ii: Make sure that the "Cancel Intermediate Builds" checkbox IS checked.
+    - Step iii: Under "Skip Intermediate Builds", in the text box, enter the following text: `!master !main !release-*`
+    - Step iv: Under "Cancel Intermediate Builds", in the text box, enter the following text: `!master !main !release-*`
+    - Step v: Click on the green "Save Build Skipping" button.
+13. Go to https://buildkite.com/julialang/julia-release-1-dot-9 and click on the white "Pipeline Settings" (or just "Settings") button in the top-right-hand corner.
+14. In the left-hand side, under "Pipeline Settings", click on "GitHub". Then, in the "Branch Limiting" section, do the following actions:
+    - Step i: For "Branch Filter Pattern", enter the following text: `release-1.9 v1.9.*`
+    - Step ii: Click on the green "Save Branch Limiting" button.
+15. You are still on the "Pipeline Settings ⟶ GitHub" page. In the "GitHub Settings" section, do the following actions:
+    - Step i: Make sure that the radio button for "Trigger builds after pushing code" IS selected.
+    - Step ii: Make sure that the checkbox for "Filter after pushing code" IS selected.
+    - Step iii: Under "Filter builds using a conditional", enter the following text: `(build.pull_request.base_branch == null) || (build.pull_request.base_branch == "release-1.9") || (build.pull_request.base_branch == "backports-release-1.9")`
+    - Step iv: Make sure that the checkbox for "Build Pull Requests" IS selected.
+    - Step v: Under "Build Pull Requests", make sure that the checkbox for "Build pull requests from third-party forked repositories" IS selected.
+    - **Step vi: Under "Build pull requests from third-party forked repositories", make sure that the checkbox for "Prefix third-party fork branch names" IS selected. It's important to make sure you complete this step.**
+    - Step vii: Make sure that the checkbox for "Build tags" IS selected.
+    - Step viii: Make sure that the checkbox for "Cancel deleted branch builds" IS selected.
+    - Step ix: Make sure that the checkbox for "Update commit statuses" IS selected.
+    - Step x: Under "Update commit statuses", for "Show blocked builds in GitHub as", from the drop-down menu, select "Pending".
+    - Step xi: Make sure that the checkbox for "Create a status for each job" is NOT selected.
+    - Step xii: Make sure that the checkbox for "Separate statuses for pull requests" is NOT selected.
+    - Step xiii: At the bottom of the page, click on the green "Save GitHub Settings" button.
+16. Go to https://buildkite.com/julialang/julia-release-1-dot-9 and click on the white "Pipeline Settings" button.
+17. In the left-hand side, under "Pipeline Settings", click on "Teams". Then do the following actions:
+    - Step i: Next to "Base Julia CI (Slack notifications)", click on the text `Full Access` and change it to `Read Only`.
+    - Step ii: Next to "[ALL USERS] Base Julia CI (build & read)", click on the text `Full Access` and change it to `Build & Read`.

pipelines/main/launch_unsigned_jobs.yml

@@ -4,9 +4,9 @@
 #
 # Yes, this is creating another layer of indirection; the flow now looks like:
 #
-#   [webui] -> launch_unsigned_builders.yml -> misc/whitespace.yml
+#   [webui] -> launch_unsigned_builders.yml -> misc/llvmpasses.yml
 #
-# when we could theoretically just have the `webui` launch `misc/whitespace.yml`,
+# when we could theoretically just have the `webui` launch `misc/llvmpasses.yml`,
 # however this raises the bar for contributors to add new (unsigned) steps to
 # our CI configuration, so I'd rather live with an extra layer of indirection
 # and only need to touch the webui configuration when we need to alter
@@ -65,7 +65,7 @@ steps:
           buildkite-agent pipeline upload .buildkite/pipelines/main/misc/pdf_docs/build_pdf_docs.yml
           buildkite-agent pipeline upload .buildkite/pipelines/main/misc/embedding.yml
           buildkite-agent pipeline upload .buildkite/pipelines/main/misc/llvmpasses.yml
-          buildkite-agent pipeline upload .buildkite/pipelines/main/misc/whitespace.yml
+          # buildkite-agent pipeline upload .buildkite/pipelines/main/misc/whitespace.yml # Currently runs in GitHub Actions instead of Buildkite
 
           buildkite-agent pipeline upload .buildkite/pipelines/main/misc/sanitizers/asan.yml
           buildkite-agent pipeline upload .buildkite/pipelines/main/misc/sanitizers/tsan.yml

pipelines/main/platforms/build_linux.yml

@@ -36,4 +36,5 @@ steps:
           # Receive some environment mappings from our templating engine
           TRIPLET: "${TRIPLET?}"
           MAKE_FLAGS: "${MAKE_FLAGS?}"
+          ROOTFS_IMAGE_NAME: "${ROOTFS_IMAGE_NAME?}"
           BUILDKITE_CANCEL_SIGNAL: "SIGQUIT"

pipelines/scheduled/launch_unsigned_jobs.yml

@@ -4,9 +4,9 @@
 #
 # Yes, this is creating another layer of indirection; the flow now looks like:
 #
-#   [webui] -> launch_unsigned_builders.yml -> misc/whitespace.yml
+#   [webui] -> launch_unsigned_builders.yml -> misc/llvmpasses.yml
 #
-# when we could theoretically just have the `webui` launch `misc/whitespace.yml`,
+# when we could theoretically just have the `webui` launch `misc/llvmpasses.yml`,
 # however this raises the bar for contributors to add new (unsigned) steps to
 # our CI configuration, so I'd rather live with an extra layer of indirection
 # and only need to touch the webui configuration when we need to alter

utilities/build_julia.sh

@@ -19,6 +19,11 @@ ld -v
 echo
 buildkite-agent --version
 
+if [[ "${ROOTFS_IMAGE_NAME-}" == "llvm_passes" ]]; then
+    echo "--- Update CMake"
+    contrib/download_cmake.sh
+fi
+
 echo "--- Collect make options"
 # These are the flags we'll provide to `make`
 MFLAGS=()