Skip to content

Commit 63791fa

Browse files
mckeesfernando79513
authored and
stanley31huang
committed
Add checkbox-gfx Vulkan tests (New) (#2029)
* Add checkbox-gfx Vulkan tests * Fix formatting issue found by Black * Move Vulkan tests into Python and use sibling tests * Remove slave language * Remove unneeded bootstraps * Reformat python file with black * Add testing and change name for vulkan CTS runner script * Consolidate CI scripts into one * Add paths to Python runner to clean up sibling job line length * Update contrib/checkbox-gfx/bin/test-vulkan Co-authored-by: Fernando Bravo <[email protected]> --------- Co-authored-by: Fernando Bravo <[email protected]>
1 parent b17c213 commit 63791fa

File tree

23 files changed

+920
-0
lines changed

23 files changed

+920
-0
lines changed

‎contrib/checkbox-gfx/.github/workflows/build-installers.yaml‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
name: "Build Installers"
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
version:
7+
description: 'Ubuntu version of the runner'
8+
required: true
9+
default: 'noble'
10+
type: choice
11+
options:
12+
- jammy
13+
- noble
14+
15+
arch:
16+
description: 'Architecture of the runner'
17+
required: true
18+
default: 'amd64'
19+
type: choice
20+
options:
21+
- amd64
22+
- arm64
23+
24+
pull_request:
25+
branches:
26+
- main
27+
paths:
28+
- bin/install-vulkan
29+
30+
jobs:
31+
manual-build:
32+
if: github.event_name == 'workflow_dispatch'
33+
strategy:
34+
matrix:
35+
test_name: [vulkan]
36+
name: "Manual Build - ${{ matrix.test_name }}"
37+
runs-on: [self-hosted, linux, "${{ inputs.arch }}", "${{ inputs.version }}"]
38+
39+
steps:
40+
- name: Checkout repository
41+
uses: actions/checkout@v4
42+
43+
- name: Run install-${{ matrix.test_name }}
44+
shell: bash
45+
run: bin/install-${{ matrix.test_name }}
46+
47+
- name: Upload artifacts
48+
uses: actions/upload-artifact@v4
49+
with:
50+
name: "output-${{ matrix.test_name }}-\
51+
${{ inputs.arch }}-${{ inputs.version }}"
52+
path: /usr/local/checkbox-gfx/*
53+
54+
pr-build:
55+
if: github.event_name == 'pull_request'
56+
strategy:
57+
matrix:
58+
arch: [amd64, arm64]
59+
test_name: [vulkan]
60+
name: "Installer testing"
61+
runs-on: [self-hosted, linux, "${{ matrix.arch }}", noble]
62+
63+
steps:
64+
- name: "Installers - ${{ matrix.test_name }} - ${{ matrix.arch }}"
65+
run: echo "Running ${{ matrix.test_name }} ${{ matrix.arch }} install"
66+
67+
- name: Checkout repository
68+
uses: actions/checkout@v4
69+
70+
- name: Run install-${{ matrix.test_name }}
71+
shell: bash
72+
run: bin/install-${{ matrix.test_name }}
73+
74+
- name: Upload artifacts
75+
uses: actions/upload-artifact@v4
76+
with:
77+
name: output-${{ matrix.test_name }}-${{ matrix.arch }}
78+
path: /usr/local/checkbox-gfx/*

‎contrib/checkbox-gfx/.github/workflows/check-pull-request.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: Do a snap build
2+
3+
on:
4+
pull_request:
5+
branches: [ "main" ]
6+
paths-ignore:
7+
- '**.md' # Don't trigger if doc updated
8+
9+
jobs:
10+
build:
11+
12+
runs-on: ubuntu-24.04
13+
14+
steps:
15+
- uses: actions/checkout@v4
16+
with:
17+
# Fetch all of history so DM can determine its own version from git.
18+
fetch-depth: 0
19+
20+
- uses: snapcore/action-build@v1
21+
name: Build checkbox-gfx snap
22+
id: build
23+
with:
24+
snapcraft-channel: latest/stable
25+
26+
- name: Verify checkbox-gfx snap
27+
run: |
28+
# Make sure it is installable.
29+
sudo snap install checkbox24
30+
sudo snap install --dangerous --classic ${{ steps.build.outputs.snap }}
31+

‎contrib/checkbox-gfx/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.snap

‎contrib/checkbox-gfx/README.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Welcome to the Checkbox GFX project!
2+
3+
This repository contains the Checkbox GFX Provider (GFX-specific test cases and test plans for [Checkbox]) as well as everything that is required to build the [checkbox-gfx] snap in the snapstore.
4+
5+
# Checkbox GFX Provider
6+
7+
Located in the `checkbox-provider-gfx` directory, it contains:
8+
9+
- the test cases (also called "jobs" in the Checkbox jargon) and test plans to be run by Checkbox (in the `units` directory)
10+
11+
# Requirements
12+
13+
- Ubuntu Noble (24.04)
14+
- Supported hardware platforms:
15+
- Intel platforms with recent GPU (>= Broadwell)
16+
17+
# Installation
18+
19+
Install the Checkbox runtime and build/install the gfx provider snaps:
20+
21+
```shell
22+
sudo snap install --classic snapcraft
23+
sudo snap install checkbox24
24+
lxd init --auto
25+
git clone https://github.com/canonical/checkbox-gfx
26+
cd checkbox-gfx
27+
snapcraft
28+
sudo snap install --dangerous --classic ./checkbox-gfx_1.0_<arch>.snap
29+
```
30+
31+
Make sure that the provider service is running and active:
32+
33+
```shell
34+
systemctl status snap.checkbox-gfx.run-agent.service
35+
```
36+
37+
# Install dependencies
38+
39+
Most tests need dependencies, and a helper script is available to install each category of tests:
40+
41+
```shell
42+
checkbox-gfx.install-lvl-zero
43+
checkbox-gfx.install-lvl-zero-rt
44+
checkbox-gfx.install-crucible
45+
checkbox-gfx.install-vulkan
46+
checkbox-gfx.install-opengl
47+
checkbox-gfx.install-opencl
48+
```
49+
50+
# Automated Run
51+
52+
Each category of tests is run separately:
53+
54+
```shell
55+
checkbox-gfx.test-lvl-zero
56+
checkbox-gfx.test-lvl-zero-rt
57+
checkbox-gfx.test-opencl
58+
```
59+
60+
Due to some tests causing dropped SSH connections, running the tests remotely should be done like this:
61+
62+
1. Install checkbox-gfx on both the remote machine and the local machine
63+
2. From the checkbox-gfx directory, run the following command
64+
65+
```shell
66+
checkbox-gfx.checkbox-cli control <REMOTE IP> bin/<test bin>
67+
```
68+
69+
# Develop the Checkbox GFX provider
70+
71+
Since snaps are immutable, it is not possible to modify the content of the scripts or the test cases. Fortunately, Checkbox provides a functionality to side-load a provider on the DUT.
72+
73+
Therefore, if you want to edit a job definition, a script or a test plan, run the following commands on the DUT:
74+
75+
```shell
76+
cd $HOME
77+
git clone https://github.com/canonical/checkbox-gfx
78+
mkdir /var/tmp/checkbox-providers
79+
cp -r $HOME/checkbox-gfx/checkbox-provider-gfx /var/tmp/checkbox-providers/
80+
```
81+
82+
You can then modify the content of the provider in `/var/tmp/checkbox-providers/checkbox-provider-gfx/`, and it's this version that will be used when you run the tests.
83+
84+
Please refer to the [Checkbox documentation] on side-loading providers for more information.
85+
86+
[Checkbox]: https://checkbox.readthedocs.io/
87+
[Checkbox documentation]: https://checkbox.readthedocs.io/en/latest/side-loading.html

‎contrib/checkbox-gfx/bin/checkbox-cli-wrapper‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/bin/sh
2+
3+
# wrapper around the checkbox-cli
4+
# use absolute path in order to not use system checkbox-cli (from deb packages)
5+
exec /snap/checkbox24/current/bin/checkbox-cli "$@"

‎contrib/checkbox-gfx/bin/configure‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
#!/usr/bin/env python3
2+
# Copyright 2018-2022 Canonical Ltd.
3+
# All rights reserved.
4+
#
5+
# Written by:
6+
# Maciej Kisielewski <[email protected]>
7+
# Sylvain Pineau <[email protected]>
8+
import os
9+
import re
10+
import sys
11+
12+
sys.path.append(os.path.expandvars("$SNAP/usr/lib/python3/dist-packages"))
13+
sitepkgpath = "$SNAP/lib/python3.12/site-packages"
14+
sys.path.append(os.path.expandvars(sitepkgpath))
15+
16+
sys.path.append(os.path.expandvars(
17+
"/snap/checkbox24/current/usr/lib/python3/dist-packages"))
18+
runtimepath = "/snap/checkbox24/current/lib/python3.12/site-packages"
19+
sys.path.append(os.path.expandvars(runtimepath))
20+
21+
try:
22+
from checkbox_support.snap_utils.config import update_configuration
23+
from checkbox_support.snap_utils.config import print_checkbox_conf
24+
except ImportError:
25+
msg = """
26+
checkbox-support not found!
27+
You need to install the checkbox24 snap:
28+
29+
snap install checkbox24
30+
"""
31+
print(os.path.expandvars(msg), file=sys.stderr)
32+
sys.exit(1)
33+
34+
35+
def main():
36+
# we need run as root to be able to write to /var/snap/...
37+
if os.geteuid() != 0:
38+
print('You have to run this command with sudo')
39+
return
40+
41+
if len(sys.argv) > 1 and sys.argv[1] == '-l':
42+
print_checkbox_conf()
43+
return
44+
45+
key_re = re.compile(r"^(?:[A-Z0-9]+_?)*[A-Z](?:_?[A-Z0-9])*$")
46+
vars_to_set = dict()
47+
for pair in sys.argv[1:]:
48+
k, _, v = pair.partition('=')
49+
if not key_re.match(k) or not v:
50+
raise SystemExit("'%s' is not a valid configuration entry. "
51+
"Should be KEY=val" % pair)
52+
k = k.replace('_', '-').lower()
53+
vars_to_set[k] = v
54+
update_configuration(vars_to_set)
55+
56+
57+
if __name__ == '__main__':
58+
main()

‎contrib/checkbox-gfx/bin/install-vulkan‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
source $(dirname "$0")/setup-env.sh
6+
7+
sudo apt update -y
8+
sudo apt install -y build-essential cmake python3-pip meson pkg-config cmake libpng-dev libvulkan-dev mesa-common-dev
9+
pip3 install lxml --break-system-packages
10+
11+
echo "Installing VK-GL-CTS for Vulkan"
12+
cd $WORKING_DIR
13+
git clone https://github.com/KhronosGroup/VK-GL-CTS
14+
cd VK-GL-CTS
15+
git checkout vulkan-cts-1.4.1.3
16+
python3 external/fetch_sources.py
17+
mkdir build && cd build
18+
19+
# Different archs need different buildflags
20+
if [ "$ARCH" == "x86_64" ]; then
21+
CMAKE_CLANG_FLAGS=-m64
22+
elif [ "$ARCH" == "aarch64" ]; then
23+
CMAKE_CLANG_FLAGS=-march=armv8-a
24+
fi
25+
26+
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS=$CMAKE_CLANG_FLAGS -DCMAKE_CXX_FLAGS=$CMAKE_CLANG_FLAGS
27+
# 64GB RAM wasn't enough for -j. Add nproc
28+
make -j`nproc`
29+
sudo mv $WORKING_DIR/VK-GL-CTS $INSTALL_DIR

‎contrib/checkbox-gfx/bin/remove‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
#!/bin/bash
2+
3+
source $(dirname "$0")/setup-env.sh
4+
sudo rm -rf $INSTALL_DIR
5+
rm -rf $WORKING_DIR

‎contrib/checkbox-gfx/bin/setup-env.sh‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/bin/bash
2+
3+
export ARCH=$(uname -p)
4+
export INSTALL_DIR=/usr/local/checkbox-gfx
5+
export WORKING_DIR=$HOME/.checkbox-gfx-working-dir
6+
7+
sudo mkdir -p $INSTALL_DIR
8+
sudo mkdir -p $WORKING_DIR
9+
sudo chown -R $USER $WORKING_DIR
10+
11+
# Get vendor
12+
if [[ $(lscpu | grep "GenuineIntel") ]]; then
13+
export VENDOR=Intel
14+
elif [[ $(lscpu | grep "AuthenticAMD") ]]; then
15+
export VENDOR=AMD
16+
elif [[ $(lscpu | grep "Qualcomm") ]]; then
17+
export VENDOR=Qualcomm
18+
fi
19+

‎contrib/checkbox-gfx/bin/shell-wrapper‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#!/bin/bash
2+
3+
echo "$SNAP_NAME runtime shell, type 'exit' to quit the session"
4+
exec bash

0 commit comments

Comments
 (0)