feat(ci): add install.yaml support for dynamic package installation#5
Open
feat(ci): add install.yaml support for dynamic package installation#5
Conversation
Add per-tool install.yaml files that map package names across different package managers (homebrew, macports, apt, dnf, yum, apk, pacman, chocolatey, winget). These files enable CI workflows to automatically install required tools before validation. Changes: - Add install.yaml files for 54 tools - Add install_tools.rb script for dynamic package installation - Update validate-profiles.yml to install tools before validation - Add INSTALL_SCHEMA.adoc documentation - Add scripts/generate_install_yamls.rb for generating install files The install_tools.rb script: - Detects platform and package manager - Reads install.yaml files from tools/*/ - Checks if tools are already installed - Installs missing packages in a single command Data sourced from x-cmd project for cross-platform package mappings.
Move install.yaml files from tools/*/ to packages/ to avoid validation conflicts with the tool.schema.yaml which requires version and profiles fields that install files don't have.
- magick: add apt, yum, apk, macports, chocolatey, winget packages - rg: add yum, apk, macports, chocolatey, winget packages - ripgrep-all: fix homebrew to use 'rga', add chocolatey, pacman - yq_jq: add macports, apt, dnf, yum, chocolatey, winget packages - Add detect_coverage_gaps.rb script to identify missing packages Also add OS_BUNDLED_TOOLS and SYSTEM_TOOLS constants to properly handle tools that are pre-installed on certain platforms.
ronaldtse
force-pushed
the
feat/install-yaml-support
branch
from
494c6cd to
ab4dc9f
Compare
yq_jq (kislyuk/yq) is only available as a system package on apt. On other platforms, it requires 'pip install yq'.
- Add ToolResult class for tracking per-tool verification status - Add verify_tool, run_smoke_test, generate_report methods - Add --verify, --fail-on-missing, --report-format CLI flags - Generate JSON and Markdown verification reports - Upload verification reports as CI artifacts - Add smoke tests to critical packages (magick, ffmpeg, pandoc, jq) - Add smoke tests to important packages (rg, gl, bat, fd, fzf) - Document v2.0 schema in INSTALL_SCHEMA_V2.md Implements Phase 2 of per-tool verification plan.
- Add smoke tests to 45 packages using add_smoke_tests.rb script
- Fix YAML alias parsing in install_tools.rb (aliases: true)
- Fix encoding issue in smoke test output handling
- All 54 packages now have smoke tests defined
Smoke tests use appropriate commands per tool:
- Standard: {tool} --version
- Special cases: gs, ssh, scp, bzip2, etc.
- Add system_tool flag for ping, ssh, scp (OS-bundled on macOS/Windows) - Update install_tools.rb to handle system_tool flag: - If no package for current platform AND system_tool=true, verify system executable - Show ◎ icon and [SYSTEM] marker for system tools - Consider system_tool as install success in reports This fixes false 'not available' errors for tools that come bundled with the OS on certain platforms.
Replace implicit system_tool logic with explicit platforms section:
platforms:
darwin:
bundled: true
windows:
bundled: true
linux:
bundled: false
This is cleaner and more explicit than the previous 'if no package entry
then check system_tool' approach.
Add warning when packages directory is not found. Log number of package files found for debugging Windows issue.
Use File.expand_path to normalize path separators when finding package files. This fixes the issue where mixed Windows/Unix path separators (D:\path/packages) caused Dir.glob to fail.
- Install packages individually instead of all at once - Continue with verification even if some packages fail to install - Add scripts/test_bundled_tools.rb to test bundled tools via Docker - Log failed packages but don't fail the build
Allows Ghostscript tool to be used with PowerShell Core (pwsh) on macOS and Linux platforms. This is needed for testing PowerShell argument handling and for cross-platform consistency.
- Ubuntu apt imagemagick (v6) provides convert/identify, not magick - Use convert --version for apt/yum smoke tests - magick --version for platforms with ImageMagick 7+
The chocolatey 'ghostscript' package has a known issue where it does not add gswin64c/gs to PATH. Using 'Ghostscript.app' package instead which properly handles PATH configuration. Also added Windows-specific provides (gswin64c, gs) for both chocolatey and winget.
On Windows, chocolatey updates PATH but the current shell session doesn't see the change. This fix: - Checks common chocolatey installation paths for executables - Checks Ghostscript installation directory (C:/Program Files/gs/gs*/bin) - Updates smoke test to use full executable path when found - Adds script to test bundled tools across Linux distributions
Major improvements: - Separate install and verify phases for clarity - Test each tool individually with clear progress output - Add Ukiryu discovery test to verify tool loading works - Fix path doubling bug in smoke test execution - Add colors and emojis for professional output - Add --color/--no-color options for CI environments - Show clear error messages for each failure type - Generate markdown reports by default The workflow now tests: 1. Package installation status 2. Executable availability 3. Smoke test execution 4. Ukiryu tool discovery/loading All failures are clearly reported with specific error messages.
ICONS[:INFO] should be just INFO since we include the module.
ukiryu validate file PATH instead of ukiryu validate --definition PATH
- bat: On Ubuntu/Debian, executable is 'batcat' not 'bat' - gl: glab package provides 'glab' executable, updated provides list - Added missing version files for tools: - awk: 5.3.yaml, 1.36.1.yaml - bzip2: 1.0.6.yaml, 1.36.1.yaml - gzip: 1.12.yaml, 1.10.yaml, 1.36.1.yaml - ping: 1.0.yaml, 2024.01.17.yaml - tar: 1.35.yaml, 1.36.1.yaml - wget: 1.24.yaml, 1.36.1.yaml
The commands field must be an array of command objects, not an object with named properties. Fixed all version files to use the correct array format.
- Add install_script field to package schema for tools that require custom shell commands instead of package manager - Add apt installation for ripgrep-all using binary download - install_script runs via bash and allows complex installation logic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add per-tool install.yaml files that map package names across different package managers. This enables CI workflows to automatically install required tools before validation.
Changes
How It Works
The install_tools.rb script:
Test Plan
🤖 Generated with Claude Code