Bug 2019457 - Refactor CI/CD test cases to make it easier to incorporate automated tests from extensions into the normal test framework

[dklawren]

• Move all of the helper libraries from qa/t/lib to Bugzilla/QA
• Move the config file to conf/
• Move the generate_test_data.pl to scripts/
• Update all of the current scripts to use the new Bugzilla/QA location
• Update the Github actions config to use the new way of calling all of the tests

[Copilot]

Pull request overview

Refactors the CI/QA test harness to use new Bugzilla::QA::* and Bugzilla::Test::* helper modules, relocates QA configuration/scripts, and updates GitHub Actions to run tests using the new invocation patterns.

Changes:

• Updated test files to import helpers from Bugzilla::QA::* / Bugzilla::Test::* and adjusted @INC (use lib) accordingly.
• Moved/updated QA config and data generation script paths (e.g., config under conf/, generator under scripts/).
• Modified Docker/GitHub Actions test commands to pass explicit prove args/patterns.

Reviewed changes

Copilot reviewed 123 out of 124 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
t/critic.t	Switches from Support::Files to Bugzilla::Test::Files and updates use lib.
t/bmo/passwords.t	Uses Bugzilla::QA::Util and updates use lib.
t/bmo/oauth2-client.t	Uses Bugzilla::QA::Util and updates use lib.
t/bmo/bounced-emails.t	Uses Bugzilla::QA::Util and updates use lib.
t/012throwables.t	Updates helper namespaces; changes error-usage scanning logic.
t/011pod.t	Switches to Bugzilla::Test::Files and updates test count.
t/010dependencies.t	Switches to Bugzilla::Test::Files and updates module extraction loop.
t/009bugwords.t	Switches to Bugzilla::Test::{Files,Templates} and updates include paths.
t/008filter.t	Switches to Bugzilla::Test::Templates and updates include paths.
t/007util.t	Switches to Bugzilla::Test::Files and updates use lib.
t/006spellcheck.t	Switches to Bugzilla::Test::Files and updates test list.
t/005whitespace.t	Switches to Bugzilla::Test::{Files,Templates} and updates test list.
t/004template.t	Switches to Bugzilla::Test::Templates and updates use lib.
t/002goodperl.t	Switches to Bugzilla::Test::Files and updates test list.
t/001compile.t	Switches to Bugzilla::Test::Files and uses Bugzilla::Test::Systemexec for compile checks.
scripts/generate_test_data.pl	Simplifies @INC setup; reads config from conf/selenium_test.conf by default.
scripts/entrypoint.pl	Updates how tests are invoked and where test data generation runs from.
qa/t/webservice_user_offer_account_by_email.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_user_login_logout.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_user_get.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_user_create.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_product_get.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_product_create.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_product.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_jsonp.t	Uses Bugzilla::QA::Util and updates client callsite.
qa/t/webservice_group_create.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_bugzilla.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_bug_update_see_also.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_update.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_search.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_legal_values.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_bug_history.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_get_bugs.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_get.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_fields.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_bug_create.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/webservice_bug_comments.t	Updates isa checks to Bugzilla::QA::RPC::* and imports Bugzilla::QA::*.
qa/t/webservice_bug_attachments.t	Updates generator file path and isa checks to Bugzilla::QA::RPC::*.
qa/t/webservice_bug_add_comment.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/webservice_bug_add_attachment.t	Updates isa checks to Bugzilla::QA::RPC::* and imports Bugzilla::QA::Util.
qa/t/rest_triage_owners.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_see_also.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/rest_search_api.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_reminders.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_relationship_trees.t	Uses Bugzilla::QA::{Util,Tests} and updates use lib.
qa/t/rest_pulsebot.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_last_change_time_non_bot.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_lando_uplift.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_group_get.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_github_push_comment.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_github_pull_request.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_components.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_classification.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_bugzilla.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/rest_bug.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/archived/test_email_preferences.t	Uses Bugzilla::QA::Util.
qa/t/4_test_webhooks.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_votes.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_user_privs.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_user_preferences.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_user_matching.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_user_groups.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_time_summary.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_target_milestones.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_sudo_sessions.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_status_whiteboard.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_show_all_products.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/4_test_shared_searches.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_see_also.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_security.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_search.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_saved_searches.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_sanity_check.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_require_login.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_reminders.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_rate_limit.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_qa_contact.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_private_attachments.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_new_release.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_net_attachments.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/3_test_component_graveyard.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_milestones.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_login_totp.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_login_duo.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_login.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_lockout.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_keywords.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_groups.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_flags2.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_flags.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_enter_new_bug.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_edit_products_properties.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_dependencies.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/2_test_default_groups.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_duo_requirement.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_custom_fields_admin.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_custom_fields.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_create_user_accounts.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_cookie_consent.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_config.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_classifications.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_choose_priority.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_bug_edit.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_bmo_retire_values.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_bmo_enter_new_bug.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_bmo_autolinkification.t	Uses Bugzilla::QA::Util and updates use lib.
qa/t/1_test_attention.t	Uses Bugzilla::QA::Util and updates use lib.
external_test_api.pl	Updates hardcoded selenium config path to /app/conf/selenium_test.conf.
extensions/JiraWebhookSync/t/bmo/jira_webhook_sync.t	Uses Bugzilla::QA::Util and updates use lib.
extensions/BMO/t/bmo/recorded_future.t	Uses Bugzilla::QA::Util and updates use lib.
extensions/BMO/t/bmo/bmo_etl.t	Uses Bugzilla::QA::Util and updates use lib.
docker-compose.test.yml	Updates BZ_QA_CONF_FILE to point at /app/conf/selenium_test.conf.
conf/selenium_test.conf	Updates attachment fixture path to /app/conf/patch.diff.
conf/patch.diff	Adds patch fixture under conf/.
Bugzilla/Test/Templates.pm	Renames package to Bugzilla::Test::Templates and updates helper imports.
Bugzilla/Test/Systemexec.pm	Renames package to Bugzilla::Test::Systemexec.
Bugzilla/Test/Files.pm	Renames package to Bugzilla::Test::Files.
Bugzilla/QA/Util.pm	Renames package to Bugzilla::QA::Util, updates config path default, updates RPC requires, and adjusts Selenium helper behavior.
Bugzilla/QA/Tests.pm	Renames package to Bugzilla::QA::Tests.
Bugzilla/QA/RPC/XMLRPC.pm	Renames package to Bugzilla::QA::RPC::XMLRPC and updates base class.
Bugzilla/QA/RPC/JSONRPC.pm	Renames package(s) to Bugzilla::QA::RPC::* and updates class references.
Bugzilla/QA/RPC.pm	Renames package to Bugzilla::QA::RPC and updates imports and isa checks.
Bugzilla/QA/REST.pm	Renames QA REST client module (but currently has a package naming issue).
.github/workflows/ci.yml	Updates CI to pass explicit prove args/patterns for sanity/webservice/selenium/bmo tests.

Comments suppressed due to low confidence (2)

Bugzilla/QA/REST.pm:12

• Bugzilla/QA/REST.pm declares package Bugzillas::QA::REST; (extra “s”), but the rest of the file references Bugzilla::QA::REST (EXPORT var, bless). This mismatch will prevent the module from being loaded/used correctly. Rename the package to Bugzilla::QA::REST (and ensure all internal package-qualified variables match).
  Bugzilla/QA/Util.pm:332
• go_to_bug removed the previous delay that was explicitly documented as avoiding a race (“Sometimes we try to click edit bug before it is ready…”). There isn’t an alternative wait here before clicking mode-btn-readonly, so this may reintroduce intermittent Selenium failures. Please add an explicit wait for the edit button to be present/clickable (preferred) or restore the short delay.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.