Set `--show-builtin-errors` flag to false by default

[dgeorgievski]

I am using both conftest verify to validate JSON documents, and opa test as companion tool to test, debug and troubleshoot Rego policies.

Recently, I have run into an interesting challenge that in the end it turned out to be in a difference how conftest and opa CLIs handle errors from Rego's builtin functions.

Some of my rules have the following structure

foo(val, ...) := result if {
   ...
   is_number(to_number(val))
   ...
}

This line, is_number(to_number(val)), is checking if the provided val has a string type and could be a number or alphanumeric string like 1:1

opa test correctly handles the to_number error for invalid values as undefined or false which means the above rule work as expected.

With the same policies, conftest verify produces an error

$ conftest verify --policy policy/dir/path
Error: running verification: run test: policies/common/sbom_pkg_ver_constraint/utils.rego:175: eval_builtin_error: to_number: strconv.ParseFloat: parsing "1:1": invalid syntax

But, if --show-builtin-errors=false is used conftest works correctly as well

$ conftest verify --policy policy/dir/path --show-builtin-errors=false

69 tests, 69 passed, 0 warnings, 0 failures, 0 exceptions, 0 skipped

Considering I am using opa and conftest together when developing rego policies it would be nice if the default setting for --show-builtin-errors is changed from true to false.

The version of the tools I am using are

$ conftest --version
Conftest: 0.57.0
OPA: 1.1.0

$ opa version
Version: 1.1.0
Build Commit: de28510b979a0fc4a40fd8d1170671511f53e21b
Build Timestamp: 2025-01-27T15:44:49Z
Build Hostname:
Go Version: go1.23.5
Platform: darwin/arm64
Rego Version: v1
WebAssembly: unavailable

I'd be glad to help with this task.