[v9.0.1 -- PHP Flavor] Cannot override `PHP_*_CLI_EXECUTABLE` correctly

[llaville]

Describe the bug
Cannot override CLI executable with a local copy of any of PHP Linters

Don't be afraid of this long report, I'll provide a PR to fix the problem soon. TL;DR only if you want to know more about such kind of issue.

To Reproduce
Steps to reproduce the behavior:

1. Retrieve a repository that may install PHP Linters as dev tools locally; For example https://github.com/llaville/php-compatinfo-db (master branch)
2. Run composer install to install standard project dependencies
3. Run composer bin all install to install all other dev tools locally on this project
4. Or just install PHPStan for demo purpose (to issue concept), That's mean run : composer bin phpstan install
5. Run the PHP Flavor with official Docker image : docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:rw -v $(pwd):/tmp/lint:rw -e LOG_LEVEL=debug -e PARALLEL=false ghcr.io/oxsecurity/megalinter-php:v9.0.1

NB: I've enabled debug mode to see additional message that will help to show problems found, and disabled parallel mode to have log grouped by linters.

Expected behavior

All linters should run without found any errors, especially PHPStan if we run local version installed (with command composer bin phpstan install)

Something like this :

Additional context
PHPStan installed with MegaLinter (9.0.1) is detailed at https://github.com/oxsecurity/megalinter/blob/v9.0.1/megalinter/descriptors/php.megalinter-descriptor.yml#L130-L140

On my project to have none errors found by MegaLinter, I should have an extra PHPStan extension, see https://github.com/llaville/php-compatinfo-db/blob/master/vendor-bin/phpstan/composer.json#L5

So, on my first approach, I though I could override CLI executable in MegaLinter config file like this
PHP_PHPSTAN_CLI_EXECUTABLE: "vendor/bin/phpstan" , see llaville/php-compatinfo-db@4b0a09d

But PATH defined does not lookup into /tmp/lint volume (mounted on docker run), and more than that explains are given by doc itself at https://docs.python.org/3/library/shutil.html#shutil.which

If cmd contains a directory component, which() only checks the specified path directly and does not search the directories listed in path or in the system’s PATH environment variable.

So I got this output

❌ Linted [PHP] files with [phpstan]: Found 1 error(s) and 0 warning(s) - (0.0s)
- Using [phpstan vERROR] https://megalinter.io/9.0.1/descriptors/php_phpstan
- MegaLinter key: [PHP_PHPSTAN]
- Rules config: [/.github/linters/phpstan.neon.dist]
- Command: [vendor/bin/phpstan analyse --no-progress --no-ansi --memory-limit 1G -c /tmp/lint/.github/linters/phpstan.neon.dist]
--Error detail:
Fatal error while calling phpstan: [Errno 2] No such file or directory: 'vendor/bin/phpstan'

The solution is given by this little patch

diff --git a/megalinter/Linter.py b/megalinter/Linter.py
index 0cfcf481d8..162bec5728 100644
--- a/megalinter/Linter.py
+++ b/megalinter/Linter.py
@@ -974,7 +974,7 @@ class Linter:
             try:
                 reporter.produce_report()
             except Exception as e:
-                logging.error("Unable to process reporter " + reporter.name + str(e))
+                logging.error("Unable to process reporter " + reporter.name + " " + str(e))

         return self

@@ -1657,6 +1657,8 @@ class Linter:
         cli_absolute = shutil.which(cmd[0])
         if cli_absolute is not None:
             cmd[0] = cli_absolute
+        cmd_resolved = " ".join(str(element) for element in cmd)
+        logging.debug("Linter version command resolved: " + cmd_resolved)
         cmd += self.cli_version_extra_args
         if self.cli_version_arg_name != "":
             cmd += [self.cli_version_arg_name]

First logging error to avoid collapsed message like Unable to process reporter CONSOLEcan only concatenate str (not "list") to str (space missing between reporter name CONSOLE and exception message

Second logging to debug how linter is resolved by shutil.which

BUT to solve my situation context, we should only fix the PATH definition to allow local linter first

diff --git a/megalinter/descriptors/php.megalinter-descriptor.yml b/megalinter/descriptors/php.megalinter-descriptor.yml
index 2fcf4fb5bb..b327a684ac 100644
--- a/megalinter/descriptors/php.megalinter-descriptor.yml
+++ b/megalinter/descriptors/php.megalinter-descriptor.yml
@@ -25,7 +25,7 @@ install:
   dockerfile:
     - RUN update-alternatives --install /usr/bin/php php /usr/bin/php84 110
     - COPY --from=composer/composer:2-bin /composer /usr/bin/composer
-    - ENV PATH="/root/.composer/vendor/bin:${PATH}"
+    - ENV PATH="/tmp/lint/vendor/bin:/root/.composer/vendor/bin:${PATH}"
 linters:
   # PHPCS
   - linter_name: phpcs

[llaville]

Actually we have binary path resolved not displayed in console text reporter. It's only shown into log file when LEVEL=debug

[phpstan] command: ['phpstan', 'analyse', '--no-progress', '--no-ansi', '--memory-limit', '1G', '-c', '/tmp/lint/.github/linters/phpstan.neon.dist']
[phpstan] CWD: /tmp/lint
[phpstan] result: 0
Linter version command: ['/root/.composer/vendor/bin/phpstan', '--version']
Linter version result: 0 PHPStan - PHP Static Analysis Tool 2.1.28


[Text Reporter] Generated TEXT report: /tmp/lint/megalinter-reports/linters_logs/SUCCESS-PHP_PHPSTAN.log
✅ Linted [PHP] files with [phpstan] successfully - (14.37s)
- Using [phpstan v2.1.28] https://megalinter.io/9.0.1/descriptors/php_phpstan
- MegaLinter key: [PHP_PHPSTAN]
- Rules config: [/.github/linters/phpstan.neon.dist]
- Command: [phpstan analyse --no-progress --no-ansi --memory-limit 1G -c /tmp/lint/.github/linters/phpstan.neon.dist]

Like in my previous screenshot, it will be cool to have a full path to linter binary resolved (like we can see with config file used)

That will help us to see what copy , local or embedded ML version are used.

WDYT ?

[llaville]

Alternative to the patch/PR is to run locally with docker the following command (overriding the PHP_PHPSTAN_CLI_EXECUTABLE configuration setting with a full path inside Docker Volume)

For example :

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:rw -v $(pwd):/tmp/lint:rw -e LOGL_LEVEL=true -e PARALLEL=false -e PHP_PHPSTAN_CLI_EXECUTABLE=/tmp/lint/vendor/bin/phpstan ghcr.io/oxsecurity/megalinter-php:v9.0.1

And we can get (on my project context with a local PHPStan version 2.1.29)

[llaville]

Or if you prefer to use only configuration files, we can adopt such EXTENDS capability facility

.mega-linter.yml config file

EXTENDS:
    - "https://raw.githubusercontent.com/llaville/.github/refs/heads/master/.mega-linter-base.yml"
    - .mega-linter-base.yml
PHP_PHPSTAN_CLI_EXECUTABLE: "/tmp/lint/vendor/bin/phpstan" # force to use PHPStan locally version when additional extensions are required
PHP_PHPSTAN_DISABLE_ERRORS: false # project is now compatible PHPStan 2.x

.mega-linter-base.yml config file

ENABLE:
    - ACTION
    - EDITORCONFIG
    - JSON
    - MARKDOWN
    - PHP
    - YAML
DISABLE_LINTERS:
    - JSON_PRETTIER
    - JSON_V8R
    - MARKDOWN_MARKDOWN_TABLE_FORMATTER
    - MARKDOWN_MARKDOWN_LINK_CHECK
    - PHP_BUILTIN
    - PHP_PSALM
    - PHP_PHPCSFIXER
    - YAML_V8R
PRINT_ALL_FILES: false

And of course https://raw.githubusercontent.com/llaville/.github/refs/heads/master/.mega-linter-base.yml for all other base settings on a remote file

[echoix]

Go ahead with the logging changes, indeed it's useful to better understand what's going on!

[llaville]

Closing this issue for reason given in associated PR