rich not handling brackets properly, and crashes EasyBuild

[sassy-crick]

Whilst test-build for PR # 23252 I encountered the following issue, and as discussed in the conference call then, that is the problem I see:

$ eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
accept-eula-for       (F) = Intel-oneAPI, NVHPC, CUDA
buildpath             (F) = /dev/shm/easybuild
containerpath         (F) = /home/sassy/apps/containers
detect-loaded-modules (F) = error
filter-deps           (F) = binutils
git-working-dirs-path (F) = /home/sassy/git
github-user           (F) = sassy-crick
installpath           (F) = /home/sassy/apps
packagepath           (F) = /home/sassy/apps/packages
parallel              (F) = 4
prefix                (F) = /home/sassy/apps
repositorypath        (F) = /home/sassy/apps/ebfiles_repo
robot-paths           (D) = /home/sassy/apps/software/EasyBuild/5.1.0/local/easybuild/easyconfigs
rpath                 (D) = True
sourcepath            (F) = /opt/sources
sourcepath-data       (F) = /home/sassy/apps/sources
tmp-logdir            (F) = /dev/shm/easybuild-log
tmpdir                (F) = /dev/shm/easybuild

And when I run the installation it crashes:

  >> running shell command:
        ./configure --prefix=/home/sassy/apps/software/PETSc/3.23.4-foss-2024a --LIBS="$LIBS -lrt" --with-mpi4py=0  --with-cc="mpicc"  --with-cxx="mpicxx" --with-c++-support  
--with-fc="mpifort"  --CFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC"  --CXXFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC 
-DOMPI_SKIP_MPICXX -DMPICH_SKIP_MPICXX"  --FFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC"  --with-mpi=1  --with-build-step-np=1  --with-shared-libraries=1  
--with-debugging=0  --with-pic=1  --with-x=0 --with-windows-graphics=0  --with-numpy=1  --with-fftw=1  
--with-fftw-lib=[/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2024a/lib/libfftw3_mpi.a,libfftw3.a]  --with-fftw-include=/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2024a/include  
--with-scalapack=1  --with-scalapack-lib=[/home/sassy/apps/software/ScaLAPACK/2.2.0-gompi-2024a-fb/lib/libscalapack.a,libflexiblas.a,libgfortran.a]  
--with-blas-lapack-lib=[/home/sassy/apps/software/FlexiBLAS/3.4.4-GCC-13.3.0/lib/libflexiblas.a,libgfortran.a]  --with-ptscotch=1 
--with-ptscotch-include=[/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/include] 
--with-ptscotch-lib=[/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptesmumps.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptscotchparmetisv3.a,/home/sassy/apps/so
ftware/SCOTCH/7.0.6-gompi-2024a/lib/libptscotch.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptscotcherr.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libesmumps.a
,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libscotch.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libscotcherr.a]  --with-suitesparse=1 
--with-suitesparse-include=[/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/include,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/include/suitesparse] 
--with-suitesparse-lib=[/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libumfpack.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libklu.so,/home/sassy/apps/soft
ware/SuiteSparse/7.10.1-foss-2024a/lib/libspqr.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcholmod.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libbt
f.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libccolamd.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcolamd.so,/home/sassy/apps/software/SuiteSparse
/7.10.1-foss-2024a/lib/libcxsparse.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libldl.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/librbio.so,/home/sass
y/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libspex.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcamd.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/l
ib/libamd.so]
        [started at: 2025-07-02 16:50:08]
        [working dir: /dev/shm/easybuild/PETSc/3.23.4/foss-2024a/petsc-3.23.4]
        [output and state saved to /dev/shm/easybuild/eb-yhi38q8h/run-shell-cmd-output/configure-togem0bj]
== ... (took < 1 sec)
== Summary:
   * [FAILED]  PETSc/3.23.4-foss-2024a

EasyBuild crashed! Please consider reporting a bug, this should not happen...

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/main.py", line 829, in <module>
    main_with_hooks()
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/main.py", line 815, in main_with_hooks
    main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/main.py", line 770, in main
    do_cleanup = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/main.py", line 592, in process_eb_args
    ecs_with_res = build_and_install_software(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/main.py", line 206, in build_and_install_software
    raise error
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/main.py", line 160, in build_and_install_software
    (ec_res['success'], app_log, err_msg, err_code) = build_and_install_one(ec, init_env)
                                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/framework/easyblock.py", line 5025, in build_and_install_one
    result = app.run_all_steps(run_test_cases=run_test_cases)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/framework/easyblock.py", line 4847, in run_all_steps
    err.print()
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/tools/run.py", line 157, in print
    print_error('\n'.join(error_info), rich_highlight=False)
  File "/home/sassy/apps/software/EasyBuild/5.1.0/lib/python3.11/site-packages/easybuild/tools/output.py", line 398, in print_error
    console.print('\n\n' + error_msg + '\n', highlight=rich_highlight)
  File "/usr/lib/python3/dist-packages/rich/console.py", line 1668, in print
    renderables = self._collect_renderables(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/rich/console.py", line 1532, in _collect_renderables
    self.render_str(
  File "/usr/lib/python3/dist-packages/rich/console.py", line 1424, in render_str
    rich_text = render_markup(
                ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/rich/markup.py", line 161, in render
    raise MarkupError(
rich.errors.MarkupError: closing tag '[/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2024a/lib/libfftw3_mpi.a,libfftw3.a]' at position 733 doesn't match any open tag

It appears to be a problem with rich but that is the first time I see it.

Some more information:

$ python3 --version
Python 3.11.2
Debian GNU/Linux 12 (bookworm)

Note: accidentally opened here 23578

[sassy-crick]

Update: the problem also persists when python3-rich has been removed. I am wondering if the [ ] in the configuration, i.e.:

        ./configure --prefix=/home/sassy/apps/software/PETSc/3.22.5-foss-2023b-MUMPS --LIBS="$LIBS -lrt" --with-mpi4py=0 --with-mumps=1  --with-cc="mpicc"  --with-cxx="mpicxx" --with-c++-support  --with-fc="mpifort"  --CFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC"  --CXXFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC -DOMPI_SKIP_MPICXX -DMPICH_SKIP_MPICXX"  --FFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC"  --with-mpi=1  --with-build-step-np=8  --with-shared-libraries=1  --with-debugging=0  --with-pic=1  --with-x=0 --with-windows-graphics=0  --with-numpy=1  --with-fftw=1  --with-fftw-lib=[/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2023b/lib/libfftw3_mpi.a,libfftw3.a]  --with-fftw-include=/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2023b/include  --with-scalapack=1  --with-scalapack-lib=[/home/sassy/apps/software/ScaLAPACK/2.2.0-gompi-2023b-fb/lib/libscalapack.a,libflexiblas.a,libgfortran.a]  --with-blas-lapack-lib=[/home/sassy/apps/software/FlexiBLAS/3.3.1-GCC-13.2.0/lib/libflexiblas.a,libgfortran.a]  --with-ptscotch=1 --with-ptscotch-include=[/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/include] --with-ptscotch-lib=[/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libptesmumps.a,/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libptscotchparmetisv3.a,/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libptscotch.a,/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libptscotcherr.a,/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libesmumps.a,/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libscotch.a,/home/sassy/apps/software/SCOTCH/7.0.4-gompi-2023b/lib/libscotcherr.a]  --with-suitesparse=1 --with-suitesparse-include=[/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/include,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/include/suitesparse] --with-suitesparse-lib=[/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libumfpack.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libklu.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libspqr.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libcholmod.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libbtf.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libccolamd.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libcolamd.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libcxsparse.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libldl.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/librbio.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libspex.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libcamd.so,/home/sassy/apps/software/SuiteSparse/7.7.0-foss-2023b/lib/libamd.so]

cause the problem in Debian, as it seems to work within a Rocky-8.9 container. That would then be awkward.

[boegel]

@sassy-crick Surely the traceback must be different when python3-rich is not installed?

Can you share more details on how it crashes then?

[sassy-crick]

Sorry for the delay.
Without rich I get that error message:

eb-shell> ./configure --prefix=/home/sassy/apps/software/PETSc/3.23.5-foss-2024a --LIBS="$LIBS -lrt" --with-mpi4py=0  --with-cc="mpicc"  --with-cxx="mpicxx" --with-c++-support  --with-fc="mpifort"  --COPTFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC"  --CXXOPTFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC -DOMPI_SKIP_MPICXX -DMPICH_SKIP_MPICXX"  --FOPTFLAGS="-O2 -ftree-vectorize -march=native -fno-math-errno -fopenmp -fPIC"  --with-mpi=1  --with-build-step-np=8  --with-shared-libraries=1  --with-debugging=0  --with-pic=1  --with-x=0 --with-windows-graphics=0  --with-numpy=1  --with-fftw=1  --with-fftw-lib=[/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2024a/lib/libfftw3_mpi.a,libfftw3.a]  --with-fftw-include=/home/sassy/apps/software/FFTW.MPI/3.3.10-gompi-2024a/include  --with-scalapack=1  --with-scalapack-lib=[/home/sassy/apps/software/ScaLAPACK/2.2.0-gompi-2024a-fb/lib/libscalapack.a,libflexiblas.a,libgfortran.a]  --with-blas-lapack-lib=[/home/sassy/apps/software/FlexiBLAS/3.4.4-GCC-13.3.0/lib/libflexiblas.a,libgfortran.a]  --with-python=1 --with-python-dir=/home/sassy/apps/software/Python/3.12.3-GCCcore-13.3.0  --with-boost=1 --with-boost-dir=/home/sassy/apps/software/Boost/1.85.0-GCC-13.3.0  --with-metis=1 --with-metis-dir=/home/sassy/apps/software/METIS/5.1.0-GCCcore-13.3.0  --with-mumps=1 --with-mumps-dir=/home/sassy/apps/software/MUMPS/5.7.2-foss-2024a-metis  --with-hypre=1 --with-hypre-dir=/home/sassy/apps/software/Hypre/2.32.0-foss-2024a  --with-parmetis=1 --with-parmetis-dir=/home/sassy/apps/software/ParMETIS/4.0.3-gompi-2024a  --with-superlu_dist=1 --with-superlu_dist-dir=/home/sassy/apps/software/SuperLU_DIST/9.1.0-foss-2024a  --with-ptscotch=1 --with-ptscotch-include=[/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/include] --with-ptscotch-lib=[/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptesmumps.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptscotchparmetisv3.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptscotch.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libptscotcherr.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libesmumps.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libscotch.a,/home/sassy/apps/software/SCOTCH/7.0.6-gompi-2024a/lib/libscotcherr.a]  --with-suitesparse=1 --with-suitesparse-include=[/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/include,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/include/suitesparse] --with-suitesparse-lib=[/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libumfpack.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libklu.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libspqr.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcholmod.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libbtf.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libccolamd.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcolamd.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcxsparse.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libldl.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/librbio.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libspex.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libcamd.so,/home/sassy/apps/software/SuiteSparse/7.10.1-foss-2024a/lib/libamd.so]
Segmentation fault (core dumped)

So no fun here either.
As I needed PETSc-3.23.5-foss-2024a.eb I tried our alma8.9 container which got rich installed, used EasyBuild-5.1.2 from my home-directory, same problem as on Debian. So I tried rocky8.9 container, as above.

Alma-8.9:

Apptainer>  rpm -qa | grep -i python
python3-setuptools-wheel-39.2.0-7.el8.noarch
python3-libs-3.6.8-56.el8_9.3.alma.1.x86_64
platform-python-3.6.8-56.el8_9.3.alma.1.x86_64
python3-libcomps-0.1.18-1.el8.x86_64
python3-hawkey-0.63.0-17.el8_9.alma.x86_64
python3-dnf-4.7.0-19.el8.alma.noarch
python3-six-1.11.0-8.el8.noarch
python3-dnf-plugins-core-4.0.21-25.el8.noarch
platform-python-pip-9.0.3-24.el8.noarch
python3-setuptools-39.2.0-7.el8.noarch
python36-3.6.8-39.module_el8.10.0+3769+3838165b.x86_64
python3-pip-wheel-9.0.3-23.el8_9.1.noarch
platform-python-setuptools-39.2.0-7.el8.noarch
python3-libdnf-0.63.0-17.el8_9.alma.x86_64
python3-gpg-1.13.1-11.el8.x86_64
python3-rpm-4.14.3-28.el8_9.x86_64
python3-systemd-234-8.el8.x86_64
python3-dateutil-2.6.1-6.el8.noarch
python3-dbus-1.2.4-15.el8.x86_64
python3-pip-9.0.3-24.el8.noarch

Rocky-8.9:

Apptainer> python3 --version
Python 3.6.8
Apptainer> rpm -qa | grep -i python
python3-libdnf-0.63.0-21.el8_10.x86_64
python3-rpm-4.14.3-32.el8_10.x86_64
python3-systemd-234-8.el8.x86_64
python3-dateutil-2.6.1-6.el8.noarch
python3-dbus-1.2.4-15.el8.x86_64
python3-setuptools-39.2.0-8.el8_10.noarch
python3-pip-9.0.3-24.el8.rocky.0.noarch
python3-pip-wheel-9.0.3-24.el8.rocky.0.noarch
platform-python-setuptools-39.2.0-8.el8_10.noarch
python3-libs-3.6.8-69.el8_10.rocky.0.x86_64
python3-gpg-1.13.1-12.el8.x86_64
python3-hawkey-0.63.0-21.el8_10.x86_64
python3-unbound-1.16.2-5.8.el8_10.x86_64
python3-dnf-4.7.0-21.el8_10.noarch
python3-six-1.11.0-8.el8.noarch
python3-dnf-plugins-core-4.0.21-25.el8.noarch
python36-3.6.8-39.module+el8.10.0+1910+234ad790.x86_64
python3-libcomps-0.1.18-1.el8.x86_64
python3-setuptools-wheel-39.2.0-8.el8_10.noarch
platform-python-pip-9.0.3-24.el8.rocky.0.noarch
platform-python-3.6.8-69.el8_10.rocky.0.x86_64

However, it seems to work in the debian-bookworm container, which got python-3.11.2 installed as a minimal installation, as there is no rich et. al:

Apptainer> dpkg -l *python3* | grep ii
ii  libpython3-dev:amd64        3.11.2-1+b1      amd64        header files and a static library for Python (default)
ii  libpython3-stdlib:amd64     3.11.2-1+b1      amd64        interactive high-level object-oriented language (default python3 version)
ii  libpython3.11:amd64         3.11.2-6+deb12u6 amd64        Shared Python runtime library (version 3.11)
ii  libpython3.11-dev:amd64     3.11.2-6+deb12u6 amd64        Header files and a static library for Python (v3.11)
ii  libpython3.11-minimal:amd64 3.11.2-6+deb12u6 amd64        Minimal subset of the Python language (version 3.11)
ii  libpython3.11-stdlib:amd64  3.11.2-6+deb12u6 amd64        Interactive high-level object-oriented language (standard library, version 3.11)
ii  python3                     3.11.2-1+b1      amd64        interactive high-level object-oriented language (default python3 version)
ii  python3-dev                 3.11.2-1+b1      amd64        header files and a static library for Python (default)
ii  python3-distutils           3.11.2-3         all          distutils package for Python 3.x
ii  python3-lib2to3             3.11.2-3         all          Interactive high-level object-oriented language (lib2to3)
ii  python3-minimal             3.11.2-1+b1      amd64        minimal subset of the Python language (default python3 version)
ii  python3-pip                 23.0.1+dfsg-1    all          Python package installer
ii  python3-pkg-resources       66.1.1-1+deb12u1 all          Package Discovery and Resource Access using pkg_resources
ii  python3-setuptools          66.1.1-1+deb12u1 all          Python3 Distutils Enhancements
ii  python3-wheel               0.38.4-2         all          built-package format for Python
ii  python3.11                  3.11.2-6+deb12u6 amd64        Interactive high-level object-oriented language (version 3.11)
ii  python3.11-dev              3.11.2-6+deb12u6 amd64        Header files and a static library for Python (v3.11)
ii  python3.11-minimal          3.11.2-6+deb12u6 amd64        Minimal subset of the Python language (version 3.11)

I double checked, and there are no packages installed in ~/.local/lib as well.

Happy to share the apptainer definition files if that helps.