Skip to content

Commit c992fc2

Browse files
committed
Some improvments to the module naming schemes section based on the ISC'22 tutorial.
1 parent 920b853 commit c992fc2

File tree

1 file changed

+47
-43
lines changed

1 file changed

+47
-43
lines changed

docs/2022-CSC_and_LO/3_Advanced/3_04_module_naming_schemes.md

Lines changed: 47 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ This typical Lmod hierarcny would map very well on the EasyBuild common toolchai
5555

5656
In fact, for the example
5757

58-
<div align="center"><img src="../../../img/hmns.png" width="60%"/></div>
58+
<div align="center"><img src="../../../img/hmns-2021b.png" width="60%"/></div>
5959

6060
software at the ``Core`` level would be installed with the ``GCCcore`` and ``SYSTEM``
6161
toolchains. Software at the ``Compiler`` level would be installed with the ``GCC``
@@ -102,10 +102,10 @@ However, the are some minor disadvantages too:
102102

103103
When using a flat module naming scheme, module names can be fairly long and perhaps confusing. For a `HDF5` installation
104104
with the EasyBuild common toolchains for example,
105-
one might have `HDF5/1.10.6-gompi-2020a` as module name. The `-gompi-2020a` part of the name refers to the toolchain that was
105+
one might have `HDF5/1.12.1-gompi-2021b` as module name. The `-gompi-2021b` part of the name refers to the toolchain that was
106106
used for this installation, but it may be confusing to some people (what kind of Pokémon is a "gompi"?!).
107107

108-
In the example module hierarchy shown above, the module for `HDF5` could simply be named `HDF5/1.10.6` which is basically the bare
108+
In the example module hierarchy shown above, the module for `HDF5` could simply be named `HDF5/1.12.1` which is basically the bare
109109
essentials: software name and version. That's way better, nice and clean!
110110

111111
#### Amount of available modules
@@ -329,9 +329,10 @@ install EasyBuild again in our hierarchical module tree before we can continue.
329329
**We strongly recommend using an EasyBuild installation that was [installed via "`pip install`"
330330
or "`pip3 install`"](../../1_Intro/1_06_installation#method-1-using-pip) in this part of the tutorial.**
331331

332-
An easy way to do this is in the prepared environment is to run:
332+
An easy way to do this is in the prepared environment used for regular EasyBuild tutorials is to run:
333333

334334
```shell
335+
unset PIP_PREFIX
335336
pip3 install --user easybuild
336337
export PATH=$HOME/.local/bin:$PATH
337338
export EB_PYTHON=python3
@@ -393,7 +394,7 @@ but EasyBuild can do all the hard work for us.
393394

394395
The steps we will have to go through are:
395396

396-
* Tell EasyBuild we want to "install" the `HDF5-1.10.7-gompi-2020b.eb` easyconfig file;
397+
* Tell EasyBuild we want to "install" the `HDF5-1.12.1-gompi-2021b.eb` easyconfig file;
397398
* Enable dependency resolution via `--robot`;
398399
* Assuming the software would have been installed already with the default naming scheme
399400
in a different module directory, instruct EasyBuild to only generate the module files,
@@ -403,9 +404,9 @@ The steps we will have to go through are:
403404
These steps translate to this single `eb` command:
404405

405406
```
406-
$ eb HDF5-1.10.7-gompi-2020b.eb --robot --module-only
407+
$ eb HDF5-1.12.1-gompi-2021b.eb --robot --module-only
407408
...
408-
== building and installing MPI/GCC/10.2.0/OpenMPI/4.0.5/HDF5/1.10.7...
409+
== building and installing MPI/GCC/11.2.0/OpenMPI/4.1.1/HDF5/1.12.1...
409410
...
410411
== sanity checking...
411412
== cleaning up [skipped]
@@ -450,10 +451,10 @@ Let us see what that gives us in terms of available modules:
450451
```
451452
$ module avail
452453
453-
--------------------- /home/easybuild/hmns/modules/all/Core ---------------------
454-
Bison/3.5.3 GCCcore/10.2.0 flex/2.6.4 help2man/1.47.4
455-
Bison/3.7.1 (D) M4/1.4.18 gettext/0.21 ncurses/6.2
456-
GCC/10.2.0 binutils/2.35 gompi/2020b zlib/1.2.11
454+
------------------------ /home/easybuild/hmns/modules/all/Core -------------------------
455+
binutils/2.37 GCC/11.2.0 gompi/2021b OpenSSL/1.1
456+
Bison/3.8.2 GCCcore/11.2.0 M4/1.4.19 pkg-config/0.29.2
457+
flex/2.6.4 gettext/0.21 ncurses/6.2 zlib/1.2.11
457458
```
458459

459460

@@ -475,9 +476,9 @@ $ module spider HDF5
475476
...
476477
477478
You will need to load all module(s) on any one of the lines below
478-
before the "HDF5/1.10.7" module is available to load.
479+
before the "HDF5/1.12.1" module is available to load.
479480
480-
GCC/10.2.0 OpenMPI/4.0.5
481+
GCC/11.2.0 OpenMPI/4.1.1
481482
```
482483

483484
This tells us we need to load two gateway modules before we can load the module
@@ -486,70 +487,73 @@ for HDF5.
486487
Let us start with loading the `GCC` compiler module:
487488

488489
```
489-
module load GCC/10.2.0
490+
module load GCC/11.2.0
490491
```
491492

492493
And then check again which modules are available:
493494

494495
```
495496
$ module avail
496497
497-
-------------- /home/easybuild/hmns/modules/all/Compiler/GCC/10.2.0 --------------
498-
OpenMPI/4.0.5
498+
--------------------------- /home/easybuild/hmns/modules/all/Compiler/GCC/11.2.0 ---------------------------
499+
OpenMPI/4.1.1
499500
500-
------------ /home/easybuild/hmns/modules/all/Compiler/GCCcore/10.2.0 ------------
501-
Autoconf/2.69 UCX/1.9.0 libpciaccess/0.16
502-
...
503-
Szip/2.1.1 libfabric/1.11.0 zlib/1.2.11 (L,D)
504-
Autoconf/2.69 XZ/5.2.5 libtool/2.4.6
501+
------------------------- /home/easybuild/hmns/modules/all/Compiler/GCCcore/11.2.0 -------------------------
502+
Autoconf/2.71 flex/2.6.4 (D) libreadline/8.1 pkg-config/0.29.2 (D)
503+
Automake/1.16.4 groff/1.22.4 libtool/2.4.6 PMIx/4.1.0
504+
Autotools/20210726 help2man/1.48.3 libxml2/2.9.10 Szip/2.1.1
505+
binutils/2.37 (L,D) hwloc/2.5.0 M4/1.4.19 (D) UCX/1.11.2
506+
Bison/3.7.6 libevent/2.1.12 ncurses/6.2 (D) xorg-macros/1.19.3
507+
DB/18.1.40 libfabric/1.13.2 numactl/2.0.14 XZ/5.2.5
508+
expat/2.4.1 libpciaccess/0.16 Perl/5.34.0 zlib/1.2.11 (L,D)
505509
506-
--------------------- /home/easybuild/hmns/modules/all/Core ---------------------
507-
Bison/3.5.3 GCCcore/10.2.0 (L) flex/2.6.4 help2man/1.47.4
508-
Bison/3.7.1 (D) M4/1.4.18 gettext/0.21 ncurses/6.2
509-
GCC/10.2.0 (L) binutils/2.35 gompi/2020b zlib/1.2.11
510+
---------------------------------- /home/easybuild/hmns/modules/all/Core -----------------------------------
511+
binutils/2.37 GCC/11.2.0 (L) gompi/2021b OpenSSL/1.1
512+
Bison/3.8.2 (D) GCCcore/11.2.0 (L) M4/1.4.19 pkg-config/0.29.2
513+
flex/2.6.4 gettext/0.21 ncurses/6.2 zlib/1.2.11
510514
```
511515

512516
Good news, we now have additional modules available!
513517

514-
The compiler level of our hierarchy actually consists of two directories here: `Compiler/GCCcore/10.2.0`
515-
and `Compiler/GCC/10.2.0`. The modules in the `GCCcore` directory are ones we can use in other compiler
518+
The compiler level of our hierarchy actually consists of two directories here: `Compiler/GCCcore/11.2.0`
519+
and `Compiler/GCC/11.2.0`. The modules in the `GCCcore` directory are ones we can use in other compiler
516520
toolchains that use GCC 10.2.0 as a base compiler (the details of that are out of scope here).
517521

518-
The module we are interested in is `OpenMPI/4.0.5`, which is another gateway module.
522+
The module we are interested in is `OpenMPI/4.1.1`, which is another gateway module.
519523

520524
Remember that the "`module spider`" output told us that there does indeed exist a module for `HDF5`, but that
521-
we need to load *both* the `GCC/10.2.0` and `OpenMPI/4.0.5` modules first.
525+
we need to load *both* the `GCC/11.2.0` and `OpenMPI/4.1.1` modules first.
522526

523-
So, let us do exactly that (remember that `GCC/10.2.0` is already loaded):
527+
So, let us do exactly that (remember that `GCC/11.2.0` is already loaded):
524528

525529
```
526-
module load OpenMPI/4.0.5
530+
module load OpenMPI/4.1.1
527531
```
528532

529-
If you now check the output of "`module avail`" again, you should see the `HDF5/1.10.7` module:
533+
If you now check the output of "`module avail`" again, you should see the `HDF5/1.12.1` module:
530534

531535
```
532536
$ module avail
533537
534-
-------- /home/easybuild/hmns/modules/all/MPI/GCC/10.2.0/OpenMPI/4.0.5 -------
535-
HDF5/1.10.7
538+
-------- /home/easybuild/hmns/modules/all/MPI/GCC/11.2.0/OpenMPI/4.1.1 -------
539+
HDF5/1.12.1
536540
537-
------------ /home/easybuild/hmns/modules/all/Compiler/GCC/10.2.0 ------------
538-
OpenMPI/4.0.5 (L)
541+
------------ /home/easybuild/hmns/modules/all/Compiler/GCC/11.2.0 ------------
542+
OpenMPI/4.1.1 (L)
539543
540544
...
541545
```
542546

543-
To use HDF5, we need to load this `HDF5/1.10.7` module. We can verify that the installation works
547+
To use HDF5, we need to load this `HDF5/1.12.1` module. We can verify that the installation works
544548
using one of the commands provided by HDF5, `h5dump` for example:
545549

546550
```
547-
module load HDF5/1.10.7
551+
module load HDF5/1.12.1
548552
```
549553

550554
```
551555
$ h5dump --version
552-
h5dump: Version 1.10.7
556+
h5dump: Version 1.12.1
553557
```
554558

555559
If you now check which modules are loaded via "`module list`", you will notice that all module names
@@ -559,10 +563,10 @@ and nice and short now, which is one of the advantages of using a hierarchical m
559563
$ module list
560564

561565
Currently Loaded Modules:
562-
1) GCCcore/10.2.0 5) numactl/2.0.13 9) hwloc/2.2.0 13) PMIx/3.1.5
563-
2) zlib/1.2.11 6) XZ/5.2.5 10) libevent/2.1.12 14) OpenMPI/4.0.5
564-
3) binutils/2.35 7) libxml2/2.9.10 11) UCX/1.9.0 15) Szip/2.1.1
565-
4) GCC/10.2.0 8) libpciaccess/0.16 12) libfabric/1.11.0 16) HDF5/1.10.7
566+
1) GCCcore/11.2.0 5) numactl/2.0.14 9) hwloc/2.5.0 13) libfabric/1.13.2 17) HDF5/1.12.1
567+
2) zlib/1.2.11 6) XZ/5.2.5 10) OpenSSL/1.1 14) PMIx/4.1.0
568+
3) binutils/2.37 7) libxml2/2.9.10 11) libevent/2.1.12 15) OpenMPI/4.1.1
569+
4) GCC/11.2.0 8) libpciaccess/0.16 12) UCX/1.11.2 16) Szip/2.1.1
566570
```
567571

568572

0 commit comments

Comments
 (0)