Skip to content
64 changes: 64 additions & 0 deletions docs/writing-easyconfig-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -940,6 +940,70 @@ Example:
eb WRF-3.5.1-ictce-5.3.0-dmpar.eb --try-toolchain=intel,2014b -r
```

## Updating existing easyconfigs for another toolchain

While `try-toolchain` is a convenient way of testing existing software versions with another toolchain
you rarely want this behavior.
Usually when using a newer toolchain you also want to use newer versions of (at least) the dependencies.
Often there are already at least easyconfigs available for the dependencies in the new toolchain,
and especially if those are already installed you should use those as dependencies for the updated easyconfig.

So the process of updating an easyconfig to a newer toolchain version could look like:

1. Copy the easyconfig and change the toolchain version
1. Usually, especially when updating to the latest toolchain,
you should search for the most recent version of the software too and use that.
1. For each listed dependency find the easyconfig for the new toolchain version.
Here you need to take [toolchain hierarchy](common-toolchains.md#toolchains_diagram) into account.
I.e. a dependency of an easyconfig for the `foss` toolchain might use the `GCC` or `GCCcore` toolchain.
Check the toolchain easyconfig file (e.g. `foss-2023b.eb`) to find the versions of the sub-toolchain.
For very recent toolchains you can also check the `develop` branch of the [easyconfigs git repository](https://github.com/easybuilders/easybuild-easyconfigs) and open PRs.
1. If an easyconfig of the dependency for the new toolchain already exists use its version in your new easyconfig.
Otherwise, you need to create a new easyconfig for the new toolchain version based on that of another toolchain version.
The steps are the same and need to also be repeated for each dependency of this dependency.
1. Finally, build and test your easyconfig and consider [contributing](contributing.md#contributing-easyconfig-files--contributing_easyconfigs) it.

Especially when creating easyconfigs for dependencies it is a good idea to put all of them into a new folder.
To allow EasyBuild to find those, pass its path via `easybuild --robot <path>`.

This manual process can be partially automated:

1. Create a new folder for the new easyconfig and its dependencies.
1. Run `easybuild <current_easyconfig.eb> --try-toolchain-version=<new-version> --try-update-deps --copy-ec <folder>`.
This will update the toolchain version, search for the versions of dependencies in the new toolchain and use them if available.
If a dependency for then new toolchain is not available this will create a new easyconfig for this toolchain
based on an existing one for another toolchain (version).
All newly create easyconfigs will be copied to the specified folder.
1. Optionally, but recommended, update the version of the software for each created easyconfig to the latest available one.
1. **Important**: Verify the newly created easyconfigs by comparing each of them against the existing one for that software.
Only the version of the software, toolchain and dependencies should have been changed.
EasyBuild often replaces local variables or templates by their value when updating easyconfigs.
It might be easier to use those easyconfigs just for getting the versions of each dependency required
and still copy & update the easyconfigs from existing ones manually.

When updating a software version the patches in the easyconfig need to be checked too.
If they don't apply they might not be required anymore, when the upstream software fixed the issue, and can be removed.
Otherwise, they need to be adjusted for the changes in the source code.

For updates to easyconfigs containing Python packages (`PythonBundle`) you should check each of them if there is a newer version and if it is still required.
The latest version of a single Python package can usually be found on [PyPI](https://pypi.org).
To find the list of Python dependencies for another Python package to be installed by an easyconfig you can use the `findPythonDeps` script
distributed with EasyBuild.
This will also output the latest, compatible version of the found packages.
A good approach is to start from an easyconfig just containing the dependency on Python but no Python packages and then run the `findPythonDeps`.
For every Python package displayed by the script first search for a suitable easyconfig containing that package and add it as a dependency one by one.
That might require updating it from another toolchain.
Then run the script again to honor Python packages from that dependency and its dependencies.
Repeat until none of the remaining packages displayed are in any easyconfig and/or unique enough to the specific software that an own easyconfig would be useful.
Those can then be added to the `ext_list` of the easyconfig.

Also keep in mind that changing the version of the software and/or the toolchain might reveal new bugs or incompatibilities.
Some can be fixed or worked around by patches, often even with help from the maintainers of the software.
But some software is simply not compatible with specific versions of compilers or other software.
Feel free to ask for help in the [EasyBuild Slack channel](https://easybuild.io/join-slack).
Also filing an issue or bug report against the software in question helps in getting more information or the bug being fixed in later versions.
This fix can then also be applied to the earlier version of necessary.

## Dynamic values for easyconfig parameters {: #easyconfig_param_templates }

String templates are completed using the value of particular easyconfig
Expand Down
Loading