High-level roadmap for containers

[daler]

Context: to get ARM containers working uniformly, we wanted to have all images built in the same repo (rather than split across multiple) to get everything pushing via manifests to quay.io. That meant moving the build-env (bioconda-utils-build-env-cos7) creation over here from bioconda-utils.

This is largely working, but we need to think about how versions are handled.

We have the build-env container, which has bioconda-utils and conda-build in it. This is what was moved here from bioconda-utils.

We have the create-env container, which only has conda in it (so as to avoid having dependencies undeclared by a recipe which helps avoid broken packages). But the version of conda used is the exact one that comes in from the build-env container -- at build time, pulls the conda version directly from running conda list in the build-env container.

This means that both containers are very tightly-coupled to bioconda-utils. One one hand it makes a lot of sense to keep them over there, because they can be built at release time, using the version number of the release and using the bioconda-utils github actions. On the other hand, it makes sense to keep them here so that all containers are built uniformly and can be maintained uniformly.

In fact, only after moving it over here was I finally able to make sense of everything. While I hope not everyone is as slow as me, anything that simplifies infrastructure or makes it more obvious or easier to understand has a lot of value. So I am leaning towards keeping all containers here.

However, doing so creates a problem. The build-env and create-env in this repo must be triggered upon release in another repo (bioconda-utils). We cannot do this just by inspecting bioconda-common/common.sh, since bioconda-recipes also looks at that. Which means that after bumping common.sh, bioconda-recipes PRs would be broken until we build the new version of the build and create containers over here. Managing that time lag (and managing the restarts of broken PRs and replying to issues asking about why its temporarily broken) seems brittle.

So I believe the workflow needs to be:

• new bioconda-utils release (from merging the releaseplease PR)
• triggers a build of the build-env in this repo
• when that completes, then build the create-env image here (it depends on the build-env since it needs to extract conda version)
• both are uploaded to quay.io with a tag for the bioconda-utils version

(Note that the base images can continue to use manual major.minor versioning. Those are easier to handle.)

Then, common.sh is manually updated. Ideally, it should be updated via PR, and the PR tests check whether the containers for the bumped bioconda-utils version exist yet in quay.io.

After that, bioconda-recipes will pick up the new version specified in common.sh.

Furthermore, we should ensure that recipes is always using the right container for building (ensure we specify the tag and provide that to bioconda-utils and mulled).

[martin-g]

Here is a possible way to accomplish this multi-repository workflow:

1. a new step will be added to the bioconda-utils workflow: when the even is push and the branch is master/main then it will trigger a repository_dispatch event to the bioconda-containers repo
2. a new workflow in the -containers repo will react on repository_dispatch event, will get the latest version of bioconda-utils(either as a parameter to repository_dispatch (if possible) or will get it from common.sh) and pass it asinputsto [workflow_dispatch](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch) that will trigger firstbuild-envand thencreate-env`

How does this sound ?
I will try to setup a simple demo of this!

[daler]

Yep, that's close to what I was thinking.

However, if repository_dispatch doesn't allow parameters, we shouldn't get the version from common.sh (since that would introduce a race condition with bioconda-recipes, which also reads common.sh, but the build-env container wouldn't be available yet).

Instead, a workflow here in this repo should inspect the bioconda-utils releases and get the latest one to build.

[daler]

Looks like this will do the trick for getting latest release:

curl -L -H "Accept: application/vnd.github+json" \
  https://api.github.com/repos/bioconda/bioconda-utils/releases \
  | jq --raw-output '.[].tag_name' | head -n1

[daler]

We discussed this a lot at yesterday's bioconda core meeting. Consolidating everything is very useful and important for long-term maintenance. However only after refactoring everything here in #75 did I realize it will be challenging to wire up the release trigger from over in bioconda-utils using the reusable workflows.

Also, in the end, the reusable workflow in #75 ends up looking a lot like just a bash script, and a bash script is waaaay easier to locally debug (case in point: all those "bump" pushes in #75...).

So the plan we agreed on during yesterday's meeting is to convert the reusable workflow in this PR into a bash script, and then move everything here (dockerfiles and GHA yamls) over to bioconda-utils.

That is, instead of bringing the bioconda-utils-build-env-cos7 over here to unify everything, we'll move everything here over there. Then we can archive this repo which will get rid of another moving part from the build infrastructure.

We agreed that bash script + everything in bioconda-utils is the simplest way to set things up for long-term maintenance and -- by not requiring knowledge of reusable workflows, familiarity with multiple repos, and syntax minutiae of GH Actions -should reduce cognitive overhead as much as possible for contributors.

[daler]

This is now implemented in bioconda/bioconda-utils#953, see that PR for details.