What is the practical difference between -preview and -stub-run

[trym-b]

Hi

One of the issues we are facing is very long running workflows, which can run for hours or days. So when we are making changes to our nextflow workflow signatures, we would like to get more timely feedback from our automatic nextflow runs in our continuous integration system.

So we wondered if there is some kind of dry-run option for evaluating these signatures and call sites, and we found 2 options: nextflow run -preview and nextflow run -stub-run.

For the -preview flag, this is the description: Run the workflow script skipping the execution of all processes.
For the -stub-run flag, this is the description: Whether to replace scripts with command stubs when executing the run.

When running with either of these options, the total runtime of one of the very time-consuming workflows becomes 18 seconds. So in practice, what is the difference between these options? Will our millage vary if we go for -preview vs -stub-run?

[bentsherman]

It's difficult to develop a true "dry run" for Nextflow because you don't know from the beginning how many tasks will be executed. So instead there are multiple related modes

Preview mode isn't really a dry run, it's more a way to inspect / test the workflow DAG that is constructed at the beginning. It's a good way to test most of your pipeline code without actually running the pipeline, but I suspect it will be fully replaced in time by the inspect and lint commands

The stub run is sort of like a dry run and is probably what you should use to test as much of your pipeline code as possible. It allows you to stub the task script, so the stub run should include everything from a normal run except for the actual command-line tools.

I have also experimented with a so-called "dry run" in #4214 but that is mainly about testing the impact of a code change on a resumed run, i.e. if I make change X to my code how far will the pipeline get before re-executing. But I haven't merged this yet

[trym-b]

Thank you for answering my question :)

Right now we do not have any stubs in our workflows what so ever, so in this case -preview == -stub-run?

I wonder if it is worth running with both -preview and -stub-run in two different runs, where the main purpose of using -preview is basically only for speed(?) / DAG creation only. Alternatively we could run with -stub-run to create the DAGs, but pay the price of slightly slower runs if we add complicated stubs. Would we ever make such heavy stubs though?

And wouldn't the usage of stubs be redundant if we already have system/integration/unit tests that uses the workflows we have created with custom input, i.e. a scaled down reference genome instead of the full one?

[bentsherman]

You don't have to define a stub for every process in order to enable stub runs. Processes without stubs will just run normally during a stub run.

But if you already have a small test input that exercises the entire pipeline and has a sufficiently small runtime, that's probably better than a stub run.

[trym-b]

Ok, thank you for your help @bentsherman your answers really helped me out :)