run-until

A self-feeding loop in Nextflow: a process whose input is its own output, run repeatedly until a score crosses a threshold. The wiring uses a topic channel, so it needs no preview flags and no recursion.

This is the companion example for the blog post "Running until: iterative loops in Nextflow".

Running it

With Nextflow 24.04 or later:

$ nextflow run nextflow-io/run-until

Or from a clone:

$ nextflow run main.nf

Tasks run on the local machine by default. To run them in Docker instead, using the python:3.13-slim image declared in nextflow.config:

$ nextflow run nextflow-io/run-until -with-docker

Each Model task reads a JSON state file, increments a step counter, adds a random amount (1 to 10) to a score, and writes the state back out. The output lands on the same topic channel the process reads from, so every completed task triggers the next one. An until operator closes the loop once the score reaches 100. A typical run takes 15 to 25 iterations:

[PROCESS ef/cf3783] Model (16)
[PROCESS 5a/7dfcf5] Model (17)
[PROCESS 1f/849d07] Model (18)
Last model: {"step": 18, "score": 102}
[SUCCESS] completed=19 failed=0 cached=0

The results/ directory ends up holding one JSON file per iteration, the full trajectory of the run.

How it works

Three pieces make the loop:

1. Model reads from the model_updates topic and publishes its output back to the same topic. That feedback edge is the whole trick: every value the process emits becomes a new item on the channel it consumes.
2. Setup seeds the topic with the initial state, which kicks off the first iteration.
3. limit = updates.until { ... } re-emits items until the score crosses the threshold, then closes. Model needs one item from each of its two input channels to fire, so once limit closes, the final model has no partner and the loop stops.

Model itself is a stand-in. Swap in your own simulation or refinement step; the loop mechanics don't care what the work is.

Tested with Nextflow 26.04.4.