Try it yourself: The full example is available at github.com/nextflow-io/run-until. You can clone it and run it directly with nextflow run nextflow-io/run-until Requires Nextflow 24.04 or later.Nextflow is a dataflow language. Channels carry data one direction through a graph, and the engine works out the dependencies before anything runs. That suits most pipelines well: a fixed set of steps, fan out over samples, collect, report.
One shape, though, has traditionally been a challenge: a process whose input is its own output, repeated until some condition is met. Think of a simulation run to convergence, or any "keep stepping until the metric crosses a threshold" task. The number of iterations isn't known ahead of time, so you can't write channel.of(1..N). And a process that consumes what it produces looks like a circular dependency, which you'd expect to deadlock. You can express exactly this with a topic channel, no preview flags and no recursion:
The Model process here is a stand-in. It reads a JSON file, adds one to the step counter, bumps a score by a random amount, and writes the result back out. Swap in your own simulation or refinement step - the code here is just an example, the loop mechanics don't care what the work is.
In the example, the process fires over and over, each iteration starting from the previous one's output, until the score reaches or exceeds 100:
The final score usually overshoots the threshold. The predicate fires on the first value at or past 100, wherever the last random jump happens to land it.
How the Loop is Wired
Three pieces make this example work: the feedback edge, seeding, and the stopping condition.
The Feedback Edge
Model takes its input from the model_updates topic and writes its output back to the same topic. That single fact is the whole trick: every value Model emits becomes a new item on the topic it reads from, so it feeds itself. Setup's output kicks off the first iteration; from there each Model task's output triggers the next. A topic is a named channel that any process can publish to via the topic: qualifier on an output. Topics were introduced for aggregating outputs across processes (collecting tool versions, say), but nothing stops you using one as the place a loop accumulates its state.
Ordinary channels can't express this wiring, since a process's output channel only exists once the process has been called, and Model would need its own output as an input. Topics sidestep that chicken-and-egg problem because producers and consumers are decoupled by name: the consuming end is declared first, and Model attaches by name afterwards.
Seeding
A feedback edge on its own is a loop with nothing in it. Setup provides the first value: it writes the initial conditions (a JSON file with step and score both at zero) and emits the file to the same topic. Model can't tell a file from Setup apart from a file from a previous Model task, so that one item is enough to set the cycle turning.
The Stopping Condition
Left alone, that feedback edge would run forever. The brake is the limit channel.
The until channel re-emits everything from the updates topic until its predicate first returns true, then closes. The item that satisfies the predicate is dropped, not re-emitted, and that detail carries the whole stopping mechanism. Because Model has two inputs, it fires once per pair of items: one from updates, one from limit. Every model below the threshold gets a partner from limit and triggers another iteration. The model that crosses 100 still lands on the topic, but limit has closed without re-emitting it, so there is no partner and no next task. The loop stops.
The predicate is the one piece you have to get right. It's an ordinary closure, so the condition can be anything you can compute from the values flowing through - a threshold, a convergence check, a step count, a combination. It’s worth making sure that this will always fire at some point, otherwise you’ll be left with a workflow that won’t finish. A sensible safety net might be to add a second check on a maximum number of steps.
Capturing the Trajectory
One nice side effect of splitting the iterations across tasks is that each step is a real process execution, so you get all the usual machinery for free.
The publishDir line writes one file per iteration, named by task.index, so results/ ends up holding the full history of the run:
Resume, retries, per-step resource directives, and execution provenance all apply per iteration too, the same as for any other process.