Announcing the Nextflow Tower CLI
We’re excited to release the Nextflow Tower CLI, available for all platforms on GitHub today. Users have been asking for the ability to control Tower from the command line, much like they do with Nextflow. Today we are making this a possibility and opening new avenues to launch pipelines, manage cloud resources and administer Tower.
It is difficult to match the flexibility and power that a CLI provides, and the Tower CLI is no different. It gives the ability to create, manage, and execute Tower resources, streamlining workflow management and enabling all the benefits of Tower with a Nextflow-like experience.
Let’s get into the details!
Tower CLI
Tower makes it easy to create environments where teams can collaborate on all aspects of their scientific workflows. Until now, users had the choice between the GUI and the OpenAPI. With the CLI we deliver a new mechanism to interact with Tower, providing the programmatic flexibility of the API approach combined with the ease of use that only a CLI can bring.
For users comfortable with the Nextflow command line, the CLI offers familiarity, coupled with all Tower benefits, including monitoring, task logging, and preconfigured compute environments, all in a collaborative, multiuser environment.
For admins and infrastructure folks, it simplifies building and managing infrastructure as code, reducing the time needed to set up working environments.
Finally, it opens up the possibility to easily script automations and integrations such as uploading datasets and triggering workflow executions.
From nextflow run to tw launch
Nextflow Tower CLI provides a familiar, flexible, developer-friendly interface. With a simple change from nextflow run to tw launch, all execution details, including logs, are available in Tower to all workspace participants.
It eliminates the need for the Nextflow master job to be running in a shell instance for the workflow to proceed. With tw launch, Tower takes care of launching the master job with Nextflow orchestrating the task jobs in the chosen compute environment.
Infrastructure as code
Tower resources, including Pipelines and Compute Environments, can now be described declaratively. It allows a complete definition of an analysis environment that can be versioned and treated as code.
Infrastructure as code dramatically simplifies sharing and re-use of configuration and provides more options for routine administration.
With the Tower CLI, you can build a complete research environment in the cloud with a few lines of code. This reduces human error and makes sharing and duplication of resources a straightforward task.
Automation
The Tower CLI opens up new opportunities for automation. An everyday use case is chaining together several different workflows, such as one for data retrieval and another for processing.
Another is the automatic creation of sample sheets after files are deposited in a location. With the CLI, it is now possible to upload the sample sheet to Tower and launch the pipeline with this new data.
Bonus
To celebrate this release and highlight the CLI's flexibility, we are giving away an Elgato Stream Deck to one user who launches a pipeline using tw launch in the Tower Cloud Community Showcase before Jan 31st 2022 (one entry per user).
To enter the draw:
Download and install the Tower CLI from GitHub
Log into Tower Cloud and create a personal access token by selecting Your tokens below the avatar profile
Export your personal access token in a terminal session
export TOWER_ACCESS_TOKEN=<your access token>Export the Community Showcase workspace ID
export TOWER_WORKSPACE_ID=40230138858677Launch a pipeline
./tw launch nf-core-rnaseq
Head over to the Tower Cloud Community Showcase to monitor your workflow executions
Conclusion
The Tower CLI provides additional options for users to Launch pipelines, automate analysis steps, and set up data infrastructure as code. The CLI is compatible with Tower Cloud and Tower Enterprise versions 20.10 and above.
If your team would like a demo or wants to discuss your broader workflow needs, reach out to us, and we would be happy to set up a call.