The next step for multi-platform collaboration at scale

We have spent a lot of time over the last months on an embarrassing problem; although this time it is not parallelization! Working backward from our user’s feedback, they all had one overwhelming request. The recurrent need to manage multiple pipelines, users and compute resources with role-based access, all while providing a genuinely seamless experience for people with little or no Nextflow expertise.

Nextflow is a powerful framework for writing complex distributed pipelines. Yet it has remained the domain of command-line savvy folks who are comfortable with the power it affords them.

Today we are super excited to introduce a major release of the Nextflow Tower platform bringing the full support for multiple users, teams, organizations, and workspaces along with fine-grained control of resources using access management for different roles.

But this is not the only enhancement. Today marks the opening up of the Nextflow ecosystem to anybody with a need for distributed data analysis. The new Launchpad feature greatly simplifies the experience of running pre-configured Nextflow pipelines and provides administrators with the tools and controls to configure and manage shared resources on any cloud or cluster infrastructure.

Launchpad

The Launchpad is the new landing page for Tower allowing users and organizations to collate, organize and share ready-to-run pipelines. Deployment settings such as the target cloud platform and credentials are provided once by an admin or maintainer, and after this, a user only needs to select the pipeline to be executed and provide their input parameters.

If the Git repository associated with the selected pipeline includes a parameters JSON schema definition, the system will render an input form that dramatically helps the user with the many parameters that a pipeline may require. We are very delighted that this feature is the result of a collaboration with the highly talented nf-core team that first envisioned this capability.

Organizations & teams

Tower also now allows administrators to create and manage one or more Organizations. Organizations can invite multiple members and Tower users can share and collaborate on pipeline projects. The members of an organization can be further organized into Teams which allows groups of users to be associated with specific operational roles.

Workspaces

Last but not least, this release introduces the new concept of a Workspace. Workspaces allow Tower users to organize projects into independent areas, keeping together all related information such as pipelines, experiments (i.e. pipeline runs), computing resources, and above all, the users who can operate on them. Different roles make it possible to control who can administer or maintain a workspace, for example, to set up a pipeline or compute environment, and who has the permissions to re-launch an experimental run.

What next?

We are very confident that today's release represents the most significant milestone for the Tower project to date. It responds to the needs of organizations with multiple stakeholders with varying needs and who all rely on Nextflow as their orchestration engine for data pipeline at scale.

As usual, you can try out all these features just by signing in to Tower. Feel free to create an Organization, a Workspace, and invite your colleagues to join. To submit a pipeline to a Workspace using the Nextflow command line tool, add the workspace ID to your environment. For example:

export TOWER_WORKSPACE_ID=215559034546123

If your organization is interested in deploying Tower in your cloud or on-prem environment, please reach out to us at sales@seqera.io, and we would be happy to discuss your requirements.

Thanks for reading and stay tuned for more exciting announcements on the way!