Kubernetes is everywhere!

For the Nextflow Tower project, it has been a pivotal first year, and the team has done a fantastic job to reach so many milestones along the way!

In June we released the Launch feature that allows users to seamlessly deploy Nextflow pipelines into AWS Batch and Google LifeSciences cloud computing services.

In July we further streamlined the launch experience for AWS users introducing Batch Forge that makes the AWS Batch configuration a breeze and added the support for Pipeline Actions.

Then, in October we released workflow sharing and added popular batch schedulers such as Slurm and IBM allowing users to deploy pipelines in both cloud and on-premises with the same user experience.

Finally in November we made publicly available the Tower API version 1.0.

We’re not (quite!) finished yet

We are closing the year with another significant milestone for the project. Today we are making generally available Tower support for Kubernetes clusters, both on-premises and two of the most popular cloud-based Kubernetes services: Amazon EKS and Google GKE.

We are incredibly delighted with this feature, firstly because Nextflow has had native support for Kubernetes for many years. It is used in hundreds of production workloads across many leading organisations and companies such as EMBL-EBI, Wellcome Sanger and Genuity Science among others. The integration for Tower simplifies the Nextflow deployment and solves some of the limits existing with the Nextflow kuberun command.

Secondly, Kubernetes is rapidly becoming the de facto standard platform for the orchestration of containerized workloads in the enterprise and beyond. We have seen a growing demand for its use from customers of all shapes and sizes.

This integration with Tower opens up Kubernetes to a whole new audience. For a typical end-user launching a pipeline, they may not even be aware that Kubernetes is running underneath. Yet, it empowers administrators to define and manage simple, portable and scalable execution environments.

Conclusion

Today’s release adds a key component to the Tower platform by enabling our users to deploy their data analysis anywhere from a local computer to on-premise batch schedulers up to large cloud-native Kubernetes clusters, always with the same intuitive UI and API experience.

A big thanks to the whole Nextflow community and Happy New Year to all our users, all those that supported the project and our team for the endless effort and dedication to the project! 🥂🥂