The State of Nextflow 2021

In February 2021, Seqera ran the first-ever State of the Nextflow Community Survey. We received a great response rate with 236 participants from a diverse set of Nextflow users. Respondents offered thoughtful feedback, sharing how they use Nextflow, what they like about the platform, and what they would like to see in the Nextflow roadmap. There were dozens of helpful ideas and suggestions that will help guide community contributors and future development priorities.

The following infographic summarizes the highlights:

We wanted to share some insights from the Nextflow survey below:

• 73% of Nextflow users are bioinformaticians. Other Nextflow users include principal investigators and managers (9%) and computer and data scientists (8%). Biologists, MRI scientists, software architects, and system administrators account for approximately 10% of users.
• Nextflow is widely used in academic settings (50%), government & research (22%), pharma and biotech (16%), and healthcare and clinical care (8%). Interestingly, Nextflow users are found in other industries, including manufacturing, data research & analytics, and the airline industry.
• Most users run workflows across multiple environments. 55% run Nextflow locally on a single computer while 75% report running on an on-premise cluster. 36% also run workflows on a public cloud platform (Amazon Web Services, Google Cloud, or Microsoft Azure). Nearly two-thirds of public cloud users also operate an on-premises cluster, making hybrid cloud a common deployment pattern.
• Traditional HPC schedulers dominate on-premises cluster deployments. Among the 75% of respondents running local clusters, nearly all ran one of Slurm™, Grid Engine™, IBM® Spectrum LSF, HTCondor™, or PBS™/Torque.
• Among public cloud users, the story is mixed. Among the 36% of respondents who report using public cloud, most report running managed cloud batch services such as AWS Batch, Google Life Sciences, or Azure Batch. Others burst to the cloud under the control of their on-premises workload managers or only run in the cloud exclusively.
• While Kubernetes adoption is modest today, with just 8% using it, there is significant interest. 59% of respondents are either using it, plan to use it, or may use Kubernetes in the future. Interest is highest among those using managed batch services, suggesting that users may see Kubernetes as a strategy to improve performance and cross-cloud portability.
• Nextflow users are happy overall, with 83% of respondents reporting that they were satisfied or extremely satisfied with the tool. Nextflow users consider the most important features to be compatibility and portability (56%), ease of use (50%), container support (49%), resume functionality (42%), and the level of community support and engagement (35%). Nextflow Tower is also a hit with Nextflow users, with almost a third of respondents (69%) already using it.

We were delighted with the high-quality survey feedback. Congratulations to Shreeraj Dabholkar of 23andMe as the winner of Apple Mac mini!

Please stay tuned for future State of the Nextflow Community Surveys to make your voice is heard.