The State of the Workflow 2022: Community Survey Results

With 384 responses, we received record-breaking participation in our latest State of the Workflow community survey. We are thrilled to present the results here. Last year we had 236 participants. This year’s 63% increase highlights both our growing global community and a high level of engagement among Nextflow users. Today, over 12,000 scientists, engineers, and developers use Nextflow in the course of their day-to-day work. The State of the Workflow survey is a critical tool for gathering community feedback, guiding development roadmaps, and helping us understand the issues and challenges faced by Nextflow users.

This year, we expanded the ‘State of the Workflow’ survey to include the nf-core community. As part of our joint Chan Zuckerberg Initiative Diversity and Inclusion funding, we asked several optional questions related to user demographics which were not included in last year’s survey. The feedback plays an essential role in helping us monitor our goals of creating an inclusive community for everyone. It also establishes a baseline against which we can measure progress year-over-year.

We segmented survey responses into three categories this year and created a separate infographic for each. Categories included users and demographics, cluster & cloud execution, and information about the community’s health and nf-core. In addition, we asked several open-ended questions that resulted in valuable insights. Some highlights of the 2022 survey are provided below.

A Vibrant, Growing Community

The past year saw rapid adoption of Nextflow in both commercial and academic environments. A surprising 44.5% of respondents are either brand new to Nextflow or have been using the software for one year or less. As with last year’s survey, most users self-identify as bioinformaticians, followed by principal investigators and software engineers, and most come from research or academic environments. Nextflow usage has gone global - we had respondents from 20 countries representing every continent except Antarctica - If you are reading this from Antarctica, we would love to hear from you!

• Nextflow adoption is accelerating – over 80% of survey respondents are new to Nextflow and have been using it for three years or less.
• 71.6% of Nextflow users come from five countries – The United States (26%), the United Kingdom (19.3%), Germany (13.8%), Sweden (8.1%), and Spain (4.4%).
• Nextflow users are a happy bunch – 98% are delighted, very satisfied, or satisfied, and 96% would recommend Nextflow to a friend or colleague.

Momentum in the Cloud Continues

While most Nextflow users run workflows on-premises clusters, the shift to the cloud continues. 35.7% report that they already use public cloud, with 68.2% planning to use public cloud in the future. Among the 27% of respondents who reported working in private sector organizations or healthcare (biotech startups, pharma, hospitals, diagnostics, or clinical care), public cloud adoption was more widespread, with 77% using public clouds today.

• Presently, 69% of Nextflow users run workflows on an on-premises cluster, and 83% use an HPC workload manager.
• Among cluster managers, open-source Slurm rules the roost, managing 53% of all Nextflow clusters, followed by Grid Engine, and IBM Spectrum LSF.
• Consistent with our 2021 findings, Amazon Web Services enjoys a considerable lead with 32.3% of respondents running workloads there today, followed by Google Cloud (6%) and Microsoft Azure (5.2%).
• Today, 13.4% of public cloud users report using more than one cloud with some respondents using all three major public cloud platforms.

Nextflow Users are Engaged and Hands-on

Part of what makes the Nextflow survey so valuable is that respondents are practitioners. In total, 65% of respondents build workflows and have firsthand experience with the software. The survey data also highlights the success of the nf-core project. 62.2% of respondents report running workflows from nf-core, and 29% are giving back by contributing to community pipeline development.

• Nextflow users value good documentation – For this “hands-on crowd”, having high-quality documentation was cited as the most important feature of a good workflow manager – right up there with performance and scalability, ensuring data provenance, and maintaining pipeline portability.
• The community matters – not only are nf-core pipelines widely used, but the community is also a major source of support. 67% use the nf-core Slack channel for support. The nf-core website is ranked as the second most important resource for users, with 73.4% seeing it as either very important or important.
• Nextflow Tower is making inroads – Despite Nextflow’s technical user community and their comfort with the command line, Tower is used by almost 30% of survey respondents either sometimes or often. Nextflow users see value in Tower for maximizing the efficiency of cloud resource selection to reduce cost, controlling workflows via the Tower CLI and API, and monitoring workflows via the Web UI.

You can find a PDF version of these infographics for easy sharing here:

We were delighted with the high quality of this year’s survey feedback. Please stay tuned for future Nextflow and nf-core Community Surveys to make sure your voice is heard.