Wisconsin State Laboratory of Hygiene

Aim

Rapidly adapt internal processes to handle growing volumes of complex sequencing data to enable efficient data analysis at scale.

Challenges

Before the COVID-19 pandemic, WSLH used Docker containers (orchestrated with shell or Python scripts) to develop their internal genomic data analysis workflows. However, achieving parallel processing and fault tolerance was challenging, labor-intensive, and impractical at scale. Such workflows could not easily be shared with external agencies and organizations, hindering public health collaboration.

During the pandemic, WSLH had to rapidly scale up team sizes, computational resources, and analytical workflows to handle the surge in complex sequencing data. This increased capacity was essential for tracking the spread and evolution of the virus, necessitating real-time data sharing across organizations for timely public health interventions. Moreover, this underscored the need for more scalable, efficient, and standardized solutions.

“In public health, standardized workflows are needed to help unite pathogen surveillance efforts across laboratories.” - Kelsey Florek, Senior Genomics and Data Scientist

Solution

• Nextflow for high parallelization, fault tolerance and comprehensive logging.
• nf-core pipelines - peer-reviewed, standardized workflows, including nf-core/viralrecon.
• Seqera Platform - Centralized workflow execution with dynamic scaling and real-time monitoring.

Results

The Wisconsin State Laboratory of Hygiene (WSLH) has transformed its public health response capabilities by improving the management and optimization of compute resources. Since the COVID-19 pandemic, WSLH has scaled their genomic surveillance capacity by 10 times, reducing data processing times, and minimizing unnecessary costs. This has enabled the more rapid identification and monitoring of infectious diseases. Additionally, this has provided WSLH the flexibility to expand the scope of pathogens that can be sequenced, building capacity for future growth.

“Public sharing of workflows, data, and methods is critical to the success of the public health mission and in the best interest of the public” Kelsey Florek, Senior Genomics and Data Scientist

Feature image credit: Bryce Richter from the University of Wisconsin–Madison