Migrating Legacy Bioinformatics Pipelines to Nextflow at Personalis for Faster Cancer Detection and Monitoring

Aim

Build a modular and scalable pipeline infrastructure to accelerate precision oncology R&D, enabling earlier cancer detection and streamlined monitoring, as well as faster, reproducible product delivery.

Challenges

The rapid pace of research and development required the Bioinformatics Engineering group at Personalis to continuously iterate on products to satisfy changing requirements, all while supporting a unified codebase for their three major products: ImmunoID NeXT®, NeXT Dx®, and NeXT Personal®. Each product within the NeXT® platform includes multiple configuration options, requiring a highly modular pipeline architecture to manage complexity effectively. However, the existing scientific workflow management system was outdated and lacked the flexibility to support this need. Even minor updates demanded deep familiarity with interconnected dependencies, making maintenance slow, and unsustainable at scale. Personalis needed a solution that could enable modular development, reduce operational overhead, and scale efficiently with their evolving research and product demands.

Solution

• Nextflow - Orchestrate modular, containerized pipelines with clear inputs and outputs that communicate via channels for reproducible and scalable analysis.
• MultiQC - Compiles analysis results into a single HTML report for easy quality control review.
• Docker - Isolates processes to ensure consistent, reproducible execution across environments.
• Git version control - Tracks all code and configuration changes for full traceability and collaboration.

Results

By migrating their legacy scientific workflow system to Nextflow, Personalis was able to build a modular, efficient, and maintainable infrastructure. This transition delivered major performance gains, including 3 times faster wall-clock runtimes across all products and disk space reductions of up to 1000 times. They were also able to streamline DevOps operations, saving hundreds of engineering hours per year through automated deployments.

These improvements not only optimized resource usage but also enhanced reproducibility, maintainability, and research agility, accelerating both development cycles and innovation across the company’s precision oncology platforms.