Seqera

News and Views: The Sarus container engine, staging outputs and the GITHUB_TOKEN

Watch on YouTube
Listen on Spotify

In this News and Views episode, Chris Hakkaart, and Marcel Ribeiro-Dantas discuss the hottest topics in the Nextflow world.

The Sarus container engine

  • Recently Sarus was added to Nextflow as a container engine.
  • Sarus is an OCI-compatible container engine for HPC.
  • Sarus is an alternative container runtime to Docker.
    • Sarus works by converting Docker images to a common format that can then be distributed and launched on HPC systems.
  • Similar deployment options to other container engines using Nextflow.

New stageOutMode options

  • Two new stageOutMode options (rclone and fcp) have been added to Nextflow.
    • The stageOutMode directive defines how output files are staged-out from the scratch directory to the process work directory.
  • rclone
    • Has been described as “The Swiss army knife of cloud storage”.
    • A command-line program to manage files on cloud storage.
    • Over 40 cloud storage products support rclone.
  • fcp
    • A significantly faster alternative to the classic Unix cp command.
    • Handle the most common use cases of cp with much higher performance.
    • fcp is optimized for systems with an SSD.
  • The new options will improve file management as there are new ways for the files to be staged from the scratch directory in different circumstances.

GITHUB_TOKEN

  • Git has become the de-facto standard for source code version control systems.
  • Nextflow provides built-in support for Git and the most popular Git hosting platforms.
  • Nextflow does not require any special configuration to access public repositories but requires repository credentials to access private repositories.
  • The recent Nextflow commit adds the ability to use the GITHUB_TOKEN environment variable.
    • The environment variable is used when no credentials are found in the source code management configuration file ($HOME/.nextflow/scm).
    • It is especially useful when accessing pipeline code from a GitHub action. Read more about the token authentication in the GitHub documentation.

Upcoming events

  • There’s a nf-core/bytesize talk on January 31 by Franziska Bonath about her work to generate transcripts of bytesize talks and what these might be used for in the future.
  • The Nextflow / nf-core training is being held March 13-16.
    • Training will be presented in different languages.
    • Registration is now open - head over to the nf-core event page to sign up.
  • The nf-core hackathon is being held March 27-29.
  • Hackathon 27-29 March.
    • We will also support local hubs where local community members are hosting other community members.
    • Watch out on Slack and Twitter for an announcement that registration is open.