Nextflow plugins: development walkthrough

Ever wondered about how Nextflow plugins work? Join us for this walk through tutorial, as Ben Sherman guides Phil Ewels to create and publish a plugin from scratch. Right from the first line of code to creating a release and publishing the plugin.

Nextflow plugins have been getting more attention lately and we’ve been getting a lot of questions in the community. There is better documentation and resources planned, but in the mean time we hope that this can be a useful guide for anyone curious in how they work, and interested in getting started.

This podcast episode is quite different to normal. Rather than an interview or a discussion, it’s more of a tutorial. It’ll work best if you follow the video, though hopefully it’s still useful with just sound.

We’d love to hear what you think of it! After this we will be back to our usual routine in the podcast, but we’d love to hear what you thought of this taster!

Links to resources used / mentioned in this episode:

• Nextflow Documentation: Plugins
• Example plugin used as a base: nextflow-io/nf-hello
• Phil’s plugin created during recording: ewels/channels-demo
• Provenance reporting plugin: nextflow-io/nf-prov
• Schema validation plugin: nextflow-io/nf-schema

Time stamps:

• 00:00:00 Welcome
• 00:03:27 Nextflow docs
• 00:08:34 Starting by forking nf-hello
• 00:10:32 Overview of project files
• 00:17:13 Trying a first compile
• 00:18:12 Different publishing methods
• 00:19:54 nf-boost local publish method
• 00:23:05 Trying the new compile
• 00:24:00 Running locally with the plugin
• 00:27:32 Looking at the nf-hello plugin code
• 00:39:29 Deleting files we don't need
• 00:42:34 Finding event names in the Nextflow source
• 00:47:46 Writing some custom functions
• 00:49:41 Testing our new code
• 00:51:51 Modifying the test pipeline to create files
• 00:54:14 Coding up the JSON output
• 00:59:20 Looking at nf-prov code for BCO files
• 01:05:30 Testing JSON output
• 01:09:39 Automatic work dir cleanup / nf-boost
• 01:12:17 Publishing a release on GitHub
• 01:19:07 Custom plugin repositories
• 01:26:47 Publishing for all Nextflow users
• 01:29:52 Conclusion and end