Full transcript

Intro

Phil Ewels: Hello and welcome to the Nextflow podcast. You are listening to episode 55, going out in February, 2026.

This episode is all about Terraform and the use of Terraform within Seqera Platform. This is a new project. It’s a very new thing we’ve spun up with in Seqera to help you with your infrastructure around Seqera.

And to tell us all about it and why, we think it’s cool, and why we think you should care, I’ve got two guests on today.

Introductions

Phil Ewels: Ken and Adam, thanks very much for joining me, both of you.

Adam Talbot, you are the scientific services lead, and Ken Brewer is the Scientific solutions lead at Seqera. I had to write those down ‘cause there was no way I was gonna remember who was who. yeah. Thank you for joining both of you.

Just before we got kind of get started, can you tell us a little bit about, what you do at Seqera? So maybe Ken, you can lead us off.

Ken Brewer: Sure. Yeah, so I, as a solutions lead at Seqera, I head up our pre-sales function. So I mostly work with new customers who are evaluating Seqera to understand like, how do they deploy it? How do they configure it? Where can they get value?

When I first joined Seqera, I was actually on the community team. I was doing developer relations. Heavily involved in the nf-core community and running hackathons, and that’s something that I’m still involved in, although that’s not my, my full-time job anymore.

Adam: Yeah, I’m, scientific services lead, which, I’m in charge of a small team of people who basically provide bioformatics engineers for hire. We’ve got so much expertise here in Seqera and a lot of people, customers and new customers and existing customers want to, engage with us.

So things like helping people set up infrastructure, pipelines, building automations, that sort of thing to help me get the most out of Nextflow.

Phil Ewels: Both of you have been around in Nextflow and nf-core for years at this point. Ken you’re in the core team for nf-core, and Adam, you’re a maintainers team, right? And Yeah, I was gonna say, I feel like you’re familiar faces at this point.

Why Terraform complements Nextflow

Phil Ewels: Great stuff. So, Adam, maybe you can just kind of give us a bit of an intro as to what, what we’re talking about today. How, how does this all kind of fit into the, the, the picture around Nextflow.

Adam: Yeah. Cool. one of the things that really attracted me to Nextflow when I first started was it’s, got this really good ability to we say write your pipeline once and then deploy many times; you write your pipelines as code. And then you set up how your pipeline’s gonna run, where they’re gonna run via configuration. So it neatly separates out infrastructure from the actual pipeline tool.

This is, invaluable. It means you can write a Nextflow pipeline and put it on your SLURM cluster, your, AWS batch environment, your Kubernetes environment. You just by changing some configuration settings.

It’s probably its strongest features, how things are nf-core, how they’re built this, everyone run writes the same pipelines.

The kind of two, like the challenges we then face are, you’ve got a great pipeline. How do I set up that configuration to make sure it’s just correctly set up? And then how do I make sure the infrastructure underneath it is reliable, trustable? I know exactly what went in there and how it works.

And if you’re at a university, you’ve probably never had to experience this ‘cause you’ve got an IT team who are building you a great big HPC cluster and maintain it for you 24 7.

But, as Nextflow starts to get used by more and more, a wider variety of people, you, just see so much more diversity. In things like smaller teams where they’re having to administer their own cloud environment, bigger enterprise units where they’re like the DevOps is a complete different country and working a different way.

And so we’ve got this two phase problem now where we’ve got bioinformaticians, writing great Nextflow pipelines, and then they need somewhere to run it and deploy it. And so we need to stand up this infrastructure ahead of time.

So this is really where tools like Seqera and Terraform and all these things start to come in into play as we need good tooling, the same tooling we’ve had with Nextflow.

We need to have it for infrastructure as well.

How Ken fell in love with Terraform

Phil Ewels: And Ken, this is, this is basically something you, you lived through right prior, prior to, to joining Seqera.

Ken Brewer: Yeah, absolutely. Like my, my, my first role after finishing my PhD was at a small biotech startup, Profound Therapeutics. I was like the second computational hire, and my first project was like: figure out how we’re gonna do bioinformatics at scale. And that’s when I fell in love with Nextflow and nf-core. was at least in my view, the, the superior options among all the things that I compared.

And, and not too long after, like adopting Nextflow and then starting on the journey of, one of the first 20 something customers of Seqera’s, which was a small software startup that had just showed up not long after, like I started my Nextflow journey. I found myself in a position where I was trying to write an alphafold pipeline for, for internal use by, company engineers. And there’s just a lot of headaches around how the data gets staged and managed.

This was before, Seqera had come out with Fusion, I was doing a lot of iteration on, infrastructure, trying to figure out like, how do we get these massive amounts of data staged in a performant way, so we can like run our pipelines efficiently.

And I was clicking through the Seqera interface, like Batch Forge was like an incredible utility for like doing that iteration, but it just felt like there was something not quite right, in terms of like how often I was clicking through the same things over and over again.

So I, I came across a tool called, Terraform. I was doing, searching about like, hey, there’s what is the best practices for infrastructure like management? And I also discovered that, a different Seqera customer called Healx had put together sort of like a homegrown fan made version of a Terraform provider that had like a couple of key things made ready into Terraform, and that’s kind of when I started experimenting with Terraform for Seqera, as developed by this other customer.

And not long after I was did that experimentation, I, I ended up changing to a new role. Joined GeneDx, which is a large clinical diagnostics company, with like a very mature DevOps practice, and Terraform was very core to the way everything was happening.

Basically anytime you wanted to make changes in the AWS account. It was something that was almost like democratic in terms of like who could make infrastructure changes, because you would make infrastructure changes by simply opening poll requests to like the central GitHub repo that describes all of your infrastructure. And then once you’ve done that, someone else like sees what the changes will cause in terraform, console output, and then someone can approve the poll requests and when it gets merged, it gets applied.

And, and so I really, really wanted to tap into all of this beautiful goodness of Terraform with, our Seqera configuration because a lot of the things that I was building to plug into Seqera were being built in Terraform, like AWS users and credentials and policies and permissions. I just wanted to easily take them from and drop them straight into like Seqera.

Phil Ewels: Makes a lot of sense if you were kind of living in that Terraform ecosystem already.

Where Terraform sits alongside CLI and API

Phil Ewels: Yeah. And so like, I mean, you talked about the Platform web UI with kind of lots of clicking, click ops. We’ve, we’ve also got like other ways to access Seqera Platform. There’s the, the CLI, there’s the Python library Seqera kit, and then the, the low level API itself.

Like where, where do they fit into all this? Is, is Terraform, I guess Terraform is kind of based on top of the API or, or how does that, how does, how do those kind of puzzle pieces fit together?

Adam: Yeah, Seqera is ultimately a web server with strong APIs that allow you to just deploy things like compute environments or credentials or things and, put them onto Seqera Platform.

Because it has a standard OpenAPI spec, you can integrate with it in a bunch of ways. Whatever you’re doing, you’re still interacting with the API, you, if you click the interface, you’re interacting with the API. If you then use CLI, you’re doing the same thing and so on. And Terraform uses those same APIs.

The difference really is your user experience. So with the interface, it is great for experimentation and, trying things out. You can click click click, and off you go. But there’s no version control. There’s no snapshot of what you did and how you did it.

The CLI is the next layer up. For a bioinformatician, this is very familiar territory. They’ve got a terminal, they type the command in tw compute environments add. And then add some specifications on onto that CLI command. What will then happen is the CLI will go off and call those APIs, which then Seqera Platform on the other end, starts doing all the work in the background.

That’s, that’s good. That’s a really great like start, and it helps you just have that kind of imperative style of, type this and do this for me. In the Scientific Development team, we were finding that we needed something a bit more reproducible.

We actually developed Seqerakit, which is a wrapper around the tower, CLI, which basically declares all the tower CLI commands, you want to run in, in, order. So for example, you say, Hey, I want to create workspace, add these users, create these compute environments. Add these pipelines. Launch these pipelines. And you can just do it in one big YAML file you write, and then just Seqerakit, run that. And it’s basically going through the YAML file and running the tower CLI, each one in turn.

Phil Ewels: This feels kind of, familiar from the Nextflow story around bioinformatics, like bioinformatics of running commands and then looking in your bash history. And then, you know, kind of

Adam: Absolutely.

Phil Ewels: Did anyone ever write a Nextflow pipeline to run the Seqera CLI? It would be very meta.

Adam: Yes, we did. We’ve done that. Of course we have, I think we’ve, done that and then launched it on the platform and all sorts of weird things. Like how many Russian dolls can we go like down to the bottom level?

The, I mean, it kind of, highlights the problem, which is not really writing down what you did. It’s, it tells you exactly what you want to do, but it doesn’t give you a nice log of how you did it, when you did it, why you did it. That’s something that, is a real problem with infrastructure.

Infrastructure drift

Adam: We had this concept in infrastructure called Drift. So if you stand up a load of compute environments using really controlled scripts, and then an SOP and all these precise things, then somebody comes along and just clicks and changes some settings in the background, you don’t know they’ve done that. And so if you need to, stand up that, resource, again, got no record of it.

We wanted to move to the next step. People were pushing Seqerakit really far, like really far, and we were finding more and more feature requests coming in that were just like. Basically people asking for Terraform.

So Terraform is the industry standard for infrastructure as code. There are other things, but Terraform by far is number one. And yeah, the parallels of Nextflow are pretty strong. What Nextflow did for bioinformatics, Terraform did for infrastructure management.

Terraform origin story

Phil Ewels: Can you tell us a little bit about, Terraform? I mean, I don’t know very much about it personally, so this is kind of also interesting for me. I, I know it was developed by HashiCorp, right. And, and it’s open source and I think they have a kind of, I, I look at them sometimes ‘cause they have a similar sort of setup to us at Seqera, where they’re a company with commercial products built on top of an open source foundation, and they also happen to have really nice websites, which I quite admire.

But like, tell, tell us a little bit about that story of, of what Terraform is and what world that came from.

Ken Brewer: Terraform really came from this, kind of broader industry challenges around, infrastructure change management and the fact that like if you want to go ahead and make changes to your infrastructure, the, the, the way to do it was to have like sort of like long guides and how-to’s of how you would do the deployment step by step. Generate a secret here. Go ahead and drop it into a user interface over there. Then like, take that onto the next step. And so this is a lot of like very manual, hands-on processes, which is difficult to do in a reproducible and reliable fashion.

So I, I kind of see Terraform very much kind of like as a, as a sister to Nextflow. Because what one of the things that Terraform does really well is that it will create the DAG, like the direct acyclic graph of your infrastructure where, you know, like this piece of infrastructure depends on this piece of infrastructure, which depends on this piece of infrastructure. And then if I make a change to, this AWS entity, this is everything that depends on it that needs to be sort of recreated and rebuilt.

So it kind of gives you essentially like, “nextflow -resume” like functionality for your infrastructure configurations and your, your infrastructure changes. And that’s kind of like the really powerful utility that’s kind of led so much of like the, the tech world to adopt it as like, hey, this is the, this is the tool to that we need for how we are managing and tracking our changes for infrastructure.

Phil Ewels: Nice. Yeah, that makes a lot of sense. I see the attraction.

Ken Brewer: And, and the other thing that I think kind of has made Terraform, such a strong utility is that it is platform agnostic. AWS for example, they have an SDK, but if you’re using the AWS SDK or AWS cloud formation templates, that’s like a, a one infrastructure, provider, solution.

And often when you’re doing infrastructure, you’re not doing just one piece of infrastructure that you’re managing. You’re managing, how these different pieces play together. You might be creating like a GitHub secret that you need to bring into your AWS in a certain way. Or creating an AWS user and policies that you need to be using in Seqera.

Terraform gives you a common language to track the dependencies of these different platforms on each other, in a way that doesn’t require you to write a bunch of custom bash scripts.

And kind of a, another piece of Terraform is that a lot of people have, including companies, have invested effort into writing Terraform providers. A Terraform provider is essentially, all of the information that Terraform needs to know how to interact with the API in a programmatic way to add, create, delete and modify different objects or resources.

And these providers you can almost think of like an nf-core module, where someone writes it once and then anybody can just , pull that provider and then use it in their own infrastructure without needing to write, a bunch of boilerplate stuff around like, this is how Terraform is supposed to do stuff.

One last thing to mention about Terraform that I think is a really powerful utility is that, it’s not, imperative where you kind of like need to go through step by step and say, oh, things need to be done in this order. It’s, declarative, and that means you simply declare what your desired end state is. And then Terraform will go through the process of figuring out like how to do the changes and then what order they need to happen in order to give you your final end state.

Use cases for Terraform

Phil Ewels: Okay. If I’m running on my laptop and I’m just sort of doing small scale stuff, I don’t, I don’t need this. And if I’ve already got an HPC cluster, which is set up for me, then I probably don’t need this. What kind of use cases do you see coming up for this?

Ken Brewer: Yeah. So a couple of examples of people who would really typically care about this. Let’s say you are a large academic medical center. And you’re trying to support the deployment of Seqera Platform workspaces to all of the labs in your university, who are running bioinformatics workflows.

There’s a lot of like, stuff, that you’re going to need to click through if you want to do that manually. Like for, let’s say you’ve got 50 labs, you need to make sure they all have like separate isolated workspaces that they, have certain levels of like cloud options and permissions in each of those that they have, like the appropriate credentials. You don’t want to have to manually click your way through that.

So instead with Terraform, you can create a module that describes: this is what a lab’s Seqera infrastructure is supposed to look like, and these are like the inputs and variables that I need, and I want to deploy this lab workspace module 50 times, for each of these different people.

Another example might be like clinical diagnostics companies. A lot of times the way they are setting up their Seqera infrastructure is going to be very, locked down. Where if you are running like the production genetic testing, Essentially, you only want to be able to make changes with multiple levels of approval to how that infrastructure is set up and how the Seqera, area is like permissioned. You essentially don’t want to have any admin users who’s allowed to just go in and make changes without approval. Instead, there’s like one sort of set of admin token credentials that are going to be managed by Terraform. So any change is going to have multiple sets of eyes on it before you’re, you’re changing how, like the pipelines and everything is set up.

You could also imagine, you’re an IT department, like for a pharma company, and you do all of your work in Terraform for creating things. And then, some clinical team starts using Seqera Platform. You want to get them up and running, but you want to do it in a way that fits your existing tooling and your flow of how you’re doing your infrastructure change management. Like Seqera’s Terraform provider is gonna be kind of like your, your solution.

And I guess one other like example of someone who would like really care about this is people who are doing infrastructure benchmarking. And this kind of takes us back to like what originally got me interested in Terraform, where I was trying to find like, hey, what’s the best way to, to run these jobs at scale, not just on like the pipeline optimization front, but on like the infrastructure optimization front. And, and this is kind of where Terraform will be really helpful.

Infrastructure is cattle, not pets

Adam: I, found it very powerful. Because Terraform has that caching mechanism, kind of similar to Nextflow, it will only replace update resources as I modify them. So if I need to twiddle with infrastructure, get it just right, like it’s way easier to sort of keep twiddling the, numbers in terraform and hit Terraform apply. And it will only changed the resources that I’ve actually modified.

And at the end of it, I’ve got a perfect, reproducible piece of code I can share with someone and say, this is the optimum settings for you.

Such a big impact in the cloud because the resources are temporary. Like they’re designed to be disposable. They’re cattle not pets, right? And, so you should be building them in such a way you can destroy them every couple of days and build them up again and not care, and it’s fine.

Phil Ewels: Poor cows.

Ken Brewer: I think that is the most sad I have felt about killing infrastructure, Adam. So thank, thank you for that moment.

Phil Ewels: Okay. So, I can see how it is useful for tweaking things, for experimentation, benchmarking, for reproducibility. We have a lot, a lot of people, especially in companies who have dev and and or staging and production instances. I’m guessing it’s probably good then as well. ‘Cause you can kind of like mirror stuff. Is that,

Adam: A really cool feature called variables. So what you end up doing is going, terraform apply, vars file development.tf vars. And it goes and deploys it all onto your development environment. And then you write another TF vars file for your staging environment. Terraform apply that one. And then Terraform apply production.

And it’s running exactly the same code, but three times with different parameters. And it saves where it got to and how it did them for all of those different environments. And so then you can go back to them.

This is pretty useful for, you can roll out a change to development, make sure it works, then roll out, staging check it works in a near, near production environment, and then roll out to production.

Phil Ewels: Okay. I feel like I’ve got a fairly good picture of what it’s useful for.

Terraform inception

Phil Ewels: So what, when you’re talking about standing up the Seqera part of things, are we talking about deploying enterprise installations of Seqera or is it more like kind of bits within Seqera. Like how, what kind of primary objects within Seqera are you configuring within Terraform?

Ken Brewer: There’s both. If you’re doing an enterprise install of Seqera, and, and just to clarify, this means like you’re standing up the actual Seqera application within your own infrastructure, then a lot of times you might be using Terraform to build the infrastructure that the Seqera application is going to run on.

What the Seqera provider is, is specifically designed for managing everything that happens inside the Seqera application. So after you’ve already stood up your Seqera application. Or if you’re running on cloud.Seqera.io, how do you manage all of like the, the orgs and workspaces, the teams, credentials, pipelines, studios, all of those different things that Seqera platform that you can click through and create.

Those are the things that the Seqera, Terraform provider is designed to help you with.

Phil Ewels: Nice because that, that was a point of confusion for me for a while. ‘cause because we have a Terraform thing for actually deploying and Yeah, that’s what I thought. So it’s two different Terraform use cases here. One for installing Seqera itself, and then one for kind of using Seqera platform.

Adam: Terraform is the standard for infrastructure. So if you want to stand up Seqera Platform itself, you’re gonna probably be using Terraform at some point to get Seqera Platform stood up. But then once you’ve done that, you can use a Seqera provider, which is a Terraform Library for Seqera. And that’s the, tool you’ll be using.

Usage walk through: Basics

Phil Ewels: As someone who’s never used Terraform before, if I want to use this today, what do I do? What are the steps I go through.

Adam: Yeah. So Terraform is a CLI application, written in Go. First thing you can do is open up a code repository. And then you’re gonna create a main.tf file. Again, another parallel with Nextflow, the main TF file. It’s kind of similar syntax to Nextflow configuration, in that you have the name of a scope and then a open curly brace, and then you know, the contents, and then closes. And then the next sort of scope you go into.

So the first one you’re probably gonna write is one called Terraform. Open the curly brace, and in there is gonna be something saying providers, required providers. And you’re gonna say, I want the Seqera provider. That’s the thing I need.

You then when in your repo you can do terraform init. And Terraform will go and fetch the Seqera provider from the registry, pull it down, have a local version. It will do a lock file and to make sure you’ve got a static version. So the recommended thing is to, snapshot, a version. Say we’re gonna use version 0.26.5, the current version. And then off you go.

So you configure some authentication, make sure you can talk to Seqera Platform using your authentication token, which you can provide in a secret way, with Terraform. And then you’re gonna start writing your resources. First thing you wanna create is an organization. You would do resource Seqera underscore organization. And then you give it a name. So you say, Hey, I want it to be called my org. So inside that, scope is the settings, name, logo, that sort of thing.

Then close that. Underneath the next one, do resource again and you maybe wanna create a workspace. So resource Seqera, underscore workspace, my workspace. Give it some details, give it some, settings, that you want to put in your workspace. And then you might do like organization, organization id, then what you’ll do is you’ll go myorg.id. And so

Phil Ewels: That’s dynamic.

Adam: Exactly, it’s gonna read it in from that organization resource.

So then come out of, writing my text and I go on the command line. I write Terraform plan. Terraform goes away and it looks at your resources and says, Hey, I need to create an organization. I need to create a workspace. Firstly, it checks if it’s done it before, and if it hasn’t done it before, it’ll create it fresh. Okay.

So knows that the workspace needs the ID so it will know to do the organization first. Goes to the Seqera API, creates an organization. It goes and creates the workspace using that id. And there you are, you’ve got a workspace set up.

Phil Ewels: And, and you said it, you said it checks if it exists already. So if the organization existed already but the workspace didn’t, it’s not gonna like delete the existing work organization and recreate it. It’s just gonna gonna say, yes, I found it. This is the id that already exists. And change any settings, I guess if they’ve changed, I.

Adam: So it saves a, a state file. state file is the entire state of your infrastructure and it is quite an important file. You need to keep it secure and yours need to keep it like shared between, any time you run Terraform. Otherwise, you’ll just be standing up new instances everywhere.

That will save the workspace details and say: this workspace is created here, or this organization is created here.

If I then go, if I run the command the second time, the first thing Terraform will do is check against that state file. Depending on the resource, it might also go up to Seqera API check that those resources haven’t changed and check for Drift. So in case someone’s changed the name or something of the resource, on the remote side.

Then if there’s a difference, in the terminal, it’ll give you all the resources that need to be modified before it if you were to run it again.

So it’ll say, Hey, are you sure you wanna do this? I’m gonna delete this resource, I’m gonna update this resource. I’m gonna, create this new resource, yes or no. and so it can just, it can give you that sanity check before you do it,

Phil Ewels: That’s nice, especially when you’re like deleting stuff. It’s good to have a confirmation step. Yeah.

Adam: this, there’s one more trick up its sleeve, which is there’s a destroy command. so if you want to tear down your resources and you don’t wanna make, and you wanna make sure you don’t leave anything lying around, you know all the horror stories of having an EC2, charge racking up a $23,000 bill or something. You can make sure that doesn’t happen. ‘cause you can do terraform apply, destroy, and it’ll just bring everything down for you.

Ken Brewer: Yeah. Making, making all of the sad cattle go away.

Regulated environments

Phil Ewels: How did he do, Ken? Did he, did he miss anything?

Ken Brewer: No, that’s, that’s like a really good summary. Like I, I think the key takeaways is like. When you write the Terraform, you’re, you describe your desired end state.

Then you’ve got this plan and apply a loop like in GitHub actions where when you open a pull request, you do the plan to see what this changes do, and then when a change gets merged, then you actually apply the changes.

Phil Ewels: Oh, that’s nice. I like that. Bit a bit less CLI again, and a bit more automation.

Ken Brewer: Yep.

Adam: It, it really brings, Ken mentioned change control earlier. And it nicely brings it into the software development lifecycle. And it’s a really nice sort of flow that matches the sort of what the auditors would require.

Phil Ewels: I was gonna say, you, you, you can go and do, “git blame” on any part of infrastructure and see who changed it and why. Yeah.

Ken Brewer: Yeah. And, and this support for regulated environments was a key part of why Seqera decided to, to invest in building a, Terraform provider.

Dogfooding Terraform for Seqera

Ken Brewer: That and the fact that as soon as I joined Seqera, like I was a man on a mission. That I needed, I Seqera Terraform provider and I started raising a ruckus, like, Hey, when are we getting a Terraform provider? When are we getting the Terraform provider?

Phil Ewels: Reminds me a little bit of that. I see it go around LinkedIn. I’m, I’m not sure it’s true, but it’s the story of someone joining a company, fixing one bug, and then quitting. It’s like, I hope that doesn’t apply to you too.

Ken Brewer: No, no plans. No plans to do anything of that sort.

Adam: We, both use it in our day-to-day. Now, I dunno about you, Ken, but like you, I, know you use it to stand up demo environments, things like, here’s, what your Seqera platform could look like. And I use it a lot for tight integration, like when people are weird requirements, basically they need to set up some sort of confusing cloud environment and tie it to Seqera.

And that’s another real strength to Terraform. We’ve already mentioned it, but you can bring in other providers so you’re not limited, just Seqera, you say, bring in the a DS provider, create me the Im user that Seqera is going to be, and then use those credentials in Seqera platform. So it is just, this is a really strong tool for this infrastructure management.

Cautionary notes

Phil Ewels: Is there anything, like, is there any gotchas that you’ve just kind of, you know, potholes, you stood in with this? Like, is there, is there any kind of common pitfalls.

Ken Brewer: There, there’s, there’s several ways, like, you know, so Terraform could be a very good rake to step on if, if, if you’re not careful. Need to be sure that you’re never hard coding secrets. Terraform does have support for like labeling certain values as sensitive, but one thing that need to be especially protected is your state file. And so typically if you’re gonna be working, as a team on, a set of Terraform deployments, you’ll set up some sort of like remote state, which could be S3, it could be HashiCorp’s own Terraform solution.

Other things is just being careful about the variables. Anything that’s a secret, you’ll typically want to inject via an environment variable rather than a TF vars file.

Adam: Or it can be a resource. So a resource and you go and get it. So 1password have a resource so you can go get it from 1password as part of your code, which is quite cool.

Phil Ewels: I was, I was, I was actually thinking of mentioning one password. The way you were talking about the state file. That reminded me of kind of sharing passwords within a team.

Ken Brewer: Yeah. And that’s actually how, I am deploying demos right now. Like I have my Seqera token in one password and I import it, via the one password, provider from two to use for Seqera deployment. That’s so, so it is all very, very, it, it, it all works together so nicely. It’s

I think another really important thing about best practices is you don’t really want to mix your management approaches, for like Terraform versus UI.

I think what works well is to kind of decide ahead of time. These are the things that I’m going to manage with Terraform and these are the things that my users are going to do themselves.

So one clean division might be, I’m going to use Terraform to create workspaces, create credentials, and create compute environments. But I’m going to choose to leave like the pipeline launchpad as something that my users can go wild on and do whatever they want.

Or if you’re like in a regulated environment, then you might decide actually the pipeline launchpad that is a protected entity. And so, I want to have that be terraformed as well.

If you, if you end up mixing the approaches, you get a lot of that drift that Adam was talking about.

Phil Ewels: Something we’ve pushed on Seqera platform fairly recently, is fine-grained authorization where we can define more specifically what different user roles can do and can’t do.

Does that fit in here? Can you kind of like use fine-grained authorization to remove that permission from certain user groups? Apologies. If you dunno the answer and I’ve just sprung this on you.

Adam: I don’t know. Probably is the answer. So we don’t, have, user management. That’s one of our limitations in the, Terraform provider at the moment. So it is something we’re working on, and will be ready by the first stable release, but it’s not, quite there yet. In theory, this is something that we like to have in the provider.

But that really large enterprise organizations, they’re gonna start managing their roles with, their provider. Things like Microsoft Entra or, Okta and things like that. So it starts to, there’s a blurry line of this moving from DevOps into IT, and someone’s gonna make a Terraform for IT Management, clearly. But, not me.

Seqera provider timeline

Phil Ewels: Adam, you mentioned just now that, coming up to the first release like this, this is all very new still, right? What, how long has the Terraform provider been around for and, and what’s the, what’s the usage and adoption been like so far?

Adam: Six months, maybe? I think it’s about six months. There’s been a load of iterations though, that we’ve done so many releases and things.

Ken Brewer: We started with a stealth release, to the Terraform registry, where we just didn’t announce that. And then I think the official announcement of it happened at the, at the Nextflow Summit where, Evan announced that among other new features.

And we’ve had some customers who have started engaging with it and started like testing it out, providing feedback about like, Hey, the way this particular resource works is like, frustrating or doesn’t fit our flow. And, our, DevOps team has been like hard at work at like incorporating that feedback and getting new versions of the provider out the door.

And, and now it feels like it’s definitely in like a very usable and functional state like. Like Adam said, the two of us are using it almost daily in terms of like, how often we’re like iterating on Seqera things.

And so we’re, we’re very eager right now for anyone who’s managing their Seqera Platform infrastructure who thinks this might fit some of their use cases to give it a try and provide feedback, while we work on these additional iterations.

Adam: If you’re learning Terraform for the first time, the AI is very good at writing Terraform because it’s very thick syntax and quite declarative like Ken said, like AI just seems to take to it like a duck to water and just write a bunch of quite good terraform. you can then read.

Phil Ewels: And so there’s the Terraform registry, I think you called it, is that the best approach if people want to try this out

Adam: yeah, there’s a Terraform registry, which has essentially the auto-generated documentation, from the code says, here’s all the resources. Here’s, it is like the API documentation. It’s very precise, but a bit, not the easiest to read.

If you go to the GitHub repo, which is seqeralabs/terraform-provider-seqera. That’s where the provider is and there’s a lot more documentation in there. There’s some guidance, best practices. There’s also a lot of examples for different cloud providers, different environments.

Ken Brewer: I, I, I do wanna say though, like. It brings me just an incredible amount of joy to see, Seqera Lab slash terraform dash provider. Seqera. I joined Seqera as like a man on a mission. This was a thing that I’ve wanted, like for, for a good two or three years before joining. And then, it took a year and a half of, you know, shaking trees and, rallying support.

And it feels so good to finally have this in the hands of customers and having people already to see the value of bringing these very Nextflow like approach to infrastructure and using that tooling within Seqera.

It just kind of, it brings a lot of sort of like open source compatibility, happiness, to like how I’m doing my changes these days.

Conclusions

Phil Ewels: That’s good. Alright. Well, from the chatter I’ve seen around this Terraform provider so far, it’s still early days, but people seem really excited about it. You guys seem really excited about it and I’m kind of like maybe thinking to myself, I need to go and try it out.

Ken Brewer: Especially Phil, if you could set up a Terraform automation that interacts with your smart home devices, like, like I, I, I expect some sort of smart home integration slash Terraform set up from you.

Phil Ewels: there must be, there must be a Home Assistant provider. That ecosystem is like big enough that must someone, someone would have done it already.

Adam: it. Someone’s done it.

Phil Ewels: Awesome. Alright, thanks so much guys for, for bringing enthusiasm, for bringing a drive and making us this actually happen, on behalf of a community. And, and thanks for joining me today on the podcast it’s been, it’s been really fun. I’ve actually learned a lot about this, so it’s good. great stuff.

Ken Brewer: Great to be here.

Adam: Thanks Phil

Phil Ewels: Cheers all.