joeduffy (u/joeduffy)

joeduffy commented on Incus – Next-generation system container, application container, and VM manager linuxcontainers.org/incus... · Posted by u/motorest

belthesar · 2 months ago

There is a Terraform provider that is actively maintained, in addition to Ansible integration. https://linuxcontainers.org/incus/docs/main/third_party/

I'm a Pulumi user myself, and I haven't seen a Pulumi provider for Incus yet. Once I get further into my Incus experiments, if someone hasn't made an Incus provider yet, I'll probably go through the TF provider conversion process.

joeduffy · 2 months ago

Note that you can now use TF providers "on the fly" from Pulumi: https://www.pulumi.com/blog/any-terraform-provider. No provider bridging/conversion necessary.

I just tried and it seems to have worked (though I haven't tested any specific resources yet):

$ pulumi package add terraform-provider lxc/incus

joeduffy commented on HashiCorp adopts Business Source License hashicorp.com/blog/hashic... · Posted by u/rpadovani

paulgb · 2 years ago

I am a paying Pulumi user. Their tool integrates with a cloud platform and we pay per resource managed by Pulumi.

Pulumi is one of several products where I like that it’s open source in case I need to move off their cloud, but hope that I don’t have to (Plausible is another).

joeduffy · 2 years ago

[Joe, Pulumi Founder here.]

Said well (and thank you for being a customer and valuable member of our community!)

The analogy I draw sometimes is that our open source infrastructure as code SDK ("Pulumi") is like Git, and our commercial offering ("Pulumi Cloud") is like GitHub.

Like GitHub, the Pulumi Cloud offers valuable features that go beyond the open source project for teams looking to manage lots of projects securely at scale, but we definitely love our open source community and want folks to have the choice to use Pulumi however makes the most sense for them.

This approach also has the nice consequence that we can be fully transparent with our community at all times while also building a strong, long-term business. If a new feature is part of the infrastructure as code SDK, it's open source and free; if it's part of the Pulumi Cloud SaaS, it's part of our commercial offering. This avoids needing to do things like artificially hold back features (like open core) or violating our commitment to the open source community (like Hashi's new license).

joeduffy commented on HashiCorp adopts Business Source License hashicorp.com/blog/hashic... · Posted by u/rpadovani

jamestanderson · 2 years ago

All that I get from this is that HashiCorp is no longer an open source company.

> However, there are other vendors who take advantage of pure OSS models, and the community work on OSS projects, for their own commercial goals, without providing material contributions back. We don’t believe this is in the spirit of open source.

This is 100% in the spirit of open source. If this is a problem for them, why not adopt an open source license that compels developers to open source their code instead, like the AGPL?

This is purely a way for HashiCorp to ensure they are the only ones who can commercialize these formerly open source projects. Which is fine. But just go closed source, then, and own that, instead of trying to have it both ways.

joeduffy · 2 years ago

Pulumi Founder/CEO here.

The blog post is disingenuous. We tried many times to contribute upstream fixes to Terraform providers, but HashiCorp would never accept them. So we've had to maintain forks. They lost their OSS DNA a long time ago, and this move just puts the final nail in the coffin.

Thankfully over time, they already pushed responsibility for most Terraform providers back onto their partners, so I'm hopeful the ecosystem of providers can still stay vibrant and open.

We are deep believers in open source---heck my last project at Microsoft was to take .NET open source and cross-platform, our CTO helped found TypeScript, and Pulumi is an Apache open source project---it seems HashiCorp no longer is.

joeduffy commented on AWS Cloud Control API, a Uniform API to Access AWS and Third-Party Services aws.amazon.com/blogs/aws/... · Posted by u/lukehoban

lukehoban · 4 years ago

CloudFormation is really three separate things rolled up into one: (1) a resource model for AWS, (2) a deployment orchestration engine and (3) a syntax for specifying desired state as CloudFormation templates.

Tools that compile to CloudFormation templates offer a way to access (2) directly - to use alternative front-end syntaxes but to still deploy via the CloudFormation orchestration engine.

Tools like Pulumi (and Terraform) have their own deployment orchestration engines, which we believe offer many benefits - performance, secrets, components, transformations, aliases/refactoring, multi-cloud provisioning, and a lot more.

Cloud Control API lets us (and others like us) access (1) directly, without having to use (2) or (3), and thus being able to still offer the full set of benefits of provisioning via Pulumi, along with the full set of benefits of a well-defined AWS resource model.

joeduffy · 4 years ago

As a concrete example of what this enables, the Pulumi Automation API lets you embed IaC straight into a bigger program. Folks have used this to create infrastructure-oriented SaaS products, self-service portals, and new higher level abstraction frameworks and tools, for instance, often spanning multiple cloud resource types (AWS, Azure, Kubernetes, Cloudflare, others) -- https://www.pulumi.com/automation/. Transpiling to YAML and handing it over to CloudFormation is clumsy and wouldn't work for these cases among others.

joeduffy commented on Ask HN: Who is hiring? (May 2021) · Posted by u/whoishiring

joeduffy · 4 years ago

Pulumi | Many Roles, Engineers, Engineering Managers | Remote | https://www.pulumi.com/careers/

Pulumi is an open source infrastructure as code platform enabling you to program the cloud in your favorite languages. We help developers and infrastructure teams build better software together. Pulumi is polyglot in many ways, so you'll have the opportunity to work with Go, Node.js, Python, and other language ecosystems, on a daily basis; Pulumi is also multi-cloud, meaning you'll also get to work with Kubernetes, serverless, AWS, Azure, and much, much more. We have open roles at all layers of our stack, including our open source platform in addition to our SaaS application. We are a remote team, growing fast, and also hiring engineering managers. If this sounds fun, apply online, or email me at joe AT pulumi DOT com!

joeduffy commented on Pulumi 3.0 pulumi.com/blog/pulumi-3-... · Posted by u/bovermyer

antoncohen · 4 years ago

> Open Source SDK

> The SDK is a CLI and collection of libraries for defining and deploying cloud apps and infrastructure in code.

https://www.pulumi.com/pricing/

With Terraform the open source CLI and libraries for defining and deploying infrastructure are fully usable, even for large systems.

What I would like to know is, how usable Pulumi is without a paid subscription?

I thought when I first looked at Pulumi, the only non-local state backend was the paid Pulumi Service. But looking at it now, they seem to support the normal object store backends that Terraform does (https://www.pulumi.com/docs/intro/concepts/state/). Though the talk at lack of concurrency control seems to imply they don't support locking like Terraform does.

joeduffy · 4 years ago

[caveat, Pulumi co-founder here]

Pulumi is fully functional in open source form. The analogy I like to draw is Git and GitHub. You can use Git fully independent of GitHub, or you can choose to use them together, for a seamless experience within a team. (Not a perfect analogy since we built both Pulumi open source and the Pulumi SaaS, which causes this very confusion!) We don't hold anything back, if it's in the SDK, it's open.

We recently added concurrency control to the alternative backends. I'm sorry the docs are confusing on this matter -- we will get that fixed up. We also have many large customers in production on the open source alone. It's easier with the SaaS just because we handle security, reliability, and sharing with your team along with access controls, auditing, etc. But if you prefer to roll your own there we are entirely happy to have you in the community and help out. Admittedly our marketing materials aren't super clear here and we are working to fix this.

Hope this helps to clear things up and again apologies for the confusion.

joeduffy commented on Pulumi 3.0 pulumi.com/blog/pulumi-3-... · Posted by u/bovermyer

leg100 · 4 years ago

According to [1], "Native providers are built directly and automatically from the cloud provider’s API...", and are not hand-coded, unlike Terraform's providers.

I'm surprised this is possible. If it was, why didn't Terraform follow this approach a long time ago?

The cloud providers' APIs provide the endpoints for creating, updating, reading and deleting resources. Terraform and Pulumi's value is to provide an idempotent abstraction on top of that. And that abstraction is not straightforward to write, because it has to handle numerous nuances and anachronisms in the underlying APIs. For example, in the event of updating an existing storage bucket, the abstraction has to determine whether the bucket can be simply updated, or if it needs to be re-created (say if you were changing the name or location). And the underlying API will not necessarily reveal this kind of information.

Hence one would think completely avoiding handwritten code is incredibly difficult if not an insurmountable problem.

[1]: https://www.pulumi.com/blog/pulumiup-native-providers/

joeduffy · 4 years ago

[caveat, Pulumi co-founder here]

You are right that it's not easy. Thankfully the cloud providers themselves have moved in the direction of auto-generation for their own SDKs, documentation, etc., which has forced this to get better over time. This is motivated by much the same reason we've done it -- keeping one's own sanity, ensuring high quality and timeliness of updates, and ensuring good coverage and consistency, when supporting so many cloud resource types across many language SDKs.

Microsoft, for instance, created OpenAPI specifications very early on, and standardized on them (AFAIK, they are required for any new service). Those API specifications contain annotations to describe what properties are immutable (as you say, the need to "re-create" the resource rather than update it in place). Google Cloud's API specifications similarly contain such information but it's based on the presence of PATCH support for resources and their properties. Etc.

The good news is that we've partnered with the cloud providers themselves to build these and we expect this to be increasingly the direction things go.

joeduffy commented on Pulumi 3.0 pulumi.com/blog/pulumi-3-... · Posted by u/bovermyer

ynouri · 4 years ago

Glad to see all the innovation happening in the IaC space.

I like the multi-language premise of Pulumi but I have the feeling that over time one of the languages will gain a significant advantage over the others (better supported, more features, native...) and thus will become the primary language for Pulumi package authors and users (similar to HCL for Terraform and TypeScript for CDK).

Meaning that eventually all the "serious" Pulumi users will end up converging on one single language (similar to this comment of a CDK user migrating from Python to TypeScript). Feedback loops will only help amplify this effect (e.g. more Pulumi users on language A -> more examples online -> more package authors working in this language -> better established best practices -> more submitted issue tickets for language A -> language better supported by Pulumi -> more users...)

Curious about how Pulumi multi-language components [0] work under the hood. Isn't writing a "Pulumi schema" describing resources the same as writing declarative HCL at the end of the day?

[0] https://www.pulumi.com/blog/pulumiup-pulumi-packages-multi-l...

joeduffy · 4 years ago

Excellent topic. [caveat, Pulumi co-founder here]

Indeed what you say is true of many other "multi-language" platforms. I was an early engineer on .NET at Microsoft, and although it was multi-language from the outset (COBAL.NET was a thing!), the reality is most folks write in C# these days. And yet, you still see a lot of excitement for PowerShell, Visual Basic, and F#, each of which has a rich community, but uses that same common core. A similar phenomenon has happened in the JVM ecosystem with Java dominating most usage until the late 2000s, at which point my impression is that Groovy, Scala, and now Kotlin won significant mindshare.

I have reasons to be optimistic the infrastructure language domain will play out similarly. Especially as we are fundamentally serving multiple audiences -- we see that infrastructure teams often elect Python, while developers often go with TypeScript or Go, because they are more familiar with it. For those scenarios, the new multi-language package support is essential, since many companies have both sorts of engineers working together.

A "default language for IaC" may emerge with time, but I suspect that is more likely to be Python than, say, HCL. (And even then, I'm not so sure it will happen.) One of the things I'm ridiculously excited about, by the way, is bringing IaC to new audiences -- many folks learn Python at school, not so much for other infrastructure languages. Again, I'm biased. But, even if a default emerges, I guarantee there will be reasons for the others to exist. I for one am a big functional language fan and especially for simple serverless apps, I love seeing that written in F#. And we've had a ton of interest in PowerShell support since many folks working with infrastructure for the Microsoft stack know it. And Ruby due to the Chef and Puppet journeys.

I also won't discount the idea of us introducing a cloud infrastructure-specific language ;-). But it would be more general purpose than not. I worked a lot on parallel computing in the mid-2000s and that temptation was always there, but I'm glad we resisted it and instead just added tasks/promises and await to existing languages.

As to the Pulumi schema, you're right, that's a step we aim to remove soon. For TypeScript, we'll generate it off the d.ts files; for Go we'll use struct tags; and so on. Now that the basic runtime is in place, we are now going to focus there. This issue tracks it: https://github.com/pulumi/pulumi/issues/6804. Our goal is to make this as ridiculously easy as just writing a type/class in your language of choice.

joeduffy commented on Use your favorite programming language to provision Infrastructure as Code opensource.com/article/20... · Posted by u/jaxxstorm

verdverm · 5 years ago

How many extra tools and runtimes will this require your CI/CD system to have? I'm not sure the complexity is worth it.

iirc, it's just using Terraform behind the scenes anyway. If that is the case, is Pulumi just a glorified transpiler?

joeduffy · 5 years ago

The defining characteristic about Pulumi compared to other tools is that it's not a transpiler, in fact. It's a multi-language runtime written in Go that can host many language plugins (Node.js, Python, .NET, Go, etc), as well as many resource provider plugins (native ones, OpenAPI-based, Terraform-based). So although yes it can use Terraform providers -- great for coverage across many infrastructure providers as well as easy portability if you're coming from Terraform/HCL -- it's not correct to say that it's "just using Terraform" or is a "transpiler".

Pulumi is open source on GitHub if you want to check it out: https://github.com/pulumi/pulumi.

joeduffy commented on Ask HN: Who is hiring? (August 2020) · Posted by u/whoishiring

joeduffy · 5 years ago

Pulumi (https://pulumi.com) | Remote (Seattle HQ) | Full Time

Pulumi is a cloud engineering startup whose flagship infrastructure as code platform helps infrastructure teams and developers create modern cloud architectures. Our open source SDK uses general-purpose languages and ecosystems of tools to deliver apps and infrastructure across any cloud -- including AWS, Azure, Google Cloud, and Kubernetes -- with software engineering practices like testing, sharing and reuse, and more. We offer a SaaS for teams and enterprises using this SDK at scale.

If you enjoy working at the intersection of cloud infrastructure and developer platforms, you'd love it here!

We have several positions open, including:

* Platform Engineer (Go/Python/JavaScript/TypeScript/C#)

* Full-Stack Engineer (JavaScript/TypeScript/Angular/Go)

* Customer Engineer (pre- and post-sales for AWS, Azure, Kubernetes, and more)

Our careers page has more details: https://pulumi.com/careers. Feel free to apply there or just email joe at pulumi dot com (I'm a founder).