I joined HashiCorp in 2016 to work on Nomad and have been on the product ever since. Definitely a lot of feelings today. When I joined HashiCorp was maybe 50 people. Armon Dadgar personally onboarded us one at a time, and showed me how to use the coffee maker (remember to wash your own dishes!). There have been a lot of ups (IPO) and downs (BUSL), but the Nomad team and users have been the best I've ever gotten to work with.
I've only ever worked at startups before, but HashiCorp itself left that category when it IPO'd. Each phase is definitely different, but then again I don't want go back to roadmapping on a ridiculously small whiteboard in a terrible sub-leased office and building release binaries on my laptop. That was fun once, but I'm ready for a new phase in my own life. I've heard the horror stories of being acquired by IBM, but I've also heard from people who have reveled in the resources and opportunities. I'm hoping for the best for Nomad, our users, and our team. I'd like to think there's room in the world for multiple schedulers, and if not, it won't be for lack of trying.
I've had the incredible displeasure of having to maintain multiple massive legacy COTS systems that were once designed by promising startups and ultimately got bought by IBM. IBM turned every last one into the shittiest enterprise software trash you can imagine.
Every IBM product I've ever used is universally reviled by every person I've met who also had to use it, without exaggeration in the slightest. If anything, I'm understating it: I make a significant premium on my salary because I'm one of the few people willing to put up with it.
My only expectation here is that I'll finally start weaning myself off terraform, I guess.
> Every IBM product I've ever used is universally reviled by every person I've met who also had to use it
During my time at IBM and at other companies a decade ago, I can name examples of this:
* Lotus Notes instead of Microsoft Office.
* Lotus Sametime Connect instead of... well Microsoft's instant messengers suck (MSN, Lync, Skype, Teams)... maybe Slack is one of the few tolerable ones?
* Rational Team Concert instead of Git or even Subversion.
* Using a green-screen terminal emulator on a Windows PC to connect to a mainframe to fill out weekly timesheets for payroll, instead of a web app or something.
I'll concede that I like the Eclipse IDE a lot for Java, which was originally developed at IBM. I don't think the IDE is good for other programming languages or non-programming things like team communication and task management.
I think this is an interesting graph comparing web searches for "terraform alternative" and "opentofu". Notice the spike when the IBM rumors began, and the current spike now that the acquisition is complete?
Unfortunately IBM is going to ruin everything that was good about working for Hashicorp and eventually everything that was good about Hashicorp products.
I worked for a company acquired by IBM, and we held hope like you are doing, but it was only a matter of time before the benefit cuts, layoffs, and death of the pre-existing culture.
Your best bet is to quit right after the acquisition and hope they give you a big retention package to stay. These things are pretty common to ease acquisition transitions and the packages can be massive, easily six figures.
Then when the package pays out you can leave for good.
none of that has happened for us at Red Hat. Other than the one round of layoffs which occurred at the time that basically every tech company everywhere was doing much larger layoffs, that was pretty much it and there's no reason to think our layoffs wouldn't have been much greater at that time if we were not under the IBM umbrella.
Besides that, I dont even remember when we were acquired, absolutely nothing has changed for us in engineering; we have the same co-workers, still using all Red Hat email / intranets / IT, etc., there's still a healthy promotions pipeline, all of that. I dont even know anyone from the IBM side. We had heard all the horror stories of other companies IBM acquired but for whatever reason, it's not been that way at all for us at least in the engineering group.
We use Nomad where I work and we LOVE it. Previous to Nomad we used K8s for several years which, at that point, allowed us to become cloud agnostic. With the move to Nomad about 3+ years ago, we were able to transition away from cloud and back to leased, bare metal machines. During our time with K8s, it didn't have a good bare-metal strategy with their ingress mechanism. In contrast, as we investigated Nomad, it was easy to deploy on pure metal without a hypervisor. The result of our migration to Nomad was having so many capable and far-less-expensive hosting options. Lastly, as part of our Nomad control plane, we also adopted Vault and Consul with great success.
I know there are horror stories around this acquisition and lots of predictions about what will happen, but only time will tell. On a minimum, it has been a delight to use the Hashicorp software stack along with the approach they brought to our engineering workflow (remember Vagrant?). These innovations and approaches aren't going away.
I used it literally this year to create a test double of the NUC that runs my home automation stack. I also used Packer to configure Flatcar and create the qcow2 that Vagrant consumes.
Vagrant is still the best tool for creating a general purpose VM on your machine. It got kind-of forgotten in the containers and Kubernetes hype, but it still gets the job done. Packer is also the best tool for creating VM images that got buried for the same reasons.
The datacenter is coming back, though. IBM would be smart to invest in these tools as loss leaders to TFE and Vault and monetize the providers, IMO.
Glad to hear Nomad is working well for you! It always means a lot. We as the dev team spend all day staring at bugs and missing features, so it's nice to be reminded of what's working well! No one files an issue for a happy cluster. :)
I would GTFO, IBM ain't your friend and ain't your savior and are unlikely to invest and the worse may come with increasing IBM management sticking their fingers in the pie. The folks who did well out of this already know, they have the checks to cash if that was your take away congratulations. Otherwise find another opportunity. If nothing else look around and find out what you are worth on the market and then have that hard discussion soon with HashiCorp/IBM.
I worked with a bunch of people who had worked at a startup that got bought by IBM. As the other commenters attested, they too experienced that IBM is not the kind of company that's going to turn on the investment taps.
There are worse companies to get bought by, but if you've only ever worked at startups then you're not likely to enjoy what this becomes.
I spoke with a guy (too long ago) that was a "genius architect" and worked for a company that was small enough, that he got to implement his castles in the air. Knowing him, they might have been quite good, but it was one person that knew the details and made changes at the architect scale. He had a quirky way of thinking.
When IBM acquired that company, after a few weeks, this guy had a meeting with new engineering people. The very first meeting, they changed things for him. Instead of a single winding road of development, they wrote out a large spreadsheet. The rows were the distinguishable parts of his large'ish and clever architecture; the columns were assignments. They essentially dismantled his world into parts, for a team to implement. He was distraught. He didn't think like that. They did not discuss it, it was the marching orders. He quit shortly afterwards, which might have been optimal for IBM.
Fairly large Nomad Enterprise user here, and I just want to say thanks for all of the work you and the team put in. I'm a big fan of Nomad and really appreciate the opportunities it has afforded me.
Regardless of the general sentiment, hoping for the best outcome for all of you.
Hey on a personal note, dealing with you and your team on Nomad's GitHub issue tracker was always a good experience. I hope nomad still has a future under IBM's roof.
I just wanted to say thank you for your work on Nomad. It's one of the most pleasant and useful pieces of software I have ever worked with. Nomad allowed us to build out a large fleet of servers with a small team while still enjoying the process.
> Nomad allowed us to build out a large fleet of servers with a small team
I really love these stories. Our customers constantly shock me with how large of clusters they're able to manage with just a few people. >10k nodes by <10 people isn't uncommon although we are not good at rigorously collecting data on this.
Nomad is far from perfect, but we really strive to ease the pains of managing your compute substrate.
No matter how long you worked at the acquiree and instrumental you were, be prepared for your opinions to be overridden by IBM lifers because you're not "true blue" (ie. directly to IBM). Also prepare for the bluewashing!
Just wanted to say thanks for your work on Nomad. Amazing tool that had me rethink a lot of things about how i work with infra and software in general and is always pleasant to work with by itself.
Thanks! I think this is a really insightful comment because Nomad is more opinionated than I think we let on. The subtle interactions between signals, service discovery, health checks, templates, etc. all have a big impact on the reliability of the system as a whole. I've always wanted to do a "HashiCorp Validated Nomad App Design" kind of doc to outline these best practices, but it's always proved too big of a lift. Maybe now is the time!
There are no resources and opportunities after being acquired by IBM. I worked for Red Hat when they were acquired. Our former CEO was quickly shown the door. We were making so much profit, almost a $1B in quarterly revenue. I left not long after the acquisition. Not long after I left, they laid off a bunch of staff.
No matter what they tell you, your day to day will not improve. For my area, it was mostly business as usual, but a net decrease in comp because IBM's ESPP is trash.
I was at a certain open source company IBM acquired, and it was certainly an interesting experience. I certainly don't harbor any overtly negative feelings towards IBM. I used to have negative feelings about IBM many years ago, as a customer in my former life as a sysadmin, many decades ago. However being under the IBM umbrellas was alright. Yall are going to be fine!
Hashicorp's stuff always struck me as pretty hacky with awkward design decisions. For Terraform (at least a few years ago) a badly reviewed PR could cause catastrophic data loss because resources are deleted without requiring an explicit tombstone.
Then they did the license change, which didn't reflect well on them.
Now it's being sold to IBM, which is essentially a consulting company trying to pivot to mostly undifferentiated software offerings. So I guess Hashicorp is basically over.
I suspect the various forks will be used for a while.
> For Terraform (at least a few years ago) a badly reviewed PR could cause catastrophic data loss because resources are deleted without requiring an explicit tombstone.
There have been lifecycle rules in place for as long as I can remember to prevent stuff like this. I'm not sure this is a "problem" unique to terraform.
IIRC, the lifecycle hook only prevents destruction of the resource if it needs to be replaced (e.g. change an immutable field). If you outright delete the resource declaration in code then it’s destroyed. I may be misremembering though
What happens if you forget the lifecycle annotations or put them in the wrong place or you accidentally delete them? Last time I checked it was data loss, but that was a few years ago.
I concur. I looked pretty hard into adapting Serf as part of a custom service mesh and it had some bonkers designs such as a big "everything" interface used just to break a cyclic module dependency (perhaps between the CLI and the library? I don't recall exactly), as well as lots of stuff that only made sense if you wanted "something to run Consul on top of" rather than a carefully-designed tool of its own with limited but cohesive scope. It seemed like a lot of brittle "just-so" code, which to some extent is probably due to how Go discourages abstraction, but really rubbed me the wrong way.
An easy way to get someone to admit that terraform is a hacky child’s language is to ask how to simply print out the values of variables and resources you are using in terraform easily. This basic programming language 101 functionality is not present in the language
If you mean somehow printing things when the configuration is being applied... I think you just need to understand that it's neither a procedural language (it's declarative) nor general-purpose (it's infrastructure configuration).
It's a design flaw. It should be nearly impossible to delete data accidentally. And it should be possible to roll back state in IIAC.
No serious organization with any scale is going to have the only thing standing between them and a production database deletion being two over tired engineers rubber stamping each other's code changes.
Bad config pushes to prod do happen and they can cause outages like the 2024 Cloudstrike outage. You don't want a tool that takes a minor (but significant) error and turns it into a catastrophic one because of poorly thought out semantics. It's better to just start with a tool that requires at least two engineers to explicitly sign off on deletion.
My hot take is just that Vault isn't a good solution, and the permissions model is wholly inadequate.
Except for not "feeling" secure, the only thing everyone wants is a Windows AD file share with ACLs.
Just no one realises this: all the Vault on disk encryption and unsealing stuff is irrelevant - it's solving a problem handled at an entirely different level.
Sorry HashiCorp, been there and got the Tee-shirt (pink) :)
Actually for me, the company I was at that IBM purchased was on the verge of folding, so in that case, IBM saved our jobs and I was there for many years.
We experienced arbitrary layoffs in 2023, followed by an ominous feeling that more layoffs were imminent. However, the announcement of a deal changed the situation.
Now, we are actively hiring for numerous positions.
Personally, I am not planning to stay much longer. I had hoped that our corp structure would be similar to RedHat, but it seems that they intend to fully integrate us into the IBM mothership.
I really wanted to work at HashiCorp in 2017/2018 and did five interviews in one day only to get ghosted[1]. That experience soured me on HC and its tools but I still admired them from afar.
Red Hat has been a very atypical approach. There has been some swapping of teams back and forth but, as far as I can tell (been out of it for a while), Red Hat is still quasi-independent. Still lots of changes (probably most notably because of a lot of growth) but strategic Red Hat areas still seem to be pretty independent.
Years before '93-'96 when I worked at Kaleida [1], a joint venture of Apple and IBM, alongside Taligent [2] their AIM Alliance [3] sister company, I laughed at the old joke:
Q: What do you get when you cross Apple and IBM?
A: IBM.
But then the joke was on me when I finally worked for a company owned by Apple and IBM at the same time, and experienced it first hand!
I gave Lou Gerstner a DreamScape [4] demo involving an animated disembodied spinning bouncing eyeball, who commented "That's a bit too right-brained for me." I replied "Oh no, I should have used the other eyeball!"
Later when Sun was shopping itself around, there were rumors that IBM might buy it, so the joke would still apply to them, but it would have been a more dignified death than Oracle ending up lawnmowering [5] Sun, sigh.
Now that Apple's 15 times bigger than IBM, I bet the joke still applies, giving Apple a great reason NOT to merge with IBM.
As it happened with the other startups that were acquired by IBM, this too shall pass through the digestion system of the dinosaur and ejected out as a dump. Hashicorp products are showing the signs of a legacy thing already. IBM is the nursing home for these sort of aging stuff.
I'm a heavy user of Terraform and Vault products. Both do not belong to this era. Also worked for a startup acquired and dumped by IBM.
What are the modern equivalents? For Terraform I'd imagine it's Pulumi or OpenTofu but what is it for Vault? Last I checked OpenBao didn't seem to have much juice but it's been a minute since I did so. Or are there unrelated projects in this space that are on the same trajectory as Hashicorp was a decade ago?
What, in this era, replaces secrets handling in Kubernetes in a way thats easy for most devs to pick up?
What, in this era, replaces provisioning cloudy stuff that doesn't require heaps of YAML or a bootstrap Kubernetes cluster for operators to run within?
Most of the these type plays the home page has stacked toolbars / marketing / popups / announcements from the parent company and their branding everywhere (IBM XXX powered by Redhat)... I see very little IBM logo or corporate pop-up policy jank on redhat.com.
Nice. When I opened their homepage, I could not find anything obvious that shows they are owned by IBM. Literally, I had to search the HTML source code to find the sequential characters "IBM"!
Readit News
I've only ever worked at startups before, but HashiCorp itself left that category when it IPO'd. Each phase is definitely different, but then again I don't want go back to roadmapping on a ridiculously small whiteboard in a terrible sub-leased office and building release binaries on my laptop. That was fun once, but I'm ready for a new phase in my own life. I've heard the horror stories of being acquired by IBM, but I've also heard from people who have reveled in the resources and opportunities. I'm hoping for the best for Nomad, our users, and our team. I'd like to think there's room in the world for multiple schedulers, and if not, it won't be for lack of trying.
Every IBM product I've ever used is universally reviled by every person I've met who also had to use it, without exaggeration in the slightest. If anything, I'm understating it: I make a significant premium on my salary because I'm one of the few people willing to put up with it.
My only expectation here is that I'll finally start weaning myself off terraform, I guess.
During my time at IBM and at other companies a decade ago, I can name examples of this:
* Lotus Notes instead of Microsoft Office.
* Lotus Sametime Connect instead of... well Microsoft's instant messengers suck (MSN, Lync, Skype, Teams)... maybe Slack is one of the few tolerable ones?
* Rational Team Concert instead of Git or even Subversion.
* Rational ClearCase instead of Git ( https://stackoverflow.com/questions/1074580/clearcase-advant... ).
* Using a green-screen terminal emulator on a Windows PC to connect to a mainframe to fill out weekly timesheets for payroll, instead of a web app or something.
I'll concede that I like the Eclipse IDE a lot for Java, which was originally developed at IBM. I don't think the IDE is good for other programming languages or non-programming things like team communication and task management.
https://trends.google.com/trends/explore?date=all&q=terrafor...
Not a product, but a service: is Red Hat Linux a counter example?
Dead Comment
I worked for a company acquired by IBM, and we held hope like you are doing, but it was only a matter of time before the benefit cuts, layoffs, and death of the pre-existing culture.
Your best bet is to quit right after the acquisition and hope they give you a big retention package to stay. These things are pretty common to ease acquisition transitions and the packages can be massive, easily six figures. Then when the package pays out you can leave for good.
none of that has happened for us at Red Hat. Other than the one round of layoffs which occurred at the time that basically every tech company everywhere was doing much larger layoffs, that was pretty much it and there's no reason to think our layoffs wouldn't have been much greater at that time if we were not under the IBM umbrella.
Besides that, I dont even remember when we were acquired, absolutely nothing has changed for us in engineering; we have the same co-workers, still using all Red Hat email / intranets / IT, etc., there's still a healthy promotions pipeline, all of that. I dont even know anyone from the IBM side. We had heard all the horror stories of other companies IBM acquired but for whatever reason, it's not been that way at all for us at least in the engineering group.
Deleted Comment
I know there are horror stories around this acquisition and lots of predictions about what will happen, but only time will tell. On a minimum, it has been a delight to use the Hashicorp software stack along with the approach they brought to our engineering workflow (remember Vagrant?). These innovations and approaches aren't going away.
I used it literally this year to create a test double of the NUC that runs my home automation stack. I also used Packer to configure Flatcar and create the qcow2 that Vagrant consumes.
Vagrant is still the best tool for creating a general purpose VM on your machine. It got kind-of forgotten in the containers and Kubernetes hype, but it still gets the job done. Packer is also the best tool for creating VM images that got buried for the same reasons.
The datacenter is coming back, though. IBM would be smart to invest in these tools as loss leaders to TFE and Vault and monetize the providers, IMO.
There are worse companies to get bought by, but if you've only ever worked at startups then you're not likely to enjoy what this becomes.
When IBM acquired that company, after a few weeks, this guy had a meeting with new engineering people. The very first meeting, they changed things for him. Instead of a single winding road of development, they wrote out a large spreadsheet. The rows were the distinguishable parts of his large'ish and clever architecture; the columns were assignments. They essentially dismantled his world into parts, for a team to implement. He was distraught. He didn't think like that. They did not discuss it, it was the marching orders. He quit shortly afterwards, which might have been optimal for IBM.
Regardless of the general sentiment, hoping for the best outcome for all of you.
I know we aren't perfect. In fact it's my turn this week, and I've utterly failed at keeping up with triage!!
We get tremendous joy and value from engaging with our community. Thanks for your patience (with me in particular!)
I really love these stories. Our customers constantly shock me with how large of clusters they're able to manage with just a few people. >10k nodes by <10 people isn't uncommon although we are not good at rigorously collecting data on this.
Nomad is far from perfect, but we really strive to ease the pains of managing your compute substrate.
No matter what they tell you, your day to day will not improve. For my area, it was mostly business as usual, but a net decrease in comp because IBM's ESPP is trash.
Deleted Comment
I'm not going anywhere! My grammar isn't always the best, so I apologize for the confusion.
Then they did the license change, which didn't reflect well on them.
Now it's being sold to IBM, which is essentially a consulting company trying to pivot to mostly undifferentiated software offerings. So I guess Hashicorp is basically over.
I suspect the various forks will be used for a while.
There have been lifecycle rules in place for as long as I can remember to prevent stuff like this. I'm not sure this is a "problem" unique to terraform.
I find this to be a very strange criticism and is probably indicative of a poor workflow or CI/CD system if anything.
No serious organization with any scale is going to have the only thing standing between them and a production database deletion being two over tired engineers rubber stamping each other's code changes.
Bad config pushes to prod do happen and they can cause outages like the 2024 Cloudstrike outage. You don't want a tool that takes a minor (but significant) error and turns it into a catastrophic one because of poorly thought out semantics. It's better to just start with a tool that requires at least two engineers to explicitly sign off on deletion.
Except for not "feeling" secure, the only thing everyone wants is a Windows AD file share with ACLs.
Just no one realises this: all the Vault on disk encryption and unsealing stuff is irrelevant - it's solving a problem handled at an entirely different level.
Actually for me, the company I was at that IBM purchased was on the verge of folding, so in that case, IBM saved our jobs and I was there for many years.
Now, we are actively hiring for numerous positions.
Personally, I am not planning to stay much longer. I had hoped that our corp structure would be similar to RedHat, but it seems that they intend to fully integrate us into the IBM mothership.
End of an era.
---
[1]: https://blog.webb.page/2018-01-11-why-the-job-search-sucks.t...
If never profitable (or terrible return on equity), why would you call the layoffs "arbitrary"? It seems pretty reasonable to me.
Q: What do you get when you cross Apple and IBM?
A: IBM.
But then the joke was on me when I finally worked for a company owned by Apple and IBM at the same time, and experienced it first hand!
I gave Lou Gerstner a DreamScape [4] demo involving an animated disembodied spinning bouncing eyeball, who commented "That's a bit too right-brained for me." I replied "Oh no, I should have used the other eyeball!"
Later when Sun was shopping itself around, there were rumors that IBM might buy it, so the joke would still apply to them, but it would have been a more dignified death than Oracle ending up lawnmowering [5] Sun, sigh.
Now that Apple's 15 times bigger than IBM, I bet the joke still applies, giving Apple a great reason NOT to merge with IBM.
[1] https://en.wikipedia.org/wiki/Kaleida_Labs
[2] https://en.wikipedia.org/wiki/Taligent
[3] https://en.wikipedia.org/wiki/AIM_alliance
[4] https://www.youtube.com/watch?v=5NytloOy7WM&t=323s
[5] https://news.ycombinator.com/item?id=5170246
I'm a heavy user of Terraform and Vault products. Both do not belong to this era. Also worked for a startup acquired and dumped by IBM.
So do you find Terraform and Vault good or bad? (sorry, not a native English speaker and I had problems to transcript the sentence)
Secrets whatever your cloud provider has (Google secrets manager etc).
What, in this era, replaces provisioning cloudy stuff that doesn't require heaps of YAML or a bootstrap Kubernetes cluster for operators to run within?
Some of this is obvious (linux and mainframes aren't a bad combo). Some of it I'm a bit surprised by (openshift revenue seems strong).
Probably already basically returned purchase price in revenue and much more than purchase price in market cap.
A noticeable thing is
https://www.redhat.com/en
Most of the these type plays the home page has stacked toolbars / marketing / popups / announcements from the parent company and their branding everywhere (IBM XXX powered by Redhat)... I see very little IBM logo or corporate pop-up policy jank on redhat.com.
People who worked at companies acquired by IBM and could not afford going anywhere else.
A mixture of both will be involved from now on in decision making regarding your platform formation core products.