I was one of the founders of tfsec (it scanned code for security issues). One of the things I learnt was if we catch issues early, i.e. when the engineer was typing their code, we save a bunch of time.
I was thinking … okay, why not build cloud costs into the code editor. Show the cloud cost impact of the code as the engineers are writing it.
So I spent some weekends and built one right into JetBrains - fully free - keep in mind it is new, might be buggy, so please let me know if you find issues. It is check it out: https://plugins.jetbrains.com/plugin/24761-infracost
I recorded a video too, if you just want to see what it does:
https://www.youtube.com/watch?v=kgfkdmUNzEo
I'd love to get your feedback on this. I want to know if it is helpful, what other cool features we can add to it, and how can we make it better?
Final note - the extension calls our Cloud Pricing API, which holds 4 million prices from AWS, Azure and GCP, so no secrets, credentials etc are touched at all.
If you want to get the same Infracost goodness in your CI/CD, check out https://www.infracost.io/cicd
Think there’s a book called “measure what matters” and the idea is what we measure shapes companies and behaviour. So I’d be very careful about implementing anything like this in my org.
This is tackling one aspect of this - highlighting the cost to the engineer.
As an electrical engineer, we scrounged for every cent on devices when they were being released. A TVS diode can be replaced with a capacitor? That's a cent. A cheaper processor? Several tens of cents. Over ten million devices plus the legacy of your design into the next generations, that's a lot of money. Someone else negotiates the price of the TVS diode and the processor, but that doesn't mean you should be isolated from the cost. A thing that does X and is more expensive is worse than a thing that does X cheaply.
Software engineers spend fractional cents on requests executed thousands of times per second. That's the same scale as the electrical engineering example when you do this.
As far as I can tell, most good software engineers are very cost aware. That doesn't mean they don't do stuff. They just understand the cost of that stuff. This does seem to be a big divide between senior and junior engineers, as well.
This is eventually taken to the extreme and you occasionally see posts here titled something like "how we saved millions by doing X", where the thing they were originally doing was extremely wasteful.
For the last decade, we've also been in a "free money" mode. Where companies were happy to spend money as long as it led to growth. Optimizing for cost wasn't a priority.
That has leaked into electrical engineering as well. There were a few products that shipped full Raspberry pis in an enclosure in the name of velocity. The savings possible there were probably in the dollars, not cents, not to mention the supply chain issues. And yet companies did it in the name of velocity.
Because it's not their job and therefore they don't have the information necessary to know what is or isn't expensive.
Their job is to come up with the best design using common sense when it comes to cost.
Cost of a part isn't the only thing that matters either. Ok, we can save 1 cent if we swap this part out. But now we have to purchase lot sizes that are 10x bigger. How does that impact production? How does that affect operational budgets? Etc.
The engineers engineer and the bean counters count beans. They meet and find a happy place.
Why would you even want that extra workload? The fact that you think it's so easy just goes to show why actual accountants do the accounting, managers do the managing, procurement does the procuring, and operations does the operating.
"I know calculus, accounting is easy!" Sure, if you throw out all of the variables other than unit cost.
Software engineers, especially at startups, are building proofs of concept that they need to iterate on very quickly. Spending significant thought on reducing opex is an absolutely terrible call in the early stages of dev.
If you have a profitable product, or one that would be if you can squeeze costs, then you play the optimization game.
The difference between electrical engineering and software engineering is that software can be updated after it’s deployed. The ability to get all of those gains later changes the strategy completely
Personally, baking this kind of spreadsheet work into a pipeline was a moderate pain in the butt and hazardous to fully automate.
The dev doesn't worry about it unless it is obviously crazy, I.e. above a team budget. If so they sanity check the code.
I wanted to watch the YouTube video but I didn’t have access to audio, and gave up. Video took to long and had too few visual cues - I think many people will do the same.
So you should make a super short gif (10-20 seconds max) that gets the point across extremely fast and visually - no audio.
Post it on Reddit on webdev, you’ll get a shit tonne of people installing this.
I’ve had lots of success with this simple method (sold a personal project with 700,000 users)
Hope this Gif is clearer - https://github.com/infracost/jetbrains-infracost/blob/244ee8...
That said, I've never personally seen any cloud project where there wasn't a business case review that didn't include a review of architectural plans/assumptions and expected cloud service consumption profiling. I think what's different here is that you can help engineers close a feedback loop that is often left open between when a project is planned & scoped and when 1) the landing zone (or data pipelines) is created, and 2) when the app/system is moved from test/QA into production. This could be helpful for the project/product manager, especially, because anything that looks out of whack with the forecast should be escalated promptly.
Going off the other threads of this post, how much engineers should know or care about billing seems to be open for debate; I think an engineer needs to appreciate that often cost is one of the many trade-offs that need to be accounted for
How "smart" is it? Say I make a copy of a large database in TF and redeploy it into a staging env, will it know that the storage is $200 a month?
Also maybe put the supported clouds on the front page? I work multi-cloud so was worried it was only AWS.
If you're using tfvars for your envs, its "smart". With easily inferable env names for tfvars, we will generate estimations for each env.
In the video, I give an example where for the prod env, I turn on multi-az and have a larger storage and the price goes up.
Good suggestion about supported clouds, I'll update!
As a data engineer in a consulting company: this would be a good way to get an idea of cost once we've written infra for projects.
Couple of questions:
1. Is Bicep supported?
2. Any plans for a VS Code plugin? (I just saw in your docs you have a plugin for this, great!)
3. How are you handling pricing of resources that are dependent on consumption?
2. There is a vscode extension - https://marketplace.visualstudio.com/items?itemName=Infracos...
3. That is a challenge, we use usage files, but in the future would like to pull it from the cloud account to be more accurate and the best suggestions. https://www.infracost.io/docs/features/usage_based_resources...
ChatGPT gave me a decent chuck of terraform and running Infracost against it gave me a $515 monthly cost.
If you go to the settings of the plugin and get the absolute path of infracost binary then run
infracost breakdown --path .
in the dir with the terraform, does it give you a breakdown?