carty7 (u/carty7) - Readit News

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

isoprophlex · a year ago

I think it's perfectly valid to voice an opinion of shallowness. That doesn't automatically make it trolling.

carty7 · a year ago

My mistake. Maybe the wrong use of the word.

And yes, it was a shallow article. It was not a deep dive and nor do I have the technical chops to write a deep dive.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

based2 · a year ago

alt

    - root
     -- /envs
     --- / .dev.env
     --- / .test.env
     --- / .prod.env
     --- / dev.tfvars
     --- / test.tfvars
     --- / prod.tfvars
     - 1_create_network.tf
     - 2_create_storage.tf
     - 3_create_service.tf

https://www.youtube.com/watch?v=WgPQ-nm_ers Compliance At Scale: Hardened Terraform Modules at Morgan Stanley

https://www.weekly.tf/

carty7 · a year ago

Thanks for sharing. I’ll add the video to my queue.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

OJFord · a year ago

This is terrible. All options seem to assume state is stored locally (not, say, in S3). Many options are just the same as others but only one environment, or with some project-specific difference like 'there is a backend and a frontend' or 'a few services' which has nothing to do with how you structure the terraform side of it.

All of them either don't address or use multiple directories to handle multiple environments (and assume a static list). What?! No, use terraform workspaces, or a single module that you instantiate for each environment. Or terragrunt if you really want. TFA is just a blogspam mess from either AI or someone that's spent about 20 minutes with a Youtube video to learn terraform & push out their latest deep dive content guys.

carty7 · a year ago

The entirety of this research was about structure of directories.

Storing state in S3 or TFC or Spacelift or somewhere else is out of scope. S3 is where 90% of the world stores their state and writing those configuration lines is not in scope. You can find other resources on that.

I struggled to find an exhaustive list of how people manage their directory structures and hence the focus of this piece.

If you’d like to provide constructive feedback and avoid comments regarding scope creep, please share.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

unop · a year ago

Nice! We'll link to this for our internal consultancy work.

It'd be nice to show the other dimension of the git branching strategies to apply. Github flow/feature-branches vs per-env branches of main vs git flow. How and when to apply changes in different environments - before vs after PRs, etc.

carty7 · a year ago

This was out of scope for my research. Have you seen any good resources on this?

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

bilekas · a year ago

The method I go to almost always is

> Multi-Environment Setup with Shared Modules

But the con of saying versioning is tricky across modules is damn near impossible to reliably manage.. especially because if I'm introducing a new variable to a shared module A) I need to also add this variable in the inputs of each of the environment.

I haven't found a way to manage multiple versions of the modules across environments if all using the same shared modules. Is it even possible?

carty7 · a year ago

Thank you for sharing. I’ve heard this module conundrum a few times and creates a proliferation of forked modules.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

lijok · a year ago

This is ever evolving but we currently do;

Composable modules such as `terraform-aws-lambda/modules/standard-function`, `terraform-aws-iam/moduls/role-for-aws-lambda`, etc, which get composed for a specific usecase in a root module (which we call stacks). The stack has directories under it such as `dev/main/primary/` `dev/sandbox-a/primary/` `dev/sandbox-a/test-a/`, etc, where `dev` is environment, `main/sandbox-a` is tenant and the `primary/test-a` is the namespace. The namespaces contain a `tfvars` file and potentially some namespace specific assets, readme's, documentation, etc. The CD system then deploys the root module for each namespace present.

Stacks are then optionally (sometimes deeply) nested under parent directories, which are used for change control purposes, variable inference and consistency testing.

OpenTofu >1.8.0 is required for all of this to keep it nice and tidy.

carty7 · a year ago

Thank you for sharing your example.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

bloopernova · a year ago

The client I'm contracted to is all-in on Terraform Cloud. (TFC)

TFC uses workspaces, which annoyingly aren't the same thing as terraform workspaces. I've divided up our workspaces into dev, qa, staging, and prod, and each group of workspaces has the OIDC setup to allow management of a specific cloud account. So dev workspaces can only access the dev account, etc etc. Each grouping of workspaces also has a specific role that can access them. Each role then has its own API key.

The issues I've run into are mostly management of workspace variables. So now I have a manager repo and matching workspace that controls all the vars for the couple hundred TFC workspaces. I use a TFC group API key for the terraform enterprise provider, one provider per group. This prevents potential mistakes where dev vars could get written to qa, etc etc.

  Manager repo
  - dev TFE provider
  - qa TFE provider
  - staging TFE provider
  - prod TFE provider

Workspace variables are set by a single directory of terraform, so there's good sharing of the data and locals blocks.

I use lists of workspaces categorized by "pipeline deployers" and "application resource deployers", along with lists of dev, qa, staging, and prod workspaces. I then use terraform's "setintersection" function to give me "dev pipeline" workspaces, "prod app" workspaces, etc. I also do the same with groups of variables, as there's some that are specific to pipeline workspaces, and so on. It works well, and it's nice to have an almost 100% terraform control of vars and workspaces.

I split app and pipeline workspaces based on historical decisions, I'm not sure if I'd replicate that on a new project. The workflow there is that an app workspace creates the resources for a given deployment, then saves pertinent details to a couple of parameters. The pipeline workspace then pulls those parameters and uses them to create a pipeline that builds and deploys the code.

Unfortunately I can't share code from this particular setup, but I do intend to write about it "someday".

carty7 · a year ago

Thank you. I’ll dive into this when I have a chance next week.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

bloopernova · a year ago

Agreed on the quality of the article, it feels very shallow.

carty7 · a year ago

Sorry it feels shallow and that you feel the need to troll.

If you read my first post related to this, I was giving myself a refresher to understand different dynamics that people think about.

I did not watch one YouTube video or spend 20 minutes on this or create with GPT.

The original source of inspiration came from me wanting to understand the examples our Eng team put together on how our config file correlates to what customers are actually using to find any gaps.

https://docs.resourcely.io/concepts/other-features-and-setti...

This is also a part 1 of the article and I clearly asked what was missing.

carty7 commented on Terraform Config Root Setups resourcely.io/post/10-ter... · Posted by u/carty7

carty7 · a year ago

I spend a lot of time speaking with clients and have found myself partially understanding organizational structure so I dove in to collect my thoughts and put myself closer to the customer on what they are navigating.

This gave myself a refresher on how they are organizing their cloud infrastructure within their source control systems. I took a lense from the world of terraform since that’s mostly the world i live in today and the last few years.

I explored 10 different ways to structure your Terraform config roots, each promising scalability but delivering varying degrees of chaos. From single-environment simplicity to multi-cloud madness, customers are stuck navigating spaghetti directories and state file hell.

I probably missed things. Might have gotten things wrong. Take a look and let me know what you think.

What patterns are you using that I missed?