Readit News logoReadit News
seeyam commented on Understanding Airflow DAG and Task Concurrency on Google Cloud Composer   cloud.google.com/blog/pro... · Posted by u/meysamazad
ibash · 2 years ago
I setup airflow for production at a consulting gig years ago.

Airflow is an abstraction and a good abstraction.

I don’t understand why these blog posts try to make it seem more complex and confusing than it actually is…

Is it trying to justify the cost of a managed service? Or something else?

seeyam · 2 years ago
At scale it's probably to ensure your environment is capable of handling as many DAGs as possible. also as a data engineer, not knowing why a task isn't running can be frustrating. so this might help those understand and resolve

u/seeyam

KarmaCake day18July 29, 2024View Original