Exactly. LLMs are trained on huge amounts of bash scripts. They “know” how to use grep/awk/whatever. ASTs are, I assume, not really part of that training data. How would they know how to work well with on? LLMs are trained on what humans do to code. Yes, I assume down the road someone will train more efficient versions that can work more closely with the machine. But LLMs work as well as they do because they have a large body of “sed” statements in their statistical models
treesitter is more or less a universal AST parser you can run queries against. Writing queries against an AST that you incrementally rebuild is massively more powerful and precise in generating the correct context than manually writing infinitely many shell pipeline oneliners and correctly handling all of the edge cases.
For the uninitiated, data lakes are used to centralize vast quantities of data - often consumer data - usually by large organizations and governments to provide insights and inform decisions within the organization. Some call this surveillance capitalism.
Cloudera IPO'd somewhere around $2B and was taken private in a deal led by KKR for around $5B.