new attention mechanisms also often need new kernels to run at any reasonable rate
theres definitely a breed of frontend-only ML dev that dominates the space, but a lot novel exploration needs new kernels
It's funny how functional programming is slowly becoming the best practice for modern code (pure functions, no side-effects), yet functional programming languages are still considered fringe tech for some reason.
If you want a language where const is the default and mutable is a keyword, try F# for starters. I switched and never looked back.
dropping down into the familiar or the simple or the dumb is so innately necessary in the building process. many things meant to be "pure" tend to also be restrictive in that regard.
Happy to answer any questions