Is this Triton's reply to NVIDIA's tilus[1]. Tilus is suposed to be lower level (e.g. you have control over registers). NVIDIA really does not want the CUDA ecosystem to move to Triton as Triton also supports AMD and other accelerators. So with Gluon you get access to lower level features and you can stay within Triton ecosystem.
There's a lot of pressure on the CUDA ecosystem at this point:
- most of the trillion dollar companies have their own chips with AI features (Apple, Google, MS, Amazon, etc.). Gpus and AI training are among their biggest incentives. They are super motivated to not donate major chunks of their revenue to nvidia.
- Mac users don't generally use nvidia anymore with their mac hardware and the apple's CPUs are a popular platform for doing stuff with AI.
- AMD, Intel and other manufacturers want in on the action
- The Chinese and others are facing export restrictions for Nvidia's GPUs.
- Platforms like mojo (a natively compiled python with some additional language features for AI) and others are getting traction.
- A lot of the popular AI libraries support things other than Nvidia at this point.
This just adds to that. Nvidia might have to open up CUDA to stay relevant. They do have a performance advantage. But forcing people to chose, inevitably leads to plenty of choice being available to users. And the more users choose differently the less relevant CUDA becomes.
It sounds like they share that goal. Gluon is a thing because the Triton team realized over the last few months that Blackwell is a significant departure from the Hopper, and achieving >80% SoL kernels is becoming intractable as the triton middle-end simply can't keep up.
it feels like Nvidia has 30 "tile-based DSLs with python-like syntax for ML kernels" that are in the works lol. I think they are very worried about open source and portable alternatives to cuda.
The fact that the "language" is still Python code which has to be traced in some way is a bit off-putting. It feels a bit hacky. I'd rather a separate compiler, honestly.
Mojo for python syntax without the ast walking decorator, cuda for c++ syntax over controlling the machine, ah hoc code generators writing mlir for data driven parametric approaches. The design space is filling out over time.
This is pretty common among these ml toolchain, and not a big deal. They use pythons ast lib and the function annotations to implement an ast walker and code generator. It works quite well.
Interesting, i can see this being very similar to Nvidia's CUTE DSL. This hints that we are converging to a (local) optimal design for Python-based DSL kernel programming.
Readit News
[1] https://github.com/NVIDIA/tilus
- most of the trillion dollar companies have their own chips with AI features (Apple, Google, MS, Amazon, etc.). Gpus and AI training are among their biggest incentives. They are super motivated to not donate major chunks of their revenue to nvidia.
- Mac users don't generally use nvidia anymore with their mac hardware and the apple's CPUs are a popular platform for doing stuff with AI.
- AMD, Intel and other manufacturers want in on the action
- The Chinese and others are facing export restrictions for Nvidia's GPUs.
- Platforms like mojo (a natively compiled python with some additional language features for AI) and others are getting traction.
- A lot of the popular AI libraries support things other than Nvidia at this point.
This just adds to that. Nvidia might have to open up CUDA to stay relevant. They do have a performance advantage. But forcing people to chose, inevitably leads to plenty of choice being available to users. And the more users choose differently the less relevant CUDA becomes.
Dead Comment
Some more info in this issue: https://github.com/triton-lang/triton/issues/7392
[1]: https://docs.nvidia.com/cutlass/media/docs/pythonDSL/cute_ds...
autogluon is popular as well: https://github.com/autogluon/autogluon
Do any of y’all have clear ideas about why it is that way? Why not have a really great bespoke language?
Is there a big reason why Triton is considered a "failure"?