First four things on the list are attention
Readit Newsjdeaton commented on Attention Wasn't All We Needed stephendiehl.com/posts/po... · Posted by u/mooreds
jdeaton commented on Programming languages should have a tree traversal primitive blog.tylerglaiel.com/p/pr... · Posted by u/azhenley
Its called recursion
> Doesn't for_tree(...) look a lot nicer and simpler and less error prone than needing to implement a recursive function for each operation you would want to do on a tree?
No it does not
Deleted Comment
jdeaton commented on Sapphire: Rust based package manager for macOS github.com/alexykn/sapphi... · Posted by u/adamnemecek
Consider rebranding to a 4 letter name or even better a 3 letter one.
I know it sounds dumb but uv was smart to go shorter than pip and sapphire feels heavier than brew no matter what it does after typing that.
Yeah i vote it should be rebranded “why”
0.061 standard deviations? Thats like almost nothing?
jdeaton commented on Nvidia adds native Python support to CUDA thenewstack.io/nvidia-fin... · Posted by u/apples2apples
Pytorch???
Maybe shes wearing those noise canceling headphones because of her auditory processing condition and not the other way around??
This seems like basic speculative attribution error- no research here.
jdeaton commented on How to scale your model: A systems view of LLMs on TPUs jax-ml.github.io/scaling-... · Posted by u/mattjjatgoogle
there is limited TPU support in pytorch via torch_xla
Sounds limited
jdeaton commented on How to scale your model: A systems view of LLMs on TPUs jax-ml.github.io/scaling-... · Posted by u/mattjjatgoogle
I'm curious, what does paradigmatic JAX look like? Is there an equivalent of picoGPT [1] for JAX?
yeah it looks exactly like that file but replace "import numpy as np" with "import jax.numpy as np" :)
jdeaton commented on How to scale your model: A systems view of LLMs on TPUs jax-ml.github.io/scaling-... · Posted by u/mattjjatgoogle
Something nice about this guide is that it generally transfers to GPU directly thanks to JAX/XLA.