Readit NewsRelated work:
Interpreting Modular Addition in MLPs https://www.lesswrong.com/posts/cbDEjnRheYn38Dpc5/interpreti...
Paper Replication Walkthrough: Reverse-Engineering Modular Addition https://www.neelnanda.io/mechanistic-interpretability/modula...
And more recently, [Language Models Use Trigonometry to Do Addition](https://arxiv.org/abs/2502.00873)
Really interesting, I have been confused about the einsum function before. As a former physicist, I would also like to see the actual tensor notation for the examples. So instead of ij,jk something like $A_i^j B_j^k$ (imagine the math here instead of the LaTeX).
This is a good idea, though one problem is that Einsum notation (as realized in Numpy and Pytorch) doesn't support the notion of co-contravariance, and the site is based on their Einsum notation. I could potentially add the variances for the examples, though that would move away from how the site currently works (where the information about the reduction comes only from the einsum input).
joelburget commented on Underrated reasons to be thankful IV dynomight.net/thanks-4/... · Posted by u/joelburget
A couple of these I'd like references on if anyone happens to have them.
1. "current science suggests that the actual health impact from consuming most types of plastic might well be essentially zero"
2. "the (weak) evidence we have now suggests running strengthens your knees"
The link in the paper to their Java implementation is now broken: does anyone have a current link?