joelburget (u/joelburget)

joelburget commented on “A calculator app? Anyone could make that” chadnauseam.com/coding/ra... · Posted by u/pie_flavor

joelburget · a year ago

I wrote an OCaml implementation of this paper a few years ago, which I've now extracted into its own [repo](https://github.com/joelburget/constructive-reals/blob/main/C...)

The link in the paper to their Java implementation is now broken: does anyone have a current link?

joelburget commented on Pre-Trained Large Language Models Use Fourier Features for Addition (2024) arxiv.org/abs/2406.03445... · Posted by u/Kye

bkitano19 · a year ago

Related work:

Interpreting Modular Addition in MLPs https://www.lesswrong.com/posts/cbDEjnRheYn38Dpc5/interpreti...

Paper Replication Walkthrough: Reverse-Engineering Modular Addition https://www.neelnanda.io/mechanistic-interpretability/modula...

joelburget · a year ago

And more recently, [Language Models Use Trigonometry to Do Addition](https://arxiv.org/abs/2502.00873)

joelburget commented on Einsum in Depth einsum.joelburget.com/... · Posted by u/joelburget

captainmuon · a year ago

Really interesting, I have been confused about the einsum function before. As a former physicist, I would also like to see the actual tensor notation for the examples. So instead of ij,jk something like $A_i^j B_j^k$ (imagine the math here instead of the LaTeX).

joelburget · a year ago

This is a good idea, though one problem is that Einsum notation (as realized in Numpy and Pytorch) doesn't support the notion of co-contravariance, and the site is based on their Einsum notation. I could potentially add the variances for the examples, though that would move away from how the site currently works (where the information about the reduction comes only from the einsum input).

joelburget commented on Underrated reasons to be thankful IV dynomight.net/thanks-4/... · Posted by u/joelburget

joelburget · a year ago

A couple of these I'd like references on if anyone happens to have them.

1. "current science suggests that the actual health impact from consuming most types of plastic might well be essentially zero"

2. "the (weak) evidence we have now suggests running strengthens your knees"

u/joelburget

KarmaCake day694September 8, 2010

About

Building programming tools. Living in Portland. Former Khan Academy and Recurser (Hacker Schooler).

https://joelburget.com

https://twitter.com/joel_burget

https://github.com/joelburget

View Original