Readit News logoReadit News
marmaduke commented on Titans: Learning to Memorize at Test Time   arxiv.org/abs/2501.00663... · Posted by u/bicepjai
marmaduke · a year ago
similar to RWKV7’s new (sub quadratic) attention mechanism which models key values as v≈kS’ and does an in-context descent on ||v - kS’||^2/2 (where the state matrix S is one attentional head) , explained more by the author here https://raw.githubusercontent.com/BlinkDL/RWKV-LM/main/RWKV-...

and i tried to unpack it a bit here https://wdmn.fr/rank-1-take-on-rwkv7s-in-context-learning/

marmaduke commented on An Introduction to Neural Ordinary Differential Equations [pdf]   diposit.ub.edu/dspace/bit... · Posted by u/gballan
Iwan-Zotow · a year ago
Not as effective as I'd like?
marmaduke · a year ago
ha, yeah, thanks.
marmaduke commented on An Introduction to Neural Ordinary Differential Equations [pdf]   diposit.ub.edu/dspace/bit... · Posted by u/gballan
sitkack · a year ago
marmaduke · a year ago
no, wrote it by hand for use with my own Heun implementation, since it’s for use within stochastic delayed systems.

jax is fun but as effective as i’d like for CPU

marmaduke commented on An Introduction to Neural Ordinary Differential Equations [pdf]   diposit.ub.edu/dspace/bit... · Posted by u/gballan
marmaduke · a year ago
looks like a nice overview. i’ve implemented neural ODEs in Jax for low dimensional problems and it works well, but I keep looking for a good, fast, CPU-first implementation that is good for models that fit in cache and don’t require a GPU or big Torch/TF machinery.
marmaduke commented on C: Simple Defer, Ready to Use   gustedt.wordpress.com/202... · Posted by u/ingve
robertlagrant · a year ago
Tis a joke my friend.
marmaduke · a year ago
i like how the contraction it’s and abbreviation T’is are anagrams
marmaduke commented on Zildjian, a 400-year-old cymbal-making company in Massachusetts   wbur.org/news/2024/12/16/... · Posted by u/arbesman
overkill28 · a year ago
> Carrington founded and directs Berklee College of Music’s Institute of Jazz and Gender Justice in Boston. She’s also a Zildjian artist

What do you think they study at the Institute of Jazz and Gender Justice ?

marmaduke · a year ago
I didn’t look it up, but at first glance, it reminded me of discussions like this one

https://youtu.be/qnT48wO0UL0

marmaduke commented on An Unreasonable Amount of Time   allenpike.com/2024/an-unr... · Posted by u/memalign
notnaut · a year ago
Practice now, realize later, feels deeply dualistic to me.
marmaduke · a year ago
that’s not what the story says. in any case, the point is to explain, in terms of dualistic if-then logic, that the if (you practice now) and then (you will wake up) are a single non-dual thing. but to communicate in in terms which make sense to the dual, if-then mind, one needs to use dualistic language.
marmaduke commented on Ask HN: Are you unable to find employment?    · Posted by u/vbi8iBEX
w4ffl35 · a year ago
Very far. I'm a fullstack web developer. Independent game dev has been my hobby for 10 years, and game dev is what got me interested in tech when I was a kid. I started teaching myself at a young age with qbasic.

Building interfaces and menu systems etc feels very similar to frontend web development. Much of the domain knowledge transfers when it comes to programming. Mainly my skills transfer to gameplay development and debugging. I make small games and only just recently released my first commercial game, which has only sold two copies.

marmaduke · a year ago
do you have a link for your game? how was the release process?
marmaduke commented on Ask HN: Are you unable to find employment?    · Posted by u/vbi8iBEX
w4ffl35 · a year ago
Terrible. I've turned to making video games while my wife works. Rough times.
marmaduke · a year ago
how far from your previous experience is the game work?
marmaduke commented on A Practitioner's Guide to Wide Events   jeremymorrell.dev/blog/a-... · Posted by u/dmazin
marmaduke · a year ago
i wonder if there are any semi automated approaches to finding outliers or “things worth investigating” in these traces, or is it just eyeballs all the way down?

u/marmaduke

KarmaCake day2806April 10, 2012
About
[ my public key: https://keybase.io/wdmn; my proof: https://keybase.io/wdmn/sigs/KaH_BqIpR_cc-JNsLgWnzWCnKy_ZnbQfoKTC0ms3-1o ]
View Original