Readit News logoReadit News
smaddox commented on How attention sinks keep language models stable   hanlab.mit.edu/blog/strea... · Posted by u/pr337h4m
smaddox · 20 days ago
> though we did not delve into the observation

Oh, the irony.

smaddox commented on Gold Is So Popular It's Making People Nervous   bloomberg.com/news/newsle... · Posted by u/walterbell
YZF · 4 months ago
There is no quick decoupling from the USD. Everyone holds long term USD instruments. Nobody wants to lose their money. The US is still the world's largest economy - by a lot.

Gold isn't that much more expensive than it was in e.g. 1980. It is pretty much just saying there's risk in other places at the moment and people are nervous. You should buy gold if your prediction is a complete collapse of economies along the lines of the great depression. I think this is extremely unlikely.

Short gold ;)

I've always wanted to get some physical gold just because it's so nice and shiny. But I'll wait for the price to go a little down. Maybe Costco selling gold bars is what's pushing up the price/demand?

smaddox · 4 months ago
Gold/M2SL(Billion USD) is currently around 0.12. In 1980, it peaked around 0.45. Monthly average since 1960 is 0.11. In late 2011 it peaked around 0.18.

Gold / Global M2 would be a better metric, but I haven't analyzed that yet.

smaddox commented on Cursor IDE support hallucinates lockout policy, causes user cancellations   old.reddit.com/r/cursor/c... · Posted by u/scaredpelican
ddxv · 4 months ago
Cursor is weird. They have a basically unused GitHub with a thousand unanswered Issues. It's so buggy in ways that VSCode isn't. I hate it. Also I use it everyday and pay for it.

That's when you know you've captured something, when people hate use your product.

Any real alternatives? I've tried continue and was unimpressed with the tab completion and typing experience (felt like laggy typing on a remote server).

smaddox · 4 months ago
I switched to Windsurf.ai when cursor broke for me. Seems about the same but less buggy. Haven't used it in the last couple weeks, though, so YMMV.
smaddox commented on Electron band structure in germanium, my ass (2001)   pages.cs.wisc.edu/~kovar/... · Posted by u/tux3
smaddox · 5 months ago
For those who are actually interested in this field, the proper way to measure this would be with a four point probe. You do need a constant current source and a high-impedence voltage meter, though.

Also, you don't need to solder wires to the sample. But if you want to measure the hall resistance of a thin film of a semiconductor, you can solder a glob of indium on to four corners of a 1 cm x 1 cm wafer, put it in a magnetic field, and then do basically the same measurement as four point probe, except not inline.

smaddox commented on Ask HN: Is anyone doing anything cool with tiny language models?    · Posted by u/prettyblocks
psyklic · 7 months ago
JetBrains' local single-line autocomplete model is 0.1B (w/ 1536-token context, ~170 lines of code): https://blog.jetbrains.com/blog/2024/04/04/full-line-code-co...

For context, GPT-2-small is 0.124B params (w/ 1024-token context).

smaddox · 7 months ago
You can train that size of a model on ~1 billion tokens in ~3 minutes on a rented 8xH100 80GB node (~$9/hr on Lambda Labs, RunPod io, etc.) using the NanoGPT speed run repo: https://github.com/KellerJordan/modded-nanogpt

For that short of a run, you'll spend more time waiting for the node to come up, downloading the dataset, and compiling the model, though.

smaddox commented on Natrium 'advanced nuclear' power plant wins Wyoming permit   wyofile.com/natrium-advan... · Posted by u/chiffre01
smaddox · 7 months ago
> That application was submitted in March 2024 and is on track for approval in December 2026

Huh? Is this something where there's multiple incremental steps in the process, and that date is just the final approval stamp, or does it actually just take more than 1.5 years?

smaddox commented on Byte Latent Transformer: Patches Scale Better Than Tokens   ai.meta.com/research/publ... · Posted by u/zxexz
modeless · 8 months ago
I really hope this works out. Death to tokenizers!

Interesting that it's a hierarchical structure but only two levels of hierarchy. Stacking more levels seems like an obvious direction for further research.

Note: I posted this comment on another related story[1] and the author replied:

"Author here :), I do think it’s a good direction to look into! That said, aside from it being a bit too much to do at once, you’d also have to be careful about how you distributed your FLOP budget across the hierarchy. With two levels, you can make one level (bytes/local encoder) FLOP efficient and the other (patches/global encoder) FLOP intensive. You’d also need to find a way to group patches into larger units. But ya, there are many directions to go from here!"

[1] https://news.ycombinator.com/item?id=42413430

smaddox · 8 months ago
Agree more levels seems like it could be beneficial. And another Meta paper published a day later shows how that might work: https://ai.meta.com/research/publications/large-concept-mode...

Deleted Comment

smaddox commented on GM exits robotaxi market, will bring Cruise operations in house   cnbc.com/2024/12/10/gm-ha... · Posted by u/atomic128
iknowstuff · 9 months ago
Nah thats old. They now use https://www.sony-semicon.com/files/62/pdf/p-15_IMX490.pdf and they run multiexposure on every frame. Stupid high dynamic range
smaddox · 9 months ago
Oh, interesting. 120db is much better than what I thought possible with CMOS sensors. Thats competetive with the human eye.

Well, that definitely changes my opinion on how feasible/competitive camera-only autonomous driving can be.

smaddox commented on GM exits robotaxi market, will bring Cruise operations in house   cnbc.com/2024/12/10/gm-ha... · Posted by u/atomic128
mensetmanusman · 9 months ago
Apparently they have been surprised at how few photons are required to see for these sensors. They are skipping the image computer vision step and going from photons to car control in as few layers as possible.
smaddox · 9 months ago
Apparently they're using this CMOS sensor: https://www.onsemi.com/products/sensors/image-sensors/ar0136...

It's not an event camera, so it's very much taking images, which are then being processed by computer vision algorithms.

Event cameras seem more viable than CMOS sensors for autonomous vehicle applications in the absence of LIDAR. CMOS dynamic range and response isn't as good as the human eye. LIDAR+CMOS is considerably better in many ways.

u/smaddox

KarmaCake day1728January 15, 2015View Original