Readit News logoReadit News

Dead Comment

goldenkey commented on Can generalist foundation models beat special-purpose tuning?   arxiv.org/abs/2311.16452... · Posted by u/wslh
rschiavone · 2 years ago
I'd argue that yours is not a counter argument, but the same thing said by grandparent rephrased. It's obvious that with the same amount of computing power, specialized > generalized. But in 2-5 years, the future generalized models can beat the current specialized ones.
goldenkey · 2 years ago
The future of this is the same as what happened with PCs. The specificity will initially be thrown away, added back later as an accelerator, and eventually, brought back into the fold as an automatically used accelerator. It all comes full circle as optimizations, hinting, tiering, and heuristics.

AI will be used to select the net to automatically load. Nets will be cached, branch predicted etc.

The future of AI software and hardware doesn't yet support the scale we need for this type of generalized AI processor (think CPU but call it an AIPU.)

And no, GPUs aren't an AIPU, we can't even fit whole some of the largest models on these things without running them in pieces. They don't have a higher level language yet, like C, which would compile down to more specific actions after optimizations are borne (not PTX/LLVM/Cuda/OpenCL.)

goldenkey commented on Q-Transformer   qtransformer.github.io/... · Posted by u/jonbaer
brcmthrowaway · 2 years ago
It's clear we're searching for the god algorithm of AI, just like physicists are searching for theory of everything. Are transformers the answer though?
goldenkey · 2 years ago
There's really no God algorithm needed, just something good enough to assist with research of the next tier of hardware, energy, and code for AI.
goldenkey commented on Brickception   brickception.xyz/... · Posted by u/duck
layer8 · 2 years ago
Make it a two-player game by making the paddle of each player the brick wall of the other player. This is also easily extended to N > 2 players.
goldenkey · 2 years ago
Might as well just include everyone's games inside of everyone else's game as nested entities whose microstates determine the above games macrostates.

This is very similar to Reflective Towers Of Interpreters: https://blog.sigplan.org/2021/08/12/reflective-towers-of-int...

goldenkey commented on Lidocaine induces apoptosis in head and neck squamous cell carcinoma   cell.com/cell-reports/ful... · Posted by u/bookofjoe
goldenkey · 2 years ago
Explains a bit or two about bitter almond, doesn't it?
goldenkey commented on Ask HN: Keep binaries in system memory never removed till manually done so    · Posted by u/dogol
kevincox · 2 years ago
Unless you manually lock the executable in memory it will be evicted from RAM just like any other file on the filesystem. When you start up the usually-idle launcher it will load everything from disk (well load what it needs then the new process will load the rest as it starts up). I doubt the performance difference is significant unless your process does lots of work at startup and you will lose out on ASLR by always forking the same memory layout.
goldenkey · 2 years ago
On Windows I believe one can call an API like VirtualProtect to prevent swap/eviction from RAM. Should be possible on Linux too?

Yeah, it'd only really be useful if one was running some type of web service that called an executable, and there is no source code available to do any fastCGI or whatnot.

goldenkey commented on Ask HN: Keep binaries in system memory never removed till manually done so    · Posted by u/dogol
goldenkey · 2 years ago
Best to write a launcher that simply forks an existing process that is in the right initial state. Other folks mention caching the ELF file but that will be slower than the loaded executable being cloned for a new process.
goldenkey commented on Italy bans cultivated meat products   chemistryworld.com/news/i... · Posted by u/Vagantem
robertlagrant · 2 years ago
> they dont treat their citizens like cattle

"We replaced our citizens with plant-based citizen alternatives, as they fart less".

goldenkey · 2 years ago
We've replaced our plant-based citizen alternatives with stars. They twinkle and we like to look at them.
goldenkey commented on Italy bans cultivated meat products   chemistryworld.com/news/i... · Posted by u/Vagantem
fooker · 2 years ago
Cultivated meat is cancer by definition.

It is just as likely that the idea will peter out slowly.

goldenkey · 2 years ago
Factory farmed meat is cancer too. I'd say, save the animals, and give a little toxicity to all the proles who haven't gone unprocessed plant-based yet.

u/goldenkey

KarmaCake day2243October 29, 2013
About
Hardy couldn't have been more wrong about the innocence of pure mathematics. But we live by his legacy. In the sands of time, where do you dine? It better be divine!

email: golden@churchofthought.org

blog: churchofthought.org

View Original