g413n (u/g413n) - Readit News

g413n commented on The First Fully General Computer Action Model si.inc/posts/fdm1/... · Posted by u/nee1r

kdrag0n · 19 days ago

what tasks can the model do out of the box? was each of the examples a different fine tuned model?

g413n · 19 days ago

it's a pretty general policy but this is all super early, it's great at exploring websites so fuzzing was easy, for CAD it has good enough base rates with the few-shot prompt when we do the repetitive stuff, and we gave it checkpoints on each step, the other stuff in the mosaic are just some of our favorite clips from internal evals

g413n commented on The First Fully General Computer Action Model si.inc/posts/fdm1/... · Posted by u/nee1r

ClaireBookworm · 19 days ago

What sort of fine tuning data was needed to allow the model to self-drive? One hour of video of someone driving, or extra labeling?

g413n · 19 days ago

relevant note is that we finetuned by having the human also use arrow keys which keeps it in-distribution but also slower to collect

g413n commented on The First Fully General Computer Action Model si.inc/posts/fdm1/... · Posted by u/nee1r

clemvonstengel · 19 days ago

I rly liked the point about ctrl-c only being able to be labelled retrocausally. I do think that with enough past context you should be able to know what was copied - in some sense the past does encode the future - but also an agentic decision is precisely the kind where the future is more informative than the past for reconstructing that decision.

It does make me wonder if you should have the inverse dynamics model split into specifically retrocausal and causal. You kind of do this already with the inverse and forward dynamics model, but the idea of a model that knows only about the future training in a feedback loop with a model that knows only about the past is kind of interesting.

I think you could just do a clever masking regime in your diffusion model to achieve the same effect without a whole architecture change.

g413n · 19 days ago

yeah we actually had some wacky ideas with ctc + a reverse-causal mask but diffusion does just make it all a bit more simple

g413n commented on The First Fully General Computer Action Model si.inc/posts/fdm1/... · Posted by u/nee1r

ennucore · 19 days ago

How do you tokenize the mouse inputs?

g413n · 19 days ago

we do exponential binning but fwiw I think we can do way better just hasn't been the main research area initially

g413n commented on The First Fully General Computer Action Model si.inc/posts/fdm1/... · Posted by u/nee1r

ennucore · 19 days ago

The car thing is very impressive By the way, do you have plans to handle the computer’s audio output?

g413n · 19 days ago

yeah we've done audio work in the past so we'll def merge the recipes at some point, long term should have full io that a human has (except maybe not generating video for video calls that seems a bit much)

g413n commented on Pantograph: Building a preschool for robots pantograph.com/blog/build... · Posted by u/agajews

g413n · 3 months ago

so cool :)

g413n commented on We collected 10k hours of neuro-language data in our basement condu.it/thought/10k-hour... · Posted by u/nee1r

g413n · 3 months ago

what's the basis for conversion between hours of neural data to number of tokens? is that counting the paired text tokens?

g413n commented on Building the heap: racking 30 petabytes of hard drives for pretraining si.inc/posts/the-heap/... · Posted by u/nee1r

Symbiote · 5 months ago

I understood that it's optional because they can walk down the road to the data center instead.

They mention plugging monitors in several times. I think I've only done that once in the last couple of years, when a firmware upgrade failed and reset the management interface IP.

g413n · 5 months ago

yep this. we just turned off management

g413n commented on Building the heap: racking 30 petabytes of hard drives for pretraining si.inc/posts/the-heap/... · Posted by u/nee1r

twoodfin · 5 months ago

If this is a real market, I’d expect AWS to introduce S3 Junkyard with a similar durability and cost structure.

They probably still won’t budge on the egress fees.

g413n · 5 months ago

we would be so down to buy s3 junkyard tbh we were going around begging various storage clouds to offer us this before giving up and building it ourselves

g413n commented on Building the heap: racking 30 petabytes of hard drives for pretraining si.inc/posts/the-heap/... · Posted by u/nee1r

toast0 · 5 months ago

My info may be dated, but power density has gone up a ton over time. I'd expect a lot of datacenters to have plenty of space, but not much power. You can only retrofit so much additional power distribution and cooling into a building designed for much less power density.

g413n · 5 months ago

yep this was the case for us.