twotwotwo (u/twotwotwo)

twotwotwo commented on Hacking Diffusion into Qwen3 for the Arc Challenge matthewnewton.com/blog/ar... · Posted by u/mattnewton

twotwotwo · 17 days ago

It is kind of wild that most coding tasks are editing tasks, and we humans care a lot about code editing tools, but automated tools use code generation for editing where a valid block must be generated top-to-bottom in one go.

Fixing a mistake requires re-generating the file or block of code. Or, if something generated later has implications for earlier code--a new import or function parameter's required, something like that--the only option is to go back and re-generate a big chunk. That'd be inefficient for humans, not implausible it's wrong for other code generators too.

I don't know if diffusion specifically will be the approach. (Maybe there's something to generating edit sequences?) This post's note that diffusion kills KV caching is something I hadn't even considered. It does seem right to experiment with things other than strict start-to-end generation.

twotwotwo commented on At a Loss for Words: A flawed idea is teaching kids to be poor readers (2019) apmreports.org/episode/20... · Posted by u/Akronymus

trane_project · 23 days ago

So direct instruction (the philosophy behind this book) has been shown to only have modest gains compared to the best interventions, which have more than double the effect size.

It works fine (not the best) for kids with no reading difficulties, but it completely lacks the understanding and the tasks that fix phonemic deficits, the actual source of most reading difficulties.

It's not entirely a bad book, but won't be of too much use for kids with reading difficulties. Since it's only a few bucks, it's not a bad investment. Just be aware of its limitations. If your kid is not developing fluent and effortless reading (not just decoding), you will need to use a method that is aware of how to fix phonemic deficits.

See my other comments in this page for more.

twotwotwo · 23 days ago

trane_project is selling a $20/mo subscription or $1000 perpetual license to their own reading program and folks should read this and their other comments aware of that context. It's disappointing to tell a personal story, come back, and see it was someone's jumping-off point for just slightly indirect self-promotion.

twotwotwo commented on At a Loss for Words: A flawed idea is teaching kids to be poor readers (2019) apmreports.org/episode/20... · Posted by u/Akronymus

mindcrime · 24 days ago

I don't know that my personal n=1 anecdote adds much to this discussion, but FWIW...

My mom taught me to read when I was young (pre kindergarten), but as far as I know she wasn't specifically trying to teach me to read. She just read to me a lot, where I could see the page she was reading from. Mostly she read me comic books. I loved the DC characters back then - Batman, Superman, Wonder Woman, Flash, Aquaman, Green Lantern, etc. and so she read me that stuff many many times. I mean, yeah, I had some of those "Little Golden Books" and stuff around as well, although I don't pointedly remember reading those the way I do the comic books. Anyway, she did all that and when I started kindergarten at 4 (due to being a summer baby) I was already reading. And then stayed well above my grade level on the reading tests all through school.

So I dunno. Maybe it was dumb luck that things worked out that way for me. Maybe there is a genetic element. Or maybe more than anything what mom conveyed to me was a passion for reading (she was a very avid reader herself). Maybe part of it was just that there were always plenty of books around the house and so reading felt like a very natural thing to do. Or maybe it was that whole Pizza Hut BOOK IT thing they had back in the day. Who knows?

In either case, I feel very fortunate in this regard, as reading has remained a big part of my life ever since, and still is to this day.

twotwotwo · 23 days ago

For what it's worth, a pivotal moment for keeping reading going after the lessons was when my partner picked up a comic book at a library event. For a few weeks after the end of the lessons, reading time had been traditional early readers and some of the books we'd previously read to them--even with us offering rewards, there had been ups and downs. As soon as kid started that comic, though, they were pushing right through our protests that it was bedtime, and chewing through the whole series. Luckily we managed to find another series to start before running out of the first one. As parents we can nudge or put stuff on the menu but but kid is pretty much in the driver's seat about what to read next.

In retrospect, of course! The kid just hadn't liked reading those books and things took off once we found stuff they liked. Best first readers are whatever your kid actually wants to read!

twotwotwo commented on At a Loss for Words: A flawed idea is teaching kids to be poor readers (2019) apmreports.org/episode/20... · Posted by u/Akronymus

twotwotwo · 24 days ago

For any parents of small kids here, I have to mention the book Teach Your Child to Read in 100 Easy Lessons. We went through it while my kid was in kindergarten, and after that, I absolutely believe what I've heard from parents who did it successfully a bit earlier. And it didn't prevent my kid from figuring out how to use context or recognize full words. Reading English is a lot, and kids are resourceful; if we teach the 'slow' but reliable way to read, they'll be happy to feel out shortcuts.

The toughest thing was getting a reliable bit of time each day to sit down and do it. Routine, cajoling, and rewards were all involved. So was keeping it lighthearted; the kid has to be on board! Each lesson has straightforward exercises then a brief story, very short at first, longer later in the book. We'd do the exercises and one read of the story, then kid would read the story to my partner. We started in September, and I remember by Halloween the kid was reading candy wrappers. After finishing it, the next big thing was finding stories the kid genuinely liked to keep it going. Continuing to read together after the lessons ended helped: for a while, kids will keep running into lots of new exceptions to the usual rules, etc.

English spelling and pronunciation are a lot, and the book is also, implicitly, a catalog of the tricks English plays on kids and other learners. Part of the book uses a semi-phonetic alphabet where e.g. ee and sh/ch/th have distinct glyphs, but it all still looks enough like English that the jump to regular writing later in the book is doable for the kid. Even with that alphabet, the book has to teach common words like "is" and "was" as exceptions (with s sounding like z). Decades later one can forget little kids deal with all this and eventually handle it like second nature.

The book's originator thought that you could teach math with a broadly similar approach--breaking things down into very small steps and practicing them in isolation then in larger tasks--and doing that was part of his career, but I haven't found similar teach-your-kid book for arithmetic/basic math. If such a book did exist I'd've given it a try!

twotwotwo commented on Developing our position on AI recurse.com/blog/191-deve... · Posted by u/jakelazaroff

entaloneralie · a month ago

I feel like John Holt, author of Unschooling, who is quoted numerous times in the article, would not be too keen on seeing his name in a post legitimizes a technology that uses inevitabilism to insert itself in all domains of life.

--

"Technology Review," the magazine of MIT, ran a short article in January called "Housebreaking the Software" by Robert Cowen, science editor of the "Christian Science Monitor," in which he very sensibly said: "The general-purpose home computer for the average user has not yet arrived.

Neither the software nor the information services accessible via telephone are yet good enough to justify such a purchase unless there is a specialized need. Thus, if you have the cash for a home computer but no clear need for one yet, you would be better advised to put it in liquid investment for two or three more years." But in the next paragraph he says "Those who would stand aside from this revolution will, by this decade's end, find themselves as much of an anachronism as those who yearn for the good old one-horse shay." This is mostly just hot air.

What does it mean to be an anachronism? Am I one because I don't own a car or a TV? Is something bad supposed to happen to me because of that? What about the horse and buggy Amish? They are, as a group, the most successful farmers in the country, everywhere buying up farms that up-to-date high-tech farmers have had to sell because they couldn't pay the interest on the money they had to borrow to buy the fancy equipment.

Perhaps what Mr. Cowen is trying to say is that if I don't learn how to run the computers of 1982, I won't be able later, even if I want to, to learn to run the computers of 1990. Nonsense! Knowing how to run a 1982 computer will have little or nothing to do with knowing how to run a 1990 computer. And what about the children now being born and yet to be born? When they get old enough, they will, if they feel like it, learn to run the computers of the 1990s.

Well, if they can, then if I want to, I can. From being mostly meaningless, or, where meaningful, mostly wrong, these very typical words by Mr. Cowen are in method and intent exactly like all those ads that tell us that if we don't buy this deodorant or detergent or gadget or whatever, everyone else, even our friends, will despise, mock, and shun us the advertising industry's attack on the fragile self-esteem of millions of people. This using of people's fear to sell them things is destructive and morally disgusting.

The fact that the computer industry and its salesmen and prophets have taken this approach is the best reason in the world for being very skeptical of anything they say. Clever they may be, but they are mostly not to be trusted. What they want above all is not to make a better world, but to join the big list of computer millionaires.

A computer is, after all, not a revolution or a way of life but a tool, like a pen or wrench or typewriter or car. A good reason for buying and using a tool is that with it we can do something that we want or need to do better than we used to do it. A bad reason for buying a tool is just to have it, in which case it becomes, not a tool, but a toy.

On Computers Growing Without Schooling #29 September 1982

by John Holt.

twotwotwo · a month ago

It was insightful of him and there absolutely are similar strains of hype today!

He also both points out the silliness of the pitch for the hype-y tech of the day and calls that same tech a tool some might use with good reason.

That doesn't read to me like "tech hyped as inevitable is never okay" but more like "ignore sales pitches and evaluate the hyped tech for yourself." Your own judgment of today's hyped tech might be "kill it with fire." Still, it seems fully in sync with the spirit of that passage for Recurse to ask for a bunch of smart people's varied opinions on it and share them with the world.

twotwotwo commented on Local LLMs versus offline Wikipedia evanhahn.com/local-llms-v... · Posted by u/EvanHahn

twotwotwo · a month ago

The "they do different things" bullet is worth expanding.

Wikipedia, arXiv dumps, open-source code you download, etc. have code that runs and information that, whatever its flaws, is usually not guessed. It's also cheap to search, and often ready-made for something--FOSS apps are runnable, wiki will introduce or survey a topic, and so on.

LLMs, smaller ones especially, will make stuff up, but can try to take questions that aren't clean keyword searches, and theoretically make some tasks qualitatively easier: one could read through a mountain of raw info for the response to a question, say.

The scenario in the original quote is too ambitious for me to really think about now, but just thinking about coding offline for a spell, I imagine having a better time calling into existing libraries for whatever I can rather than trying to rebuild them, even assuming a good coding assistant. Maybe there's an analogy with non-coding tasks?

A blind spot: I have no real experience with local models; I don't have any hardware that can run 'em well. Just going by public benchmarks like Aider's it appears ones like Qwen3 32B can handle some coding, so figure I should assume there's some use there.

twotwotwo commented on The Capacity, Performance, and Reliability of MicroSD Cards bahjeez.com/the-great-mic... · Posted by u/userbinator

twotwotwo · 2 months ago

Flash endurance gets a sort of odd amount of discussion on the Internet, and this adds to the small amount of public data from running lots of Flash devices to exhaustion. (MicroSD != SSD, of course, but it's something.)

Seems possible that by charting it, you could find a "bathtub curve" (early failures/steady state/failures due to use) of failures, probably more than one curve breaking up by category--tiny "industrial" cards made for endurance are unsurprisingly surviving the longest, and counterfeits and (some but not all!) unknown brands are at the bottom. It would be interesting to also see the data on cards that haven't failed yet, i.e. how many write cycles they've survived so far.

(You can sort the table at https://www.bahjeez.com/the-great-microsd-card-survey/all-mi... by cycles until first error to see what I mean. Love the Bart Simpson card coming in at #8.)

One thing the reported averages already show, which is more about reliability stats in general than about Flash, is that the average write cycles survived is way higher than, say, the cycle count after which you'd see 5% of devices fail. The lower "n% will fail" number might be what informs the TBW spec on the box. So if you're able to handle failures gracefully and run drives 'til they drop, that probably adds substantially to how long you can run each device.

twotwotwo commented on What 'Project Hail Mary' teaches us about the PlanetScale vs. Neon debate blog.alexoglou.com/posts/... · Posted by u/konsalexee

crims0n · 2 months ago

Tangentially related, is this book worth the hype? I don't read a lot of genre fiction, but don't like to miss out on the exceptional (just finished and loved Flowers for Algernon, as an example).

Edit: Sounds like an enjoyable, low commitment book. Will give it a try, thanks for the feedback.

twotwotwo · 2 months ago

I give credit to Andy Weir for knowing what The Martian did well--setting up a bunch of technical problems as load-bearing elements of a plot--and going and executing that same general plan, but with new particulars and bigger and with fun new ideas. The made-up sciencey stuff feels infused with principled ideas about how new things we haven't discovered might work, rather than designed for their role in the story alone. And he's willing to write an ending!

There are things he does not stand out at, but those don't take you out of the story. As people work through things on Earth a lot of the nontechnical parts are, I guess, simplified, but I can't care that much; I didn't pick this up wanting a bureaucratic or psychological thriller. And he (or he + early readers and editors) usually make sure to quickly and efficiently get you through all of that to the next fun part.

twotwotwo commented on Why JPEGs still rule the web (2024) spectrum.ieee.org/jpeg-im... · Posted by u/purpleko

imageformatssux · 2 months ago

How in the world do people store images / photos nowadays?

Just as there is a clear winner for video - av1 - there seems to be nothing in the way of "this is clearly the future, at least for the next few years" when it comes to encoding images.

JPEG is... old, and it shows. The filesizes are a bit bloated, which isn't really a huge problem with modern storage, but the quality isn't great.

JPEG-XL seemed like the next logical step until Google took their toys and killed it despite already having the support in Chrome, which pretty much makes it dead in the water (don't you just love monopolies making decisions for you?)

HEIC is good, as long as you pinky promise to never ever leave Apple's ecosystem, ie HEIC sucks.

AVIF seems computationally expensive and the support is pretty spotty - 8bit yuv420 might work, but 10b or yuv444 often doesn't. Windows 10 also chokes pretty hard on it.

Alternatives like WebP might be good for browsers but are nigh-unworkable on desktops, support is very spotty.

PNG is cheap and support is ubiquitous but filesizes become sky-high very quick.

So what's left? I have a whole bunch of .HEIC photos and I'd really like if Windows Explorer didn't freeze for literal minutes when I open a folder with them. Is jpeg still the only good option? Or is encoding everything in jpeg-xl or avif + praying things get better in the future a reasonable bet?

twotwotwo · 2 months ago

I recognize it as beating a dead horse now, but JPEG XL did what was needed to be actually adopted. AVIF has not been widely adopted given the difficulty of a leap to a new format in general and the computational cost of encoding AVIF specifically.

One of JPEG XL's best ideas was incorporating Brunsli, lossless recompression for existing JPEGs (like Dropbox's Lepton which I think might've been talked about earlier). It's not as much of a space win as a whole new format, but it's computationally cheap and much easier to just roll out today. There was even an idea of supporting it as a Content-Encoding, so a right-click and save would get you an OG .jpg avoiding the whole "what the heck is a WebP?" problem. (You might still be able to do something like this in a ServiceWorker, but capped at wasm speeds of course.) Combine it with improved JPEG encoders like mozjpeg and you're not in a terrible place. There's also work that could potentially be done with deblocking/debanding/deringing in decoders to stretch the old approach even further.

And JXL's other modes also had their advantages. VarDCT was still faster than libaom AVIF, and was reasonable in its own way (AVIFs look smoother, JXL tended more to preserve traces of low-contrast detail). There was a progressive mode, which made less sense in AVIF because it was a format for video keyframes first. The lossless mode was the evolution of FUIF and put up good numbers.

At this point I have no particular predictions. JPEG never stopped being usable despite a series of more technically sophisticated successors. (MP3 too, though its successors seemed to get better adoption.) Perhaps it means things continue not to change for a while, or at least that I needn't rush to move to $other_format or get left behind. Doesn't mean I don't complain about the situation in comments on the Internet, though.