andai (u/andai) - Readit News

andai commented on An Implementation of J jsoftware.com/ioj/ioj.htm... · Posted by u/ofalkaed

jandrese · 3 hours ago

I have to admit I got to https://www.jsoftware.com/ioj/iojATW.htm and seriously considered if the site is just pulling my leg. I think they're being sincere but I can't be 100% sure.

andai · an hour ago

I think this seems insane from programmer land, but quaint from pure math land.

(I say this as someone who got filtered by pure math!)

andai commented on I fed 24 years of my blog posts to a Markov model susam.net/fed-24-years-of... · Posted by u/zdw

andai · an hour ago

In 2020, a friend and I did this with our mutual WhatsApp chat history.

Except instead we fine-tuned GPT-2 instead. (As was the fashion at the time!)

We used this one, I think https://github.com/minimaxir/gpt-2-simple

I think it took 2-3 hours on my friend's Nvidia something.

The result was absolutely hilarious. It was halfway between a markov chain and what you'd expect from a very small LLM these days. Completely absurd nonsense, yet eerily coherent.

Also, it picked up enough of our personality and speech patterns to shine a very low resolution mirror on our souls...

###

Andy: So here's how you get a girlfriend:

1. Start making silly faces

2. Hold out your hand for guys to swipe

3. Walk past them

4. Ask them if they can take their shirt off

5. Get them to take their shirt off

6. Keep walking until they drop their shirt

Andy: Can I state explicitly this is the optimal strategy

andai commented on I fed 24 years of my blog posts to a Markov model susam.net/fed-24-years-of... · Posted by u/zdw

sebastianmestre · 5 hours ago

Cool article, it got me to play around with Markov models, too! I first did a Markov model over plain characters.

> Itheve whe oiv v f vidleared ods alat akn atr. s m w bl po ar 20

Using pairs of consecutive characters (order-2 Markov model) helps, but not much:

> I hateregratics.pyth fwd-i-sed wor is wors.py < smach. I worgene arkov ment by compt the fecompultiny of 5, ithe dons

Triplets (order 3) are a bit better:

> I Fed tooks of the say, I just train. All can beconsist answer efferessiblementate

> how examples, on 13 Debian is the more M-x: Execute testeration

LLMs usually do some sort of tokenization step prior to learning parameters. So I decided to try out order-1 Markov models over text tokenized with byte pair encoding (BPE).

Trained on TFA I got this:

> I Fed by the used few 200,000 words. All comments were executabove. This value large portive comment then onstring takended to enciece of base for the see marked fewer words in the...

Then I bumped up the order to 2

> I Fed 24 Years of My Blog Posts to a Markov Model

> By Susam Pal on 13 Dec 2025

>

> Yesterday I shared a little program calle...

It just reproduced the entire article verbatim. This makes sense as BPE removes any pair of repeated tokens, making order-2 Markov transitions fully deterministic.

I've heard that in NLP applications, it's very common to run BPE only up to a certain number of different tokens, so I tried that out next.

Before limiting, BPE was generating 894 tokens. Even adding a slight limit (800) stops it from being deterministic.

> I Fed 24 years of My Blog Postly coherent. We need to be careful about not increasing the order too much. In fact, if we increase the order of the model to 5, the generated text becomes very dry and factual

It's hard to judge how coherent the text is vs the author's trigram approach because the text I'm using to initialize my model has incoherent phrases in it anyways.

Anyways, Markov models are a lot of fun!

andai · an hour ago

Nice :) I did something similar a few days ago. What I ended up with was a 50/50 blend of hilarious nonsense, and verbatim snippets.There seemed to be a lot of chains where there was only one possible next token.

I'm considering just deleting all tokens that have only one possible descendant, from the db. I think that would solve that problem. Could increase that threshold to, e.g. a token needs to have at least 3 possible outputs.

However that's too heavy handed: there's a lot of phrases or grammatical structures that would get deleted by that. What I'm actually trying to avoid is long chains where there's only one next token. I haven't figured out how to solve that though.

andai commented on A 'toaster with a lens': The story behind the first handheld digital camera bbc.com/future/article/20... · Posted by u/selvan

andai · 16 hours ago

>"But Joy had followed me back because she was curious, you know, and she was standing in the hallway. We turned around, and Joy says: 'Needs work,' and turned out and walked away."

This part reminded me of the Black Triangle (2004):

https://archive.ph/qqOnP

https://news.ycombinator.com/item?id=698753

andai commented on Apple has locked my Apple ID, and I have no recourse. A plea for help hey.paris/posts/appleid/... · Posted by u/parisidau

x3sphere · a day ago

It's just insane that a gift card redemption can trigger this. What's the rationale? It would make more sense if they just locked the person out of redeeming gift cards or something, not the entire account.

But reading horror stories like this is is why I only use the very bare minimum of any of these cloud services. Keep local copies of everything. For developer accounts, I always create them under a separate email so they're not tied to my personal. At least it can minimize the damage somewhat.

It sucks that I have to take all these extra precautions though. It's definitely made me develop a do not trust any big corp mindset.

andai · 17 hours ago

> It's definitely made me develop a do not trust any big corp mindset.

I've been reading about Lovecraft's Old Ones. Apparently they have no ill will towards humans. They just sometimes cause harm without realizing it, while going about their business.

andai commented on We built another object storage fractalbits.com/blog/why-... · Posted by u/fractalbits

andai · 17 hours ago

HN's version of this title is unintentional comedy :)

andai commented on GPT-5.2 openai.com/index/introduc... · Posted by u/atgctg

svara · 2 days ago

In my experience, the best models are already nearly as good as you can be for a large fraction of what I personally use them for, which is basically as a more efficient search engine.

The thing that would now make the biggest difference isn't "more intelligence", whatever that might mean, but better grounding.

It's still a big issue that the models will make up plausible sounding but wrong or misleading explanations for things, and verifying their claims ends up taking time. And if it's a topic you don't care about enough, you might just end up misinformed.

I think Google/Gemini realize this, since their "verify" feature is designed to address exactly this. Unfortunately it hasn't worked very well for me so far.

But to me it's very clear that the product that gets this right will be the one I use.

andai · 2 days ago

So there's two levels to this problem.

Retrieval.

And then hallucination even in the face of perfect context.

Both are currently unsolved.

(Retrieval's doing pretty good but it's a Rube Goldberg machine of workarounds. I think the second problem is a much bigger issue.)

andai commented on Nokia N900 Necromancy yaky.dev/2025-12-11-nokia... · Posted by u/yaky

andai · 2 days ago

Can someone explain the use of super capacitors here? Do they function as a battery?

andai commented on The highest quality codebase gricha.dev/blog/the-highe... · Posted by u/Gricha

xnorswap · 3 days ago

Claude is really good at specific analysis, but really terrible at open-ended problems.

"Hey claude, I get this error message: <X>", and it'll often find the root cause quicker than I could.

"Hey claude, anything I could do to improve Y?", and it'll struggle beyond the basics that a linter might suggest.

It suggested enthusiastically a library for <work domain> and it was all "Recommended" about it, but when I pointed out that the library had been considered and rejected because <issue>, it understood and wrote up why that library suffered from that issue and why it was therefore unsuitable.

There's a significant blind-spot in current LLMs related to blue-sky thinking and creative problem solving. It can do structured problems very well, and it can transform unstructured data very well, but it can't deal with unstructured problems very well.

That may well change, so I don't want to embed that thought too deeply into my own priors, because the LLM space seems to evolve rapidly. I wouldn't want to find myself blind to the progress because I write it off from a class of problems.

But right now, the best way to help an LLM is have a deep understanding of the problem domain yourself, and just leverage it to do the grunt-work that you'd find boring.

andai · 3 days ago

The current paradigm is we sorta-kinda got AGI by putting dodgy AI in a loop:

until works { try again }

The stuff is getting so cheap and so fast... a sufficient increment in quantity can produce a phase change in quality.