I've zero interest in the AI tool, I'm discussing the broader problem.

The references were made up, and this is easier and faster to do with LLMs than with humans. Easier to do inadvertently, too.

As I said, LLMs are a force multiplier for fraud and inadvertent errors. So it's a big deal.

A company buys laptops for its employees and they get shipped from outside the US, and before they get delivered nice changes have been made.

Any specific individual that is high value will walk into a store and buy from stock.

Some of the funnest work, if you could get it, was swapping ssds out of laptops coming through customs for high value targets.

I've only read the abstract, but there is also plenty of evidence to suggest that people trust the output of LLMs more than other forms of media (or that they should). Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds.

The LLM bot army stuff is concerning, sure. The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.

I think this is more about mechanistic understanding vs fundamental insight kind of situation. The linear algebra picture is currently very mechanistic since it only tells us what the computations are. There are research groups trying to go beyond that but the insight from these efforts are currently very limited. However, the probabilistic view is very much clearer. You can have many explorable insights, both potentially true and false, by jıst understanding the loss functions, what the model is sampling from, what is the marginal or conditional distributions are and so on. Generative AI models are beautiful at that level. It is truly mind blowing that in 2025, we are able to sample from the megapixel image distributions conditioned on the NLP text prompts.

I've generally found an inverse correlation between "understands AI" and "exuberance for AI".

I'm the only person at my current company who has had experience at multiple AI companies (the rest have never worked on it in a production environment, one of our projects is literally something I got paid to deliver customers at another startup), has written professionally about the topic, and worked directly with some big names in the space. Unsurprisingly, I have nothing to do with any of our AI efforts.

One of the members of our leadership team, who I don't believe understands matrix multiplication, genuinely believes he's about to transcend human identity by merging with AI. He's publicly discussed how hard it is to maintain friendship with normal humans who can't keep up.

Now I absolutely think AI is useful, but these people don't want AI to be useful they want it to be something that anyone who understands it knows it can't be.

It's getting to the point where I genuinely feel I'm witnessing some sort of mass hysteria event. I keep getting introduced to people who have almost no understanding of the fundamentals of how LLMs work who have the most radically fantastic ideas about what they are capable of on a level I have ever experienced in my fairly long technical career.

It's more manual than you might think! Sent a message to the email in your profile with more details :)

> What’s the middle for taking cyanide ? 1g? 1kg?

Cyanide has an LD50 (50% chance of death) in the 1-2 mg/kg range when taken orally. So middle for taking cyanide is probably 1.5mg/kg. 90mg for someone 60kg.

Sadly the middle ground in other topics is less easy to define!

Sounds good, but I'm scared that for some other people it's not an improvement since there will always be someone saying that X amount is too much or not enough. But yeah, I could say that 1/4 or 1/5 of coding (or rather, googling of stuff I simply don't know) I do is now delegated to an llm, question is if another person would look at this statement and say something along the lines of "cool, I use it more/less but I'm happy that it helped you/I'm sad that it caused you trouble", I slowly think that we might be discussing kind off the wrong thing. But yeah, numerical/fact based approach doesn't sound half bad even though I have some feeling at the back of my head that it can also be kind off self fulfilling but nonetheless it helps in conveying the message better than what I used before (the I use it a lot/not much/I try to stay in the middle).