photonthug (u/photonthug)

photonthug commented on Using AI to negotiate a $195k hospital bill down to $33k threads.com/@nthmonkey/po... · Posted by u/stevenhubertron

thaumasiotes · 4 months ago

This reflects a common complaint in social engineering.

People keep trying to enact rules to stick it to the elites and make the downtrodden better off.

And as the rules get more and more complex, the position of the elites gets more and more solid.

photonthug · 4 months ago

Adding complexity is just one aspect. Everywhere there is someone whose job is to ensure the bottom line never changes and status quo for the powerful is preserved. Insurance, taxes, rents.. in the absence of effective regulation, the average number of successful appeals will simply get factored in and average costs go up so that profit stays the same and grows at the same rate as before. Similar to how chains factor in losses due to spoilage or theft.. of course they don't actually take a profit loss, they just price it in.

I really don't get people who see this kind of thing as empowering because in the end your (now strictly necessary) appeal with lawyers or AI to get a more fair deal just becomes a new tax on your time/money; you are worse off than before. A good capitalist will notice these dynamics, and invest in AI once it's as required for life as healthcare is, and then work on driving up the costs of AI. Big win for someone but not the downtrodden.

photonthug commented on The new calculus of AI-based coding blog.joemag.dev/2025/10/t... · Posted by u/todsacerdoti

photochemsyn · 4 months ago

Well, 'calculus' is the kind of marketing word that sounds more impressive than 'arithmetic' and I think 'quantum logic' has gone a bit stale, and 'AI-based' might give more hope to the anxious investor class, as 'AI-assisted' is a bit weak as it means the core developer team isn't going to be cut from the labor costs on the balance sheet, they're just going to be 'assisted' (things like AI-written unit tests that still need some checking).

"The Arithmetic of AI-Assisted Coding Looks Marginal" would be the more honest article title.

photonthug · 4 months ago

Yes, unfortunately a phrase that's used in an attempt to lend gravitas and/or intimidate people. It sort of vaguely indicates "a complex process you wouldn't be interested in and couldn't possibly understand". At the same time it attempts to disarm any accusation of bias in advance by hinting at purely mechanistic procedures.

Could be the other way around, but I think marketing-speak is taking cues here from legal-ese and especially the US supreme court, where it's frequently used by the justices. They love to talk about "ethical calculus" and the "calculus of stare decisis" as if they were following any rigorous process or believed in precedent if it's not convenient. New translation from original Latin: "we do what we want and do not intend to explain". Calculus, huh? Show your work and point to a real procedure or STFU

photonthug commented on It's insulting to read AI-generated blog posts blog.pabloecortez.com/its... · Posted by u/speckx

the_af · 4 months ago

> It's clearly ridiculous, yet at the point where papers or PRs are written by robots, reviewed by robots, for eventual usage/consumption/summary by yet more robots, it becomes very relevant. At some point one must ask, what is it all for, and should we maybe just skip some of these steps or revisit some assumptions about what we're trying to accomplish

I've been thinking this for a while, despairing, and amazed that not everyone is worried/surprised about this like me.

Who are we building all this stuff for, exactly?

Some technophiles are arguing this will free us to... do what exactly? Art, work, leisure, sex, analysis, argument, etc will be done for us. So we can do what exactly? Go extinct?

"With AI I can finally write the book I always wanted, but lacked the time and talent to write!". Ok, and who will read it? Everybody will be busy AI-writing other books in their favorite fantasy world, tailored specifically to them, and it's not like a human wrote it anyway so nobody's feelings should be hurt if nobody reads your stuff.

photonthug · 4 months ago

As something of a technophile myself.. I see a lot more value in arguments that highlight totally ridiculous core assumptions rather than focusing on some kind of "humans first and only!" perspectives. Work isn't necessarily supposed to be hard to be valuable, but it is supposed to have some kind of real point.

In the dating scenario what's really absurd and disgusting isn't actually the artificiality of toys.. it's the ritualistic aspect of the unnecessary preamble, because you could skip straight to tea and talk if that is the point. We write messages from bullet points, ask AI to pad them out uselessly with "professional" sounding fluff, and then on the other side someone is summarizing them back to bullet points? That's insane even if it was lossless, just normalize and promote simple communications. Similarly if an AI review was any value-add for AI PR's, it can be bolted on to the code-gen phase. If editors/reviewers have value in book publishing, they should read the books and opine and do the gate-keeping we supposedly need them for instead of telling authors to bring their own audience, etc etc. I think maybe the focus on rituals, optics, and posturing is a big part of what really makes individual people or whole professions obsolete

photonthug commented on It's insulting to read AI-generated blog posts blog.pabloecortez.com/its... · Posted by u/speckx

nbardy · 4 months ago

You know you can AI review the PR too, don't be such a curmudgeon. I have PR's at work I and coworkers fully AI generated and fully AI review. And

photonthug · 4 months ago

> fully AI generated and fully AI review

This reminds me of an awesome bit by Žižek where he describes an ultra-modern approach to dating. She brings the vibrator, he brings the synthetic sleeve, and after all the buzzing begins and the simulacra are getting on well, the humans sigh in relief. Now that this is out of the way they can just have a tea and a chat.

It's clearly ridiculous, yet at the point where papers or PRs are written by robots, reviewed by robots, for eventual usage/consumption/summary by yet more robots, it becomes very relevant. At some point one must ask, what is it all for, and should we maybe just skip some of these steps or revisit some assumptions about what we're trying to accomplish

photonthug commented on You are how you act boz.com/articles/you-are-... · Posted by u/HiPHInch

swiftcoder · 4 months ago

> this one is much less teenager trying to sound clever

On the other hand, it's very much freshman-who-misunderstood-philosophy-101-and-integrated-it-into-his-worldview-anyway...

photonthug · 4 months ago

In philosophy 101 the usual foil for Rousseau vs.. would be Hobbes, but that framing with a realist/pessimist would not be popular with the intended audience, where the goal is to lionize the nationalist, the inventors/owners, the 1%.

> Despite his own moral lapses, Franklin saw himself as uniquely qualified to instruct Americans in morality. He tried to influence American moral life through the construction of a printing network based on a chain of partnerships from the Carolinas to New England. He thereby invented the first newspaper chain. https://en.wikipedia.org/wiki/Benjamin_Franklin#Newspaperman

To be clear Franklin's obviously a complicated historical figure, a pretty awesome guy overall, and I do like American pragmatism generally. But it matters a lot which part of the guy you'd like to hold up for admiration, and elevating a preachy hypocrite that was an early innovator in monopolies and methods of controlling the masses does seem pretty tactical and self-serving here.

photonthug commented on A definition of AGI arxiv.org/abs/2510.18212... · Posted by u/pegasus

Der_Einzige · 4 months ago

Most people who say "AGI" really mean either "ASI" or "Recursive Self Improvement".

AGI was already here the day ChatGPT released: That's Peter Norvig's take too: https://www.noemamag.com/artificial-general-intelligence-is-...

photonthug · 4 months ago

Funny but the eyebrow-raising phrase 'recursive self-improvement' is mentioned in TFA in an example about "style adherence" that's completely unrelated to the concept. Pretty clearly a scam where authors are trying to hack searches.

Prerequisite for recursive self-improvement and far short of ASI, any conception of AGI really really needs to be expanded to include some kind of self-model. This is conspicuously missing from TFA. Related basic questions are: What's in the training set? What's the confidence on any given answer? How much of the network is actually required for answering any given question?

Partly this stuff is just hard and mechanistic interpretability as a field is still trying to get traction in many ways, but also, the whole thing is kind of fundamentally not aligned with corporate / commercial interests. Still, anything that you might want to call intelligent has a working self-model with some access to information about internal status. Things that are mentioned in TFA (like working memory) might be involved and necessary, but don't really seem sufficient

photonthug commented on A definition of AGI arxiv.org/abs/2510.18212... · Posted by u/pegasus

runarberg · 4 months ago

I never really understood what made the Turing test so special. On the face of it, it is a rather dumb test. And it was debunked within two decades by the Chinese room thought experiment. And now that we have language models that are obviously not intelligent, it should have been the last nail in the coffin for this test.

Alan Turing was a mathematician not a psychologist, this was his attempt of doing philosophy. And while I applaud brilliant thinkers when they attempt to do philosophy (honestly we need more of that) it is better to leave it to actual philosophers to validate the quality of said philosophy. John Searle was a philosopher which specialized in questions of psychology. And in 1980 he pretty convincingly argued against the Turning test.

photonthug · 4 months ago

Yeah, I mean I hope there are not many people that still think it's a super meaningful test in the sense originally proposed. And yet it is testing something. Even supposing it were completely solved and further supposing the solution is theoretically worthless and only powers next-gen slop-creation, then people would move on to looking for a minimal solution, and perhaps that would start getting interesting. People just like moving towards concrete goals.

In the end though, it's probably about as good as any single kind of test could be, hence TFA looking to combine hundreds across several dozen categories. Language was a decent idea if you're looking for that exemplar of the "AGI-Complete" class for computational complexity, vision was at one point another guess. More than anything else I think we've figured out in recent years that it's going to be hard to find a problem-criteria that's clean and simple, much less a solution that is

photonthug commented on A definition of AGI arxiv.org/abs/2510.18212... · Posted by u/pegasus

p1esk · 4 months ago

Me:

What color is a green eggplant?

ChatGPT:

A green eggplant (also called Thai eggplant or green brinjal, depending on the variety) is actually green on the outside — sometimes solid green, sometimes streaked with white or pale green.

The inside flesh is typically off-white or light cream, just like purple eggplants.

If you mean a green-colored variety (not an unripe purple one), then the skin color can range from pale mint green to deep jade, often with light striping.

Me:

How would an average human answer this question?

ChatGPT:

An average person would probably answer something simple and direct like:

“It’s green.”

Or if they’re being a bit more specific:

“It’s green on the outside and white inside.”

Most people wouldn’t overthink the variety or mention Thai vs. unripe types — they’d just describe what they see.

photonthug · 4 months ago

Outcome would depend on the rest of the test, but I'd say the "human" version of this answer adds zero or negative value to chances of being human, on grounds of strict compliance, sycophancy, and/or omniscience. "No such thing" would probably be a very popular answer. Elaboration would probably take the form of "love it" or "hate it", instead of reaching for a comprehensive answer describing the inside and the outside.

Experimental design comes in here and the one TT paper mentioned in this thread has instructions for people like "persuade the interrogator [you] are human". Answering that a green eggplant is green feels like humans trying to answer questions correctly and quickly, being wary of a trap. We don't know participants background knowledge but anyone that's used ChatGPT would know that ignoring the question and maybe telling an eggplant-related anecdote was a better strategy

photonthug commented on A definition of AGI arxiv.org/abs/2510.18212... · Posted by u/pegasus

rmunn · 4 months ago

I would argue that current LLMs are passing the Turing test because many observers have a hard time distinguishing them from humans: just look at the difficulty many schools have in enforcing rules like "Not allowed to use LLMs for your homework". The teachers often (not always) can't tell, looking at a piece of text, whether a human produced it or whether ChatGPT or some other LLM produced it.

And that "not always" is the crux of the matter, I think. You are arguing that we're not there yet, because there are lines of questioning you can apply that will trip up an LLM and demonstrate that it's not a human. And that's probably a more accurate definition of the test, because Turing predicted that by 2000 or so (he wrote "within 50 years" around 1950) chatbots would be good enough "that an average interrogator will not have more than 70% chance of making the right identification after five minutes of questioning". He was off by about two decades, but by now that's probably happened. The average interrogator probably wouldn't come up with your (good) strategy of using counterfactuals to trick the LLM, and I would argue two points: 1) that the average interrogator would indeed fail the Turing test (I've long argued that the Turing test isn't one that machines can pass, it's one that humans can fail) because they would likely stick to conventional topics on which the LLM has lots of data, and 2) that the situation where people are actually struggling to distinguish LLMs is one where they don't have an opportunity to interrogate the model: they're looking at one piece of multi-paragraph (usually multi-page) output presented to them, and having to guess whether it was produced by a human (who is therefore not cheating) or by an LLM (in which case the student is cheating because the school has a rule against it). That may not be Turing's actual test, but it's the practical "Turing test" that applies the most today.

photonthug · 4 months ago

I think the TT has to be understood as explicitly adversarial, and increasingly related to security topics, like interactive proof and side channels. (Looking for guard-rails is just one kind of information leakage, but there's lots of information available in timing too.)

If you understand TT to be about tricking the unwary, in what's supposed to be a trusting and non-adversarial context, and without any open-ended interaction, then it's correct to point out homework-cheating as an example. But in that case TT was solved shortly after the invention of spam. No LLMs needed, just markov models are fine.

photonthug commented on A definition of AGI arxiv.org/abs/2510.18212... · Posted by u/pegasus

GolDDranks · 4 months ago

Note that Turing test allows a lot leeway for the test settings, i.e. who interrogates it, how much they know about the weakness of current SOA models, are they allowed to use tools (I'm thinking of something like ARC-AGI but in a format that allows chat-based testing), and how long a chat is allowed etc. Therefore there can be multiple interpretations of whether the current models pass the test or not.

One could say that if there is maximally hard Turing test, and a "sloppy" Turing test, we are somewhere where the current models pass the sloppy version but not the maximally hard version.

photonthug · 4 months ago

Hah, tools-or-no does make things interesting, since this opens up the robot tactic of "use this discord API to poll some humans about appropriate response". And yet if you're suspiciously good at cube roots, then you might out yourself as robot right away. Doing any math at all in fact is probably suspect. Outside of a classroom humans tend to answer questions like "multiply 34 x 91" with "go fuck yourself", and personally I usually start closing browser tabs when asked to identify motorcycles