The AI industry, and SV tech generally, has a pattern of recruiting talent by flattering people's self-image as builders and discoverers, which makes it psychologically very difficult for those people to reckon honestly with downstream harm.
Readit Newspadolsey commented on Why I Joined OpenAI brendangregg.com/blog/202... · Posted by u/SerCe
> I'm surprised to see so little coverage of AI legislation news here tbh.
Because no one believes these laws or bills or acts or whatever will be enforced.
But I actually believe they'll be. In the worst way possible: honest players will be punished disproportionally.
> Because no one believes these laws or bills or acts or whatever will be enforced.
Time will tell. Texas' sat on its biometric data act quite quietly then hammered meta with a $1.4B settlement 20 years after the bill's enactment. Once these laws are enacted, they lay quietly until someone has a big enough bone to pick with someone else. There are already many traumatic events occurring downstream from slapdash AI development.
I'm surprised to see so little coverage of AI legislation news here tbh. Maybe there's an apathy and exhaustion to it. But if you're developing AI stuff, you need to keep on top of this. This is a pretty pivotal moment. NY has been busy with RAISE (frontier AI safety protocols, audits, incident reporting), S8420A (must disclose AI-generated performers in ads), GBL Article 47 (crisis detection & disclaimers for AI chatbots), S7676B (protects performers from unauthorized AI likenesses), NYC LL144 (bias audits for AI hiring tools), SAFE for Kids Act [pending] (restricts algorithmic feeds for minors). At least three of those are relevant even if your app only _serves_ people in NY. It doesn't matter where you're based. That's just one US state's laws on AI.
It's kinda funny the oft-held animosity towards EU's heavy-handed regulations when navigating US state law is a complete minefield of its own.
padolsey commented on The browser is the sandbox aifoc.us/the-browser-is-t... · Posted by u/enos_feedler
Agree! And this is why it is a bad idea IMHO for agents to sit at the abstraction layer of browser or below (OS). Even at the browser-addon level it's dangerous. It runs with the user’s authority across contexts and erodes zero-trust by becoming a confused deputy: https://en.wikipedia.org/wiki/Confused_deputy_problem
Not sure formidable founders getting whatever they want whatever the obstacle (sure market obstacles but what about laws, morality, etc) is a good thing to be honest.
Sometimes folks need to be stopped. Sometimes those walls are there for a reason.
And after IPO, maybe a founder should consider the good of the world instead of what you think you want next for yourself and just bashing down more walls.
But I dunno… I’m just a rando.
Yeh I don't think there's much value in a credo if it celebrates Altman. He's a terrible idol to have. He compared Trump to Hitler in 2016, then donated $1M to his inauguration and tweeted about being in an "NPC trap" when he criticized him. Took about six weeks after the election to flip. Testified to Congress that AI regulation is "essential," then lobbied against California's safety bill when it actually showed up. His own board fired him for lying to them for years. His safety team leads quit in protest saying safety took a backseat to shiny products. Multiple former colleagues, including the people who left to start Anthropic, describe psychological abuse and manipulation. Claims a $65k salary while sitting on a billion-dollar fortune built through conflicts of interest. He's not a good guy. He's a guy who says whatever serves him in the moment and has left a trail of people warning us about exactly that.
padolsey commented on Reading across books with Claude Code pieterma.es/syntopic-read... · Posted by u/gmays
For me this looks like a great way to build connections between books in order to create a recommendation engine - something better than what Goodreads & Co provides. Something actually useful.
The cost of indexing using third party API is extremely high, however. This might work out well with an open source model and a cluster of raspberry pi for large library indexing?
The incumbants Goodreads and their owner Amazon have indeed done such a poor job at this. Seven years ago I tried creating a basic graph using collaborative-filtering (effectively using our actual reading patterns as the embeddings space instead of semantics [human X likes book Y so likers of Y might like other things that human X has enjoyed]). It works well to this day (ablf.io) but the codebase is so ugly I've not had the bravery to update its data in a couple of years.
padolsey commented on After my dad died, we found the love letters jenn.site/after-my-dad-di... · Posted by u/eatitraw
Agreed. I just hope they repair their Shift keys soon.
I think it's become a bit of a cliche/clique'y thing amongst a certain population. I don't know its origins (tumblr emo crowd??) but I first encountered it in Silicon Valley. The Collison brothers used to love doing it, as did Altman. I feel it projects a kind of stream-of-thought with an aloofness, like "i dont care enough for correct form. language bends to my unique thoughts. read this if you like, i dont care lol".
All-lowercase comes accross as the text equivalent of a hoodie and jeans: comfortable, a bit defensive against being seen as trying too hard, and now so common it barely reads as rebellion.
padolsey commented on Anthropic’s paper smells like bullshit djnn.sh/posts/anthropic-s... · Posted by u/vxvxvx
The prompts aren't the key to the attack, though. They were able to get around guardrails with task decomposition.
There is no way for the AI system to verify whether you are white hat or black hat when you are doing pen-testing if the only task is to pen-test. Since this is not part of a "broader attack" (in the context), there is no "threat".
I don't see how this can be avoided, given that there are legitime uses to every step of this in creating defenses to novel attacks.
Yes, all of this can be done with code and humans as well - but it is the scale and the speed that becomes problematic. It can adjust in real-time to individual targets and does not need as much human intervention / tailoring.
Is this obvious? Yes - but it seems they are trying to raise awareness of an actual use of this in the wild and get people discussing it.
I agree that there will be no single call or inference that presents malice. But I feel like they could still share general patterns of orchestration (latencies, concurrencies, general cadences and parallelization of attacks, prompts used to granulaize work, whether prompts themselves have been generated in previous calls to Claude). There's a bunch of more specific telltales they could have alluded to. I think it's likely they're being obscure because they don't want to empower bad actors, but that's not really how the cybersecurity industry likes to operates. Maybe Anthropic believes this entire AI thing is a brand new security regime and so believe existing resiliences are moot. That we should all follow blindly as they lead the fight. Their narrative is confusing. Are they being actually transparent or transparency-"coded"?