this was bound to happen with the number of compounding pharmacies getting into the glp-1 game. from a risk side, it’s a total mess it’s basically impossible for insurers to model long-term liability when the supply chain for these drugs is a black box. fda had to drop the hammer before the claims started flying.
Readit Newsinsuranceguru commented on Software factories and the agentic moment factory.strongdm.ai/... · Posted by u/mellosouls
the agentic shift is where the legal and insurance worlds are really going to struggle. we know how to model human error, but modeling an autonomous loop that makes a chain of small decisions leading to a systemic failure is a whole different beast. the audit trail requirements for these factories are going to be a regulatory nightmare.
as someone looking at this from a risk and liability perspective, the move toward world models is pretty wild. the edge cases where a model predicts a safe path that doesn't align with physics is basically the final boss of insurance underwriting for autonomous vehicles. curious how they actually validate for 'common sense' reality vs just predicting the next frame.
It's interesting that he did this to stop people from using phones while driving, but he ended up creating a bigger public safety hazard by jamming emergency comms.
From a liability perspective if that jammer had blocked a 911 call during a nearby accident his exposure would have been far higher than just the $48k FCC fine. Federal preemption on signal jamming is one of the few areas where the hammer drops consistently hard.
I think this will act as a brake on the agentic shift as a whole.
if a company sells an autonomous agent that is marketed as doing a task without human oversight, the courts will eventually move that burden back to the manufacturer. we saw the same dance with autonomous driving disclaimers the "human must stay in control" line works as a legal shield for a while, but eventually the market demands a shift in who holds the risk.
if we stick to 100% human responsibility for black-box errors that a human couldn't have even predicted, that "brake" won't just slow down the agentic shift, it'll effectively kill the enterprise market for it. no C-suite is going to authorize a fleet of agents if they're holding 100% of the bag for emergent failures they can't audit.