This is one of the clearest takes I've seen that starts to get me to the point of possibly being able to trust code that I haven't reviewed.
The whole idea of letting an AI write tests was problematic because they're so focused on "success" that `assert True` becomes appealing. But orchestrating teams of agents that are incentivized to build, and teams of agents that are incentivized to find bugs and problematic tests, is fascinating.
I'm quite curious to see where this goes, and more motivated (and curious) than ever to start setting up my own agents.
Question for people who are already doing this: How much are you spending on tokens?
That line about spending $1,000 on tokens is pretty off-putting. For commercial teams it's an easy calculation. It's also depressing to think about what this means for open source. I sure can't afford to spend $1,000 supporting teams of agents to continue my open source work.
I don't take your comment as dismissive, but I think a lot of people are dismissing interesting and possibly effective approaches with short reactions like this.
I'm interested in the approach described in this article because it's specifying where the humans are in all this, it's not about removing humans entirely. I can see a class of problems where any non-determinism is completely unacceptable. But I can also see a large number of problems where a small amount of non-determinism is quite acceptable.