The worst part about this:
> Running experiments until you get a hit
Is that it's literally what us software optimization engineers do. We keep writing optimizations until we find one that is a statistically significant speed-up.
Hence we are running experiments until we get a hit.
The only defense I know against this is to have a good perf CI. If your patch seemed like a speed-up before committing, but perf CI doesn't see the speed-up, then you just p-hacked yourself. But that's not even fool proof.
You just have to accept that statistics lie and that you will fool yourself. Prepare accordingly.
There's another cheeky example of this where you select a pseudo-random seed that makes your result significant. I have a personal seed, I use it in every piece of research that uses random number generation. It keeps me honest!
In general, I think the best way to think about the differences between human muscles and robot actuators is that human muscles are simultaneously incredible in terms of strength and power density, and also incredibly fragile. Robot actuators are fairly robust, but comparatively poor. Humans are essentially falling apart at all times, but the repair mechanisms in the body do a good enough job that it doesn't matter (although you probably know someone with a "career-disruptive/ending" sports-related injury that shows these mechanisms have limits). Robotics is a long way away from making actuators that can be fixed online in such a process. Even cable stretching in cable-driven mechanisms remains hard to handle effectively, and that's one of the simplest types of mechanism wear.