At that point, you lose the feeling that there are magic things out there that you will never understand and that for the most part everything is just regular old code that regular people wrote.
We can't build a robot which can walk down a sidewalk without running into people either. The sensor tech and mapping fidelity are red herrings. People drive well because only people are good at predicting human behavior.