If tomorrow, everyone said "we don't want IP's from Frankfurt showing up somewhere in Dubai", you'd have a massive technical problem and rearranging to start with but once that was sorted you could geo-lock. IANA and Network providers simply haven't been doing that.
The reason it doesn't happen is Devs/Stakeholders want uptime from ISPs/Networks and not something they can't abstract. Basically its just a status quo much like the entire internet reverse-proxying through CDNs is a status quo. It wasn't always like that, and it may not always be like that in the future - just depends which way the winds blow over time.
From a network perspective statements like that make no sense. IP addresses don't have any sort of physicality,
Texas (measured from Texas):
Virginia (measured from Maryland): And California (measured from California): The speed of light doesn't lie, IP addresses don't have any sort of physicality.