Very cool. The use of a webcam really makes me wonder if there's a future where our regular single ~78° FOV webcams are going to be replaced by dual (stereo) fisheye webcams that can:
- Enable all sorts of new UX interactions (gestures with eye tracking)
- Enable all sorts of new peripheral interactions (stylus like this, but also things like a steering wheel for racing games)
- Enable 3D 180° filming for far more flexible webcam meetings, including VR presence, etc.
The idea of being able to use the entire 3D space in front of your computer display as an input method feels like it's coming, and using a webcam the way OP describes feels like it's a little step in that direction.
I thought this was around the corner years ago when Intel and partners had RealSense modules being built into laptops but it seems like all the players have shifted focus to more enterprise and industrial markets.
These images are super far away though eg. entire Earth so idk