Just a heads up – your main website is showing an error. You might want to fix it since your post is gaining traction. Here's the link: https://www.sievedata.com/
The error message reads: 'Application error: a client-side exception has occurred (check the browser console for more details).'
The model tries to copy the blinks of the original video so it's possible that in other conditions, you'd notice less of this.
Fun to see this feedback though, definitely something worth improving :)
https://www.sievedata.com/blog/eye-contact-correction-gaze-c...
Newer models have come out that allow the same thing to be done and control even more than the eyes.
See here: https://github.com/KwaiVGI/LivePortrait/blob/main/assets/doc...
For web-conferencing, local use is great so NVIDIA's tools are what we recommend in that case.
Building further on this idea, I wonder if instead of changing the image to look at the camera, we could change the "camera" to be where we're looking.
In other words we could simulate a virtual camera somewhere in the screen, perhaps over the eyes of the person talking.
We could simulate a virtual camera by using the image of the real camera (or cameras), constructing a 3D image of ourselves and re-rendering it from the virtual camera location.
I think this would be really cool. It would be like there was a camera in the centre of our screen. We could stop worrying about looking at the camera and look at the person talking.
Of course this is all very tricky, but does feel possible right now. I think the Apple Vision Pro might do something similar already?
But the resultant video has a tad bit of uncanny valley going on.
I'd rather learn from the guy on the right.
Would recommend trying it on other videos, it is surprisingly good. Although there definitely are areas to improve.
https://www.sievedata.com/
Application error: a client-side exception has occurred (see the browser console for more information).