Readit News logoReadit News
mvoodarla commented on Eye Contact Correction: Redirecting the eyes to look at the camera   sievedata.com/functions/s... · Posted by u/thunderbong
patrickhogan1 · 10 months ago
BTW your main site is throwing an error. Probably want to edit since your post is growing.

https://www.sievedata.com/

Application error: a client-side exception has occurred (see the browser console for more information).

mvoodarla · 10 months ago
Original dev here. Unable to replicate this on my end, try refreshing?
mvoodarla commented on Eye Contact Correction: Redirecting the eyes to look at the camera   sievedata.com/functions/s... · Posted by u/thunderbong
patrickhogan1 · 10 months ago
Really cool application.

Just a heads up – your main website is showing an error. You might want to fix it since your post is gaining traction. Here's the link: https://www.sievedata.com/

The error message reads: 'Application error: a client-side exception has occurred (check the browser console for more details).'

mvoodarla · 10 months ago
Original dev here. Unable to replicate this on my end, try refreshing?
mvoodarla commented on Eye Contact Correction: Redirecting the eyes to look at the camera   sievedata.com/functions/s... · Posted by u/thunderbong
Retr0id · 10 months ago
Does what it says on the tin, but honestly I find the "uncorrected" video more comfortable to watch.
mvoodarla · 10 months ago
Original dev here. I tend to agree for this particular demo video as I'm reading a book and I don't blink in the original.

The model tries to copy the blinks of the original video so it's possible that in other conditions, you'd notice less of this.

Fun to see this feedback though, definitely something worth improving :)

mvoodarla commented on Eye Contact Correction: Redirecting the eyes to look at the camera   sievedata.com/functions/s... · Posted by u/thunderbong
zamadatix · 10 months ago
They also release it as an SDK these days so you can use it from tools like OBS https://www.nvidia.com/en-us/geforce/broadcasting/broadcast-...
mvoodarla · 10 months ago
Original dev here. That's right, NVIDIA has a version available which we reference in our blog.

https://www.sievedata.com/blog/eye-contact-correction-gaze-c...

Newer models have come out that allow the same thing to be done and control even more than the eyes.

See here: https://github.com/KwaiVGI/LivePortrait/blob/main/assets/doc...

For web-conferencing, local use is great so NVIDIA's tools are what we recommend in that case.

mvoodarla commented on Eye Contact Correction: Redirecting the eyes to look at the camera   sievedata.com/functions/s... · Posted by u/thunderbong
richdougherty · 10 months ago
Kudos to the dev for coming up with the eye position fixing solution.

Building further on this idea, I wonder if instead of changing the image to look at the camera, we could change the "camera" to be where we're looking.

In other words we could simulate a virtual camera somewhere in the screen, perhaps over the eyes of the person talking.

We could simulate a virtual camera by using the image of the real camera (or cameras), constructing a 3D image of ourselves and re-rendering it from the virtual camera location.

I think this would be really cool. It would be like there was a camera in the centre of our screen. We could stop worrying about looking at the camera and look at the person talking.

Of course this is all very tricky, but does feel possible right now. I think the Apple Vision Pro might do something similar already?

mvoodarla · 10 months ago
This is an interesting idea. We are a little farther off from being able to do this but agree it would look really cool.
mvoodarla commented on Eye Contact Correction: Redirecting the eyes to look at the camera   sievedata.com/functions/s... · Posted by u/thunderbong
thekevan · 10 months ago
From a development standpoint, this is cool.

But the resultant video has a tad bit of uncanny valley going on.

I'd rather learn from the guy on the right.

mvoodarla · 10 months ago
Original dev here. Agree this video looks like uncanny valley but it's likely because the lighting of the original video is off + I baggy eyes (I was sleep deprived).

Would recommend trying it on other videos, it is surprisingly good. Although there definitely are areas to improve.

u/mvoodarla

KarmaCake day410September 8, 2019
About
https://mokshith.xyz/

mokshith@sievedata.com

https://www.sievedata.com/

View Original