Edit: for clarity, by "multiple OS" I mean multiple Linux versions. Like if one project has a dependency on Ubuntu22 and another is easier with Ubuntu24. You don't have to stress "do I update my OS?"
I do that with KVM too, and each has their own kernel, not one shared kernel made and controlled by one vendor.
To be fair I stay away from NVIDIA to, I would probably run a separate headless box for those GPU workloads if I needed to
In my experience, it has zero issues. I use nvidia binary build. I have since 2006 through various nvidia GPU's.