Forgive my ignorance, but why would you implement physics and animations on the server? I feel like you can get away with basic positional data and do the rest of that on the clients.
You need physics in order to make movement server authoritative. Otherwise player will be able to fly around and ignore collusions. You might need animation state in server if you are making a fps shooter and want to calculate raycasts properly.