Without proof to the contrary the assumption should be that the trans population has a similar proportion of rapists as any other does.
Since its been well established that essentially all rape no matter who is raping who goes under reported it is completely rational for women to be just as fearful as they would with any other stranger.
Cis women can perpetuate rape. Should we ban them from changing rooms too? How far through the looking glass does this have to go?!
How about predatory men pretending to be trans-women?