So - how did this virus spread through the population? 20 kyrs ago people didn't live in cities but in small bands of a few dozen people. It's easy to see how an epidemic might spread through a village but how would it move between isolated groups?
Maybe the virus was not a pandemic 20k years ago. Small tribes that carried genes that are resistant to the coronaviruses spread later on as they established larger communities/cities.