My understanding of it has been that aphantasiacs can only imagine in terms of verbal descriptions, not images. If that's the case, it seems like visual analogy would be a good differentiator.

For example: without any internal monologue, think of the Sydney Opera House, and then name some other objects it resembles.

Someone with visual imagination should be able to rattle off stuff like sailboats or seashells or folded napkins based purely on visual similarity, while a true aphantasiac should be lost without being able to look at a picture or derive an answer from a mental list of attributes.

(Likewise, if you gave a non-aphantasiac a written list of visual attributes the Sydney Opera House and ask them to name similar objects without picturing anything visually, it might be much more difficult to get the same range of answers.)