Stocking a book in a library is not speech from the government. If it were, we couldn't have religious books in school libraries, but we do.
If stocking a book is not speech then it is not a restriction on speech to decide not to stock a book.
The people who fight for free speech in these cases, devoting time and money to it, and have real meaningful effect, self-describe in more ordinary ways.