Ah, yes, safety, because what is more safe than to help DoD/Palantir kill people[1]?
No, the real risk here is that this technology is going to be kept behind closed doors, and monopolized by the rich and powerful, while us scrubs will only get limited access to a lobotomized and heavily censored version of it, if at all.
[1] - https://www.anthropic.com/news/anthropic-and-the-department-...
Long-term safety for free people entails military use of new technologies. Imagine if people advocating airplane safety groused about the use of bomber and fighter planes being built and mobilized in the Second World War.
Now, I share your concern about governments who unjustly wield force (either in war or covert operations). That is an issue to be solved by articulating a good political philosophy and implementing it via policy, though. Sadly, too many of the people who oppose the American government's use of such technology have deeply authoritarian views themselves — they would just prefer to see a different set of values forced upon people.
Last: Is there any evidence that we're getting some crappy lobotomized models while the companies keep the best for themselves? It seems fairly obvious that they're tripping over each other in a race to give the market the highest intelligence at the lowest price. To anyone reading this who's involved in that, thank you!
Long-term safety also entails restraining the military-industrial complex from the excesses it's always prone to.
Remember, Teller wanted to make a 10 gigaton nuke. https://en.wikipedia.org/wiki/Sundial_(weapon)
The integrity of a free society's government is the central issue here, not the creation of tools which could be militarily useful to a free society.