However, below the OS level much new technology has been introduced that has greatly increased the attack surface, from processor performance enhancements such as branch prediction to subsystems such as Intel ME. I almost feel like Intel broke a social compact that their products would be predictable, safe commodities on which I can build my systems. But did those good old days ever really exist?. And of course, Intel naturally doesn't want their products to be commodities, which likely is why they introduced these new features.
Focusing on OS and application security may be living in a fantasy world, one I hesitate to give up because the reality is much more complex. What good are OpenBSD's or Chrome's security efforts, for example, if the processor on which they run is insecure and if there are insecure out-of-band management subsystems? Why does an attacker need to worry about the OS?
(Part of the answer is that securing the application and OS makes attacks more expensive; at least we can reduce drive-by JavaScript exploits. But now the OS and application are a smaller part of the security puzzle, and not at all sufficient.)
In terms of cryptographic robustness, it's good for an app like Signal to have a closed-source server, because it forces you to not trust the server.
(This is of course separate from whether it's good for the Signal server to be free software for inherent free-software morality reasons.)
Still, I think the most important breaches of privacy are not necessarily in decoding the messages themselves, but rather everything from location data to contact lists including time and number of communications these apps have access to by default.