Readit NewsCasks are the only things Homebrew does that some other package manager available on macOS doesn't reliably do better. Nix, Pkgsrc, MacPorts, and (and now Spack) all have better fundamental designs; sane, multi-user-friendly permissions; and enough isolation from the base system that they break neither each other nor manually-installed software.
I use Homebrew exclusively tucked away in isolated prefixes, only to install casks, and without ever putting any binaries it installs along the way on my PATH. I don't remember which programs it is, exactly, but I do use a few that are unsigned.
It also doesn't seem to me that the signing process is as vital in determining actual risk as the curation and moderation processes involved in maintaining "third-party" software distributions like Homebrew or Debian or whatever.
`--no-quarantine` in particular is one of the conveniences that makes Homebrew casks useful. If I have to give my consent anew for each app update, I might as well install the apps manually and live in the usual auto-update pop-up hell.
Specifically, if you restart and opt to restart apps, they can come up before all volumes have been decrypted and mounted. If your shell is on one such volume, your terminal emulator may fail to start, for example. This can happen when using Nix to install your shell, for example.
I imagine this may be even easier to hit over SSH unless the underlying problem was resolved.
FWIW you can fix the shell issue by wrapping your shell in a shim that essentially runs wait4path on the nix store before exec'ing your real shell. I set up my environment to install that shim binary directly onto the data volume at a known path so it can be used as my login shell.