After years of dealing with this (first Jenkins, then GitLab, then GitHub), my takeaway is:
* Write as much CI logic as possible in your own code. Does not really matter what you use (shell scripts, make, just, doit, mage, whatever) as long as it is proper, maintainable code.
* Invest time that your pipelines can run locally on a developer machine as well (as much as possible at least), otherwise testing/debugging pipelines becomes a nightmare.
* Avoid YAML as much as possible, period.
* Don't bind yourself to some fancy new VC-financed thing that will solve CI once and for all but needs to get monetized eventually (see: earthly, dagger, etc.)
* Always use your own runners, on-premise if possible
- Treat pipelines as code. - Make pipelines parts composable, as code. - Be mindful of vendor lock-in and/or lack of portability (it is a trade-off).
For on-promise: if you're already deeply invested in running your own infrastructure, that seems like a good fit.
When thinking about how we build Namespace -- there are parts that are so important that we just build and run internally; and there are others where we find that the products in the market just bring a tremendous amount of value beyond self-hosting (Honeycomb is a prime example).
Use the tools that work best for you.
At Namespace (namespace.so), we also take things one step further: GitHub jobs run under a cgroup with a subset of privileges by default.
Running a job with full capabilities, requires an explicit opt-in, you need to enable "privileged" mode.
Building a secure system requires many layers of protection, and we believe that the runtime should provide more of these layers out of the box (while managing the impact to the user experience).
(Disclaimer: I'm a founder at Namespace)