Lock-free data structures and algorithms access shared memory via various atomic operations such as compare-and-swap and atomic arithmetic. The throughout of these operations do not scale with the number of CPU cores. Contrary, the throughput usually reduces with the growing number of CPU cores because they need more time for synchronizing local per-CPU caches with the main memory. So, lock-free data structures and algorithms do not scale on systems with big number of CPU cores. It is preferred to use "shared nothing" data structures and algorithms instead, where every CPU core processes its own portion of state, which isn't shared among other CPU cores. In this case the local state can be processed from local per-CPU caches at the speed which exceeds the main memory read/write bandwidth and has smaller access latency.

Performance drop for exposing application metrics in Prometheus format is close to zero. These metrics are usually some counters, which are updated atomically in a few nanoseconds. Prometheus scrapes these metrics once per 10-30 seconds, so generating the /metrics response in Prometheus text exposition format doesn't need a lot of CPU, especially if using a library optimized for simplicity and speed like https://github.com/VictoriaMetrics/metrics/

These tricks are essential for every database optimized for metrics / logs / traces. For example, you can read on how VictoriaMetrics can compress production metrics to less than a byte per sample (every sample includes metric name, key=value labels, numeric metric value and metric timestamp with millisecond precision). https://faun.pub/victoriametrics-achieving-better-compressio...

This is called "progress". Humans always generate the amounts of data which can be stored and processed by the tools they have. The more data the tool can process under the given budget limit, the more data will be generated and stored.