Same but then I saw "only 2 GB image"

I also think when looking at how BWT works, it is initially surprising it is invertible even with such little information as the first character. It appears a bit like sorting the letters and that certainly would require a lot more information to back into place. It is a bit magic even when you know the inversion algorithm.

I think you should drop the "stochastic text transformer" label you have probably heard applied, and instead think of them as neural networks that they are. Reason being that the term says absolutely zero about capabilities but creates a subjective 'reduction'. It's just a thought terminating cliché.

Let's for the sake of argument assume current LLM's are a mirage but in the future some new technology emerges that offers true intelligence and true reasoning. At the end of the day such a system will also input text and output text, and output will probably piece-meal as current LLM's (and humans) do. So voila: They are also "stochastic text transformers".

Yes LLM's were trained to predict next token. But clearly they are not just a small statistical table or whatever. Rather, it turns out that to be good at predicting the next token, after some point you need a lot of extra capabilities, so that's why they emerge during training. All the "next-token-prediction" is just a way abstract and erasing name of what is going on. A child learning how to write, fill in math lessons etc. is also learning 'next token prediction' from this vantage point. It says nothing about what goes on inside the brain of the child, or indeed inside the LLM. It is a confusion between interface and implementation. Behind the interface getNextToken(String prefix) may either be hiding a simple table or a 700 billion-size neural network or a 100 billion sized neuron human brain.

Java's new philosophy (in "Loom" - in production OpenJDK now) seems to be virtual threads that are cheap and can therefore be plentiful compared to native threads. This allows you to write the code in the old way without programmer-visible async.

As to the complexity: It is complex because it is very low-level. In JavaScript (using that as an example but I suspect Python is the same) they build in async/await keywords such that they are in cahoots with the Promise class. C++ takes a different path where there isn't a built-in Promise class, rather it provides you lower-level primitive you can use to build a Promise class. You can build a library around it, and it will be as simple as in other languages - both for the awaiter and for implementing libraries that you can await on :) But I agree it is really complicated. I remember once in a while thinking "ah it can't really be that complicated" only to dive into it again. It doesn't help that practically any term they use (promise, awaiter etc.) is used differently than in all other contexts I've worked. If you just expect it will be as easy as understand JavaScript async/await/promise you are in for a rude surprise. Raymond Chen has written co-routines tutorials which span three SERIES. Here's a map of those: https://devblogs.microsoft.com/oldnewthing/20210504-01/?p=10...

As for how we got to here without:

1) Using large number processes/threads 2) Raw callback oriented mechanisms (with all the downsides) 3) Structured async where you pass in lambda's - benefit is you preserve the sequential structure and can have proper error handlign if you stick to the structure. Downside is you are effectively duplicating language facilities in the methods (e.g. .then(), .exception() ). Stack traces are often unreadable. I. 4) Raw use of various callback-oriented mechanisms like epoll and such, with the cost in code readability etc. and/or coupled with custom-written strategies to ease readability (so a subset of #3 really)

With C++ couroutines the benefit is you can write it almost like you usually do (line-by-line sequentially) even though it works asynchronously.

What I've seen so far suggest they were just ignorant and victims of confirmational bias etc. You can see that when they won some cases they wrote internally something to the effect of "Final we can put to rest all those concerns about these cases blablabla". So it became self-validating. Also the courts and defense lawyers didn't manage to the see the pattern and in the huge numbers of such cases. Each defendant was fighting their own battle. Also, a mathematician from Fujitsu gave "convincing" testimony they didn't have any errors. A lot was down to lack of understanding of how technology works. The fact that xx millions of transactions were processed without errors doesn't preclude that there could be errors in a small number, as was the case. In this case sometimes coming down to random effects like if race conditions were triggered.