It can't be successful at that any more than 1+1 can equal 3. Fundamentally, if every token wants to be able to look at every previous token without loss of information, it must be O(n^2); N tokens looking at N tokens is quadratic. Any sub-quadratic attention must hence necessarily lose some information and be unable to support perfect recall on longer sequences.
> N tokens looking at N tokens is quadratic
Convolving two arrays can be done perfectly accurately in O(n log n), despite every element being combined with every other element.
Or consider the even more basic sum of products a[i] * b[j] for all possible i, j:
total = 0
for i in range(len(a)):
for j in range(len(b)):
total += a[i] * b[j]
Your logic that 'the result contains terms of all pairs, therefore the algorithm must be quadratic' simply doesn't hold.
> “Scattering” is the scientific term of art for molecules deflecting photons. Linguistically, it’s used somewhat inconsistently. You’ll hear both “blue light scatters more” (the subject is the light) and “atmospheric molecules scatter blue light more” (the subject is the molecule). In any case, they means the same thing
There's nothing ambiguous or inconsistent about this. In English a verb is transitive if it takes one or more objects in addition to the subject. In "Anna carries a book", "carries" is transitive. A verb is intransivite if it takes no object as with "jumps" in "The frog jumps.".
Many verbs in English are "ambitransitive" where they can either take an object or not, and the meaning often shifts depending on how it's used. There is a whole category of verbs called "labile verbs" where the subject of the intransitive form becomes the object of the transitive form:
* Intransitive: The bell rang.
* Transitive: John rang the bell.
"Scatter" is simply a labile verb:
* Intransitive: Blue light scatters.
* Transitive: Atmospheric molecules scatter blue light more.
If something "carries" or is "a carry", it means it is so strong it metaphorically carries the rest of the setup with it. For example:
> This card carries.
> These two are the carries of the team.