What’s the motivation of the CEO to increase employee wages if his compensation isn’t tied to theirs?

There’s this perverse belief that companies should exist to enrich the wealthy shareholders at the expense of the workers and it’s put us dangerously close to a complete collapse of the social contract.

How does this not lead to a situation where no honest person can use any AI in their submissions? Surely pull requests that acknowledge AI tooling will be given significantly less attention, on the grounds that no one wants to read work that they know is written by AI.

The vast majority of feedback on the GitHub issue was respectful — unless you consider opposing the proposal disrespectful.

> @whatwg whatwg locked as too heated and limited conversation to collaborators

Too heated? Looked pretty civil and reasonable to me. Would it be ridiculous to suggest that the tolerance for heat might depend on how commenters are aligned with respect to a particular vendor?

Not at all.

There are mainly 2 forms of AD: forward mode (optimal when the function being differentiated has more outputs than latent parameter inputs) and reverse mode (when it has more latent parameter inputs than outputs). If you don't understand why, you don't understand AD.

If you understand AD, you'd know why, but then you'd also see a huge difference with symbolic differentiation. In symbolic differentiation, input is an expression or DAG, the variables being computed along the way are similar such symbolic expressions (typically computed in reverse order in high school or uni, so the expression would grow exponentially with each deeper nested function, and only at the end are the input coordinates filled into the final expression, to end up with the gradient). Both forward and reverse mode have numeric variables being calculated, not symbolic expressions.

The third "option" is numeric differentiation, but for N latent parameter inputs this requires (N+1) forward evaluations: N of the function f(x1,x2,..., xi + delta, ..., xN) and 1 reference evaluation at f(x1, ..., xN). Picking a smaller delta makes it closer to a real gradient assuming infinite precision, but in practice there will be irregular rounding near the pseudo "infinitesimal" values of real world floats; alternatively take delta big enough, but then its no longer the theoretical gradient.

So symbolic differentiation was destined to fail due to ever increasing symbolic expression length (the chain rule).

Numeric differentiation was destined to fail due to imprecise gradient computation and huge amounts (N+1, many billions for current models) of forward passes to get a single (!) gradient.

AD gives the theoretically correct result with a single forward and backward pass (as opposed to N+1 passes), without requiring billions of passes, or lots of storage to store strings of formulas.

Isn't it just kinda a natural thing once you have the chain rule?

> ability to fund shadow libraries without fear of censorship

Bitcoin is much worse than cash in that regard

GPT-2 was the first wake-up call - one that a lot of people slept through.

Even within ML circles, there was a lot of skepticism or dismissive attitudes about GPT-2 - despite it being quite good at NLP/NLU.

I applaud those who had the foresight to call it out as a breakthrough back in 2019.