> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.
How do you fix a waterlogged smartphone? Put out a bowl of rice, which attracts an Asian guy who will repair it for you.
Languages have pedigrees. If you pretend like your company is enamored with javascript you'll get people who love fedoras and call themselves Ninjas. Big teams, single function libraries, lots of code shipped - move fast and break things. Cats pawing at Macbook keyboards. Mumble rap.
If you pretend you love Haskell you'll attract mathematicians in elbow patches. Great stable code will sporadically appear once every couple of years seemingly at random. Genius solutions to neat problems that have nothing to do with what the company is actually trying to accomplish. Ents. Classical music.
If you pretend to love lisp you'll attract people who read PG essays and will quit to start their own companies. Maybe they'll help you close out some tickets in Jira before they bounce if they can get your Rube Goldberg monstrosity working on their laptop. Honey badgers and hamsters. U2.
If you pretend to love latin you might get elected PM.
If you actually learn a few orthogonal languages to cover the very finite amount of paradigms you'll eventually come to realize they are all crap.
If you want to code, code. Don't talk.
{ ⊃ 1 ω ∨ . ∧ 3 4 = +/ +⌿ 1 0 ‾1 ∘.θ 1 - ‾1 Φ″ ⊂ ω }