Prompt: Write an interpreter for a simple but practical scripting language. Write the interpreter in JavaScript to be run on the Node.JS platform. You can import any of the standard Node.JS modules.
Churned out ~750 lines and a sample source code file to run the interpreter on. Ran on the first try completely correctly.
Definitely a step up. Perhaps it's in the training data. I don't know. But no other model has ever produced an error-free and semantically correct program on the first try, and I don't think any ever managed to implement closures.
Also tried 1973 — same (The Day of the Jackal, Soylent Green, Westworld, The Wicker Man, Papillon, American Graffiti, The Sting, Serpico, Mean Streets, High Plains Drifter, Don't Look Now, Badlands, The Long Goodbye, Jesus Christ Superstar, The Three Musketeers, Fantastic Planet, etc.).
I think they simply made better movies decades ago.
No he has a team that uses computers to find out those plays based on what other player played as all past matches are available.
Source: I watched interview with a guy that was hired as a computer scientist consulting gig by Magnus team.
It does not take away how good he is as I don’t think many people could learn to remember weird openings and win from that against grand master level players anyway.