Readit NewsInstallation for GPU is covered here: https://github.com/chrisbutner/ChessCoach#installation (a little messy, sorry)
Partly because of the way it tries to search more widely to avoid tactical traps, it can also be a little sloppy in holding advantages or minimizing losses (this could use some more work and tuning). This ends up making it a little drawish, so it loses less than you'd expect to Stockfish 14, but also doesn't beat up weaker engines as well as Stockfish 14 does.
You can see some of this in the raw tournament results[1]. At 40 moves per 15 minutes, repeating, each engine draws with the ones above and below it, but starts to win and lose at a distance of 2 or 3.
At 5+3 time control, ChessCoach goes 1-0-29 vs. Stockfish 12, but Stockfish 12 is better at beating Stockfish 8-11 than ChessCoach is, so CC ends up between SF11 and SF12 in the end.
On Lichess, where there's no "free time" to get ready for searches, ChessCoach's naïve node allocation/deallocation makes it waste time, and means it can't ponder for very long on the opponent's time - a big opportunity for improvement (it needs a multi-threaded pool deallocator that can feed nodes back to local pools for the long-lived search threads). I think it's also hitting a bug with Syzygy memory mapping that Stockfish works around via reloading every "ucinewgame" (which I don't trigger on Lichess). So, overall, its performance on Lichess is worse.
Also, you can't read too much into this data - very few games, and no opening book.
[1] https://chrisbutner.github.io/ChessCoach/data.html#appendix-...
Actually, I can't figure out from your explanation why you trained the whole network yourself instead of just using Leela's network and training the commentary head on top?
If you wanted to in-cooperate the search, maybe you could just take the 1800 or so probabilities output by the MCTS and add some layers on top of that before concatenating with the other data fed into the transformer.
In either case, this is a fantastic project and perhaps an even more impressive write up! Congrats and thank you!
In terms of search-into-commentary, concatenating like that may be interesting, as long as it can learn to map across - definitely plausible without too much work. I was originally thinking of something more complicated, combining multiple raw network outputs across the tree through some kind of trained weighting, or additional model via recurrence, and punted it.
Ignore my BLEU comment, mixed those up between replies - that was the other potential use of search trees for commentary, an MCTS/PUCT-style alternative to traditional sequential top-k/top-p sampling, once you have logits and are deciding which paragraph to generate.
Thanks!
how much does a game cost in CPU time money?
How do I get the commentary for a game I played? Oh, it's in Analysis page.
It plays chess very well, but the commentary is incoherent and doesn't match the game well -- The attacks described are nonsense and the coordinates are wrong. It seems a little confused about which side is which? It thinks a rook can diagonally attack a bishop, and seems to name squares opposite from their actual name.
Costs are difficult to work out - it depends on cloud vs. self-hosting, what kind of TPUs/GPUs, how long you're calculating over.
The advantage that classical/NNUE engines have is that they can more easily spread over distributed frameworks like Fishtest.
Getting deep into RL specifically wasn't so necessary for me because I was just replicating AlphaZero there, although reading papers on other neural architectures, training methods, etc. helped with other experimentation.
You may be well past this, but my biggest general recommendation is the book, "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" to quickly cover a broad range of statistics, APIs, etc., at the right level of practicality before going further into different areas (for PyTorch, I'm not sure what’s best).
Similarly, I was familiar with the calculus underpinnings but did appreciate Andrew Ng's courses for digging into backpropagation etc., especially when covering batching.
Next steps could be using one of Lc0's backends for GPU scenarios, or taking the other side of the trade and using the C++ API for TPU.
There's also your typical CPU and memory optimizations that could be made - some baseline work there but not targeted.
I had a nice long conversation with two of the authors of [0] at ACL.
One thing we discussed was the reverse problem. That is, as a player, could I give commands to the model and have the engine figure the moves that would best satisfy them.
This ranges from concrete like "take the black square bishop" (there is still variability like which piece should take it or if it's even possible) to more complex positional stuff like "set up to attack the kingside."
Any thoughts on this line of research?
[0] Automated Chess Commentator Powered by Neural Chess Engine (Zang, Yu & Wan, 2019) https://arxiv.org/pdf/1909.10413.pdf
I think this line of thinking could eventually lead to automated metrics for commentary evaluation, which could in turn lead to better methods than top-k/top-p for turning a bunch of sequential logits into a sentence or paragraph - basically treat it like MCTS/PUCT also.
The problem is that if you look at high-level commentary - maybe Radjabov-MVL on https://www.chess.com/news/view/2021-champions-chess-tour-fi... (I'm not the best judge, just a quick search) - it's not often possible to predict the move starting with the comment. And if you did, you might end up with very dry metrics and reverse commentary.
But this direction has a lot of potential I think, beyond just chess, into more of an algorithmic/generational support for pure NN-based language models.
I wonder if we are getting snippets of variations in some cases.
That is, in addition to the usual evaluation and policy heads, this takes the intermediate board representation and outputs a seed vector that is fed into a transformer text generator?
Or do other things go into the seed? Like the search tree somehow? Otherwise I suppose the commentary will not be able to comment on deeper tactics?
Or maybe this doesn't work using a seed vector at all, but with a custom integration from the board into the transformer somehow?
So, now the commentary decoder is just trained separately on the final primary model. The previous and current game positions are fed into the primary model, and the outputs are taken from the final convolutional layer, just before the value and policy heads. Then, that data plus the side to play is positionally encoded and fed into a transformer decoder.
It would be better for a search tree/algorithm to be used for commentary too so that tactics could be better understood, but that would need some kind of subjective BLEU equivalent, and metrics like those don't work well for chess commentary.
You can see a diagram of the architecture here: https://chrisbutner.github.io/ChessCoach/high-level-explanat...
I'm particularly interested in the second neural net that generates explanations. Is it only a neural net to generate the natural language using artifacts from the original engine net, or is it actually inspecting the state of the engine net to derive the insights?
It's an interesting application of explainability of AI algorithms that Christian talks about in his chapter on transparency. In particular, he discusses "saliency" of algorithms (knowing what parts of the input were most important in producing a prediction) and "multitask nets" that output multiple predictions (so, maybe here, one output is the best move, and another output is the explanation).
The writeups are fantastic reading. I see almost no sources in common with the bibliography in The Alignment Problem (which has a 50 page bib) which makes them nice complements. Only common citations I could find were Sutton 1988 (Temporal Differences) and Silver et al 2016 (AlphaGo), 2017 (AlphaGo Zero) and 2018 (AlphaZero).
There's a note (44) from chapter 5 entitled "Shaping" where Christian talks about "meta-reasoning: the right way to think about thinking. When you play a game–for instance, chess–you win because of the moves you chose, but it was the thoughts you had that enabled you to choose those moves...Figuring out how an aspiring chess player–or any kind of agent–should learn about its thought process seemed like a more important but also dramatically harder task than simply learning how to pick good moves." This note might as well be direct inspiration for this project. It goes on to quote Stuart Russell about "a computation that changes your mind about what is a good move to make...reward that computation by how much you changed your mind...so you could change your mind in the sense of discovering that what was the second-best move is actually even better than what was the best move." That's in the context of a cautionary tale where only optimizing for those "changes of mind" doesn't necessarily find a correct outcome, and that you have to "arrange these internal pseudorewards so that along a path, they add up to the same as true, eventually." This sounds pretty much like the task of a coach.
Deeper introspection is a really important goal, but by the time you make serious progress there, chess is the least of your worries.
I do really like the work people have put into introspection and visualization so far though: DeepDream comes to mind. There was also another great paper or page that I can't find.