> I remain sceptical about any form of combination of reinforcement learning and LLMs.

I'm inclined to agree with this. And if we treat language learning as a paradigmatic example of a non-trivial problem, it certainly requires extensive environmental interaction (regardless of the stance taken on the degree to which language is "modular" - hopefully the basic statement that "real-world language learning is hard and situational" is non-controversial).

The intention of the linked article is to start from a good-faith position on claims of plan-making AGI, and then to say, "Okay, we've got this thing that outputs language that is supposed to reflect deeper knowledge of the ways of the world. If we don't have access to some underlying structure that convinces us of its capabilities, what enquiries can we make of the way it deals with somewhat known aspects of language itself that might at least begin to point to some of the necessary conditions for concept-formation and plan-making?"

The post outlines what are intended to be some fairly concrete tests that could be run in a systematic empirical way (or also preliminarily prodded in a more ad hoc way by anyone with access to an LLM). Like you, I'm pretty sceptical of a strong positive outcome for a model trained on nothing other than a very large sample of distributional data about linguistic symbols. But hopefully this is an interesting starting point for giving an AGI a chance to prove itself.