There's a paragraph about alcohol.
As it happens LLMs work comparatively well with code. Is this because code does not refer (a lot) to the outside world and fits well to the workings of a statistical machine? In that case the LLMs output can also be be verified more easily by inspection through an expert, compiling, typechecking, linting and running. Although there might be hidden bugs that only show up later.
Like any technical 25 page report it'll be ballpark with reality, shorter to read and grasp than crawling through a wall of document filled boxes, and passed to other people to 'verify' / offer their opinions on.
Once contracts are in place with millions of dollars in play (or tens of millions, or billions) there will be clauses addressing responsibility and recompense should key parts of the reports upon which an agreement is based prove to be false.
The world runs on technical reports that aren't perfect, but "near enough"; errors are assumed and a frequency of deliberate malfeasance (knowingly lying, misleading, faking results) can be estimated.
Part of my career consisted of producing summaries of two to three thousand documents a day from stock markets about the globe, documents that ranged from three lines announcing a change on a board, a table disclosing a change in holdings by largest investors, etc. to large (hundred+ page) quarterly and annual reports, to small book economic feasibility reports with wads of raw data, interpretation, proposed plans, costings, timelines, etc.
> It will strip through our documents and the legislation and produce a 25-page document for a client as a first draft.
is the key point here, it's a rapid first draft of the major dot points seen to be most important for <whatever>. It is intended to be crawled through with a finer comb and a keen eye before contracts are signed based on a separate framing of <deal>.
The big change here is that an AI churns out a draft faster, the quality of the document will be as suspect as a non AI created human first draft .. untrusted.
"That speed is important," he added. "If we have a client who is about to do a merger, and they want to understand the tax implications, getting that knowledge in a day is much more important than getting it in two weeks' time."
---
I really wonder what is the foundation for their confidence in LLMs. If you have ever used ChatGPT you will be highly skeptic that the output is correct. If it's code, you can at least compile, typecheck, run it, to verify it to some extent. How do you do that with a 25 page report?
We will continue iterating Le Lamp to make it expressive and useful in the upcoming weeks.
our github: https://github.com/humancomputerlab/LeLamp/
community: https://discord.gg/wVF99EtRzg