To contrast with this, my org tried using a simple QA bot for internal docs and has been struggled to move anything beyond proof of concept. The proof of concepts have been awful. It answers maybe 60-70% of questions correctly. The major issue seems to be related to taking PDFs laced with images and poorly written explanations. To get decent performance from these RAG bots, a large FAQ has to be written for every question it gets wrong. Of course this is just my org so it can’t necessarily be extrapolated across industry. However, how often have people come across a new team and find there is little to no documentation, poorly written documentation, or outdated documentation?
Where am I going with these two thoughts? Maybe the blocker to pushing more adoption within orgs is twofold, getting the correct context into the model and having decent context to start with.
Extracting value from these things is going to require a heavy lift in data curation and developing the harnesses. So far most of that effort has gone into coding. It will take time for the nontechnical and technical to work together to move the rest of an org into these tools in my opinion.
The big bet of course then is ROI and time to adoption vs current burn rates of the model providers.
If we consider that the real major's move about 400k-500k passengers/day, let's be really optimistic and say that they check their booking 6 times a day for the week before they fly. That's around 250 requests/sec.
Anyone know about the consumer facing tech stacks at airlines these days? Seems unlikely that they'd have databases that would auto scale 400x...
I think more likely the API would be behind some kind of bot protection that would shut this down before any kind of technical rate limit is reached.