Now that everybody and their mother are fuzzing in social media about LLM agents and agentic LLM systems (or something), are there actual examples of live applications that are based on an agentic LLM process flow?
I'd be curious to know and see such examples in order to derive some inspiration from them.
I do contracting work, we're building a text-to-sql automated business analyst. It's quite well-rounded: it tries to recover from errors, allows automatic creation of appropriate visualisations, has a generic "faq" component to help the user understand how to use the tool. The tool is available to some 10.000 b2b users.
It's just a bunch of prompts conditionally slapped together in a call graph.
The client needed AGENTIC AI, without specifying exactly what this meant. I spent two weeks pushing back on it, stating that if you replace the hardcoded call graph with something that has """free will""", accuracy and interpretability goes down whilst runtimes go up... but no, we must have agents.
So I did nothing, and called the current setup "constrained agentic ai". The result: High fives all around, everyone is happy
Make of that what you will... ai agents are at least 90% hype.
I've implement countless LLM based "agentic" workflows over the past year. They are simple. It is a series of prompts that maintain state with a targeted output.
The common association with "a floating R2D2" is not helpful.
They are not magic.
The core elements I'm seeing so far are: the prompt(s), a capacity for passing in context, a structure for defining how to move through the prompts, integrating the context into prompts, bridging the non-deterministic -> deterministic divide and callbacks or what-to-do-next
The closest analogy that I find helpful is lambda functions.
What makes them "feel" more complicated is the non-deterministic bits. But, in the end, it is text going in and text coming out.
Are you using a separate state manager + function calling so the LLM knows where it is?
I love those stories but I could never do that with a straight face. However, the AI field is such an uphill battle against all the crap that LinkedIn influencers are pushing into the minds of the C-suite... I feel it's okay to get a bit creative to get a win-win here ;)
I guess everything with an algorithm in it is AI if you look at it from enough of a distance...
Everything is very boring tech-wise, using vanilla postgres/pgvector and a few hundred lines of python. Every RAG-searchable text field (mostly column descriptions and a list of LLM-generated example queries) is linked to nodes holding metadata, at most 2 hops out. The tool is available to 10.000 users, but load is only a few queries per minute at peak... so performance wise it's fine.
Hope you can find a tool; the big data players are of course jumping on this (snowflake, databricks, they all talk about their text-to-sql tools).
If you have the budget and want something bespoke built that has some magic sauce tuned to your exact problem field, send me an email!
The right approach to build toward agents is to start with something that gives pretty good responses to prompts and build up an agentic mode to let it do more and more in response to each prompt. It should be thought of as extending how much you get per prompt, and doing so by chaining together components you've already worked at making to good at.
Cursor (the LLM powered VS Code fork) has an agentic mode and they are doing this the right way. The normal chat window is good at producing changes to your code, and at applying them, at looking at lints, at suggesting terminal commands, at doing directory listings or RAG on your codebase. Agentic mode is tying those together to do more of the work you want with fewer prompts from you.
1. Its still perceived as an issue of competitive advantage
2. There is a serious concern about backlash. The public's response to finding out that companies have used AI has often not been good (or even reasonable) -- particularly if there was worker replacement related to it.
It's a bit more complicated with "agents" as there are 4 or 5 competing definitions for what that actually means. No one is really sure what an 'agentic' system is right now.
This is the only one that makes sense. People want to conflate it with their random vague conceptions of AGI or ASI or make some kind of vague requirement for a certain level of autonomy, but that doesn't make sense.
An agent is an agent and an autonomous agent is an autonomous agent, but a fully autonomous agent is a fully autonomous agent. An AGI is an AGI but an ASI is an ASI.
Somehow using words and qualifiers to mean different specific things is controversial.
The only thing I will say to complicate it though is if you have a workflow and none of the steps give the system an option to select from more than one tool call, then I would suggest that should be called an LLM workflow and not an agent. Because you removed the agency by not giving it more than one option of action to select from.
I implore you to look into that to see how some people relate it to autonomy or AGI or ASI(wrongly, imo - I think shoehorning OOP and UML diagrams plus limited database like memory/context is not a path to AGI. Clever use of final layers, embeddings and how you store/weight them (and even more interesting combinations) may yield interesting results because we can (buzzword warning) transcend written decoding paradigms - the Human brain clearly does not rely on language).
However what gets marketed today is, as you say, not capable of any real agent autonomy like in academia - they are just self-recursive ChatGPT prompts with additional constraining limits. One day it might be more, but libraries now are all doing that from my eye. And recursion has pros but emphasizes the unreliability con/negative of LLMs.
For me, agentic means that at least at some stage, one model is prompting another model.
I look at my workplace and I see places where they might fit in but if the reliability isn’t 99.5% they won’t be trusted and I think that’s a problem.
I made a toy in n8n that collects transactions in YNAB via API and matches them to Amazon orders in GMail. It then uses GPT-4o with vision to categorize the product pictures according to my budget’s categories but I have to add the order link to the transaction memo and add a flag for human review because it’s only 80% or so. It has sped up the workflow for sure but nowhere near good enough to set it and forget it.
I had no idea about Make.com or n8n, they seem interesting. Thanks for the tip! Will check them out.
And non ui flow diagram but essentially the same thing: inngest, hatchet.
- Auth bypass/arbitrary file read in Scoold: https://xbow.com/blog/xbow-scoold-vuln/
- SSRF in 2FAuth: https://xbow.com/blog/xbow-2fauth-ssrf/
- Stored XSS in 2FAuth: https://xbow.com/blog/xbow-2fauth-xss/
- Path traversal in Labs.AI EDDI: https://xbow.com/blog/xbow-eddi-path/
Each of those has an associated agent trace so you can go read exactly what the agent did to find and exploit the vulnerability.
If you ask two different people in the AI space to define "agent" you almost always get two slightly (or significantly) different definitions!
Here are just some of the definitions I've seen over time: https://news.ycombinator.com/item?id=42216217#42228364
For the purpose of this thread the most cynical definition, "LLMs that do something useful", might actually be the best fit!
Deleted Comment
Agentic workflows are a much higher bar that are just barely starting to work. I can't speak to their efficacy but here's a few of the ones that are sort of starter-level agents that I've started seeing some companies adopt:
- https://www.intercom.com/fin
- https://www.rox.com/
- https://devin.ai/
- https://bolt.new/
- https://v0.dev/
Last week Wednesday I participated in Anthropic's Model Context Protocol hackathon, and built a system with my team partner Zia to automatically search and find restaurants for your dietary preferences and group size.
It also automatically downloads social media of the restaurant to get a vibe for the place.
There's a video of it in action here: https://www.youtube.com/watch?v=c6vGrfHFyu8
And a Github repo here: https://github.com/zia-r/gotta-eat