Readit Newsmarcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
A context inference service sounds valuable but I wonder what your moat would be.
Yep, that's something I'd have to figure out.
marcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
I think MCP is the right place to declare the context management API; the C in MCP is Context. As far as building goes, you could build a (universal) context store. I guess the value would be to bring the context closer to the model?
The value is building the context itself.
Using MCP, this could be a method that would get the context to take decisions.
For example, here's an example of how I use it currently:
```
const context = await getContext();
const response = await generateText({ system: `Here's the user context: ${context}`, messages: [...] });
console.log(context);
// "First-time visitor using Google Chrome on a MacBook, browsing from San Francisco.
// Landed on the pricing page from a Google ad, clicked to compare plans,
// then visited the enterprise section before initiating a support chat."
```
It's like a session recorder for LLMs that captures rich user behavior and traits (like device, browser, location, and journey) and turns them into LLM context. Your agent or app instantly becomes more helpful, relevant, and aware without wiring up your own tracking and enrichment pipeline.
marcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
I haven’t solved this, but sounds super useful!
Would love to have something like a hotjar/analytics script that could automatically collect context and then I could query it to produce context for a prompt
Great idea, you should build it. Then do a Show HN with it
Exactly! Something like a tag you install and then query prompt-ready contexts.
marcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
You might find this useful: https://context7.com/
Super interesting! However, it focuses on external sources rather than the user journey.
marcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
I embed tons of separate pieces of information, save the vectors in a db. Embed the user's question, then have a stored procedure in the db to calculate the top 10 (or 20 or 50 depending on the model) similar pieces of information.
I have an editor where I can ask a question and it brings up the most related pieces of info, and if I change any of those pieces it will update the embedding in the db
That's a good approach. But what I'm looking for is a bit different, more like Segment, but for LLMs. Something that when a user lands on your website, clicks around, and interacts with your app, you get a full behavioral context out of the box, including click path, location, language, currency, etc. You can then inject that context directly into your prompt so the LLM understands what the user is doing and responds without guessing or asking.
marcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
MCP maybe? You could provide tools for the LLM to discover that data at runtime.
It might help with context generation. But honestly, most of the work is still in tracking, processing, enriching (different services, like IP location, etc), and all the plumbing around it.
marcospassos commented on Ask HN: Building LLM apps? How are you handling user context? · Posted by u/marcospassos
I don't know of anything off-the-shelf, but you could query analytics tools at runtime (e.g. Mixpanel, PostHog) to gather the raw data, and use a generic summarizer to turn that into behavioral context that's usable downstream.
Yeah, exactly. My whole point is to avoid doing all that. It adds up fast. What I really want is something that handles the heavy lifting end-to-end: tracking, interpreting, and outputting a prompt-ready summary like:
"The user landed on the pricing page from a Google ad, clicked to compare plans, then visited the enterprise section before initiating a support chat."