A couple of months ago we were building agents that interacted with different apps and were frustrated when they struggled to handle vague natural language requests like "resolve that one Linear issue about missing auth configs", "if you get an email from an unsatisfied customer, reimburse their payment in Stripe", or "what were the returns for Q1 based on the financials sheet in gdrive?", only to have the agent inefficiently chain together loads of function calls to find the data or not find it at all and hallucinate.
We also noticed that despite the rise of MCP creating more desire for agents to interact with external resources, the majority of agent dev tooling focused on function calling and actions instead of search. We were annoyed by the lack of tooling that enabled agents to semantically search workspace or database contents, so we started building Airweave first as an internal solution. Then we decided to open-source it and pursue it full time after we got positive reactions from coworkers and other agent builders.
Airweave connects to productivity tools, databases, or document stores via their APIs and transforms their contents into searchable knowledge bases, accessible through a standardized interface for the agent. The search interface is exposed via REST or MCP. When using MCP, Airweave essentially builds a semantically searchable MCP server on top of the resource. The platform handles the entire data pipeline from connection and extraction to chunking, embedding, and serving. To ensure knowledge is current, it has automated sync capabilities, with configurable schedules and change detection through content hashing.
We built it with support for white-labeled multi-tenancy to provide OAuth2-based integration across multiple user accounts while maintaining privacy and security boundaries. We're also actively working on permission-awareness (i.e., RBAC on the data) for the platform.
So happy to share learnings and get insights from your experiences. looking forward to comments!
Deleted Comment
Typically speaking an LLM is the code driving the control flow and the MCP servers are kind of dumb API endpoints (find_flights, search_hotels, etc) say for a travel MCP.
With your product, how is the LLM made aware of the underlying data store in a more useful way than “func search(query)”?
It seems to be that if you could expose some precomputed API structure into the MCP for a given data store then the LLM could reason more effectively about the data rather than throwing search queries into the void and hoping for the best?
So since its all represented by entities, you could treat them like any other vectorised data in your vector data store and use vector search.
It's a nice technique, but probably tricky if they ever venture into encapsulating endpoints in realtime for rapidly changing b2c applications (ratelimits/cronjob latency)
Also, are these one-time/event-based syncs well supported by the integration providers? I know for instance that discord (and i assume others like slack) frown upon that sort of wholesale archival/syncing of entire chat rooms, presumably due to security concerns and to maintain their data moats.
Finally (i think), do you have to write custom "diff" logic for each integration in order to maintain up-to-date retrieval for each one? I assume it would be challenging to keep this accurate and well structured across so many different integration providers. Is there something i'm missing that makes keeping a local backup of your data easier for each service?
All in all, looks very cool. Have starred the repo to mess around with tonight.
1) the integrations are done traditionally so with REST/SQL. The MCP/REST search layer rests on the data that gets synced.
2) most providers are painless. Slack doesn’t want major exports in one go but most developers point at a single channel anyway so the rate limit errors don’t bite too much.
3) this is all orchestrated by the platform itself. Incremental syncs will receive the latest “watermark state” and sync from there. Hashes are used to compare data for persist actions (update/insert/keep)
Don’t they just adjust existing apis to mcp protocol basically just wrapping them?