One of the biggest lessons for me while riding the MCP hype was that if you're writing backend software, you don't actually need to do MCP. Architecturally, they don't make sense. Atleast not on Elixir anyway. One server per API? That actually sounds crazy if you're doing backend. That's 500 different microservices for 500 APIs. After working with 20 different MCP servers, it then finally dawned on me, good ole' function calling (which is what MCP is under the hood) works just fine. And each API can be just it's own module instead of a server. So, no need to keep yourself updated on the latest MCP spec nor need to update 100s of microservices because the spec changed. Needless complexity.
Unless the client speaks to each micro service independently without going through some kind of gateway or BFF, I'd expect you'd just plonk your MCP in front of it all and expose the same functionality clients call via API (or graphql or RPC or whatever), so it's basically just an LLM-specific interface.
No reason why you couldn't just use tool calls with OpenAPI specs though. Either way, making all your microservices talk to each other over MCP sounds wild.
> One server per API? That actually sounds crazy if you're doing backend
Not familiar with elixir, but is there anything prohibiting you from just making a monolith MCP combining multiple disparate API's/backends/microservices as you were doing previously?
Further, you won't get the various client-application integrations with MCP merely using tool-calling; which to me is the "killer app" of MCP (as a sibling comment touches on).
(I do still have mixed feelings about MCP, but in this case MCP sorta wins for me)
> just making a monolith MCP combining multiple disparate API
This is what I ended up doing.
The reason I thought I must do it the "MCP way" was because of the tons of YouTube videos about MCP which just kept saying how much of an awesome protocol it is, an everyone should be using it, etc. Once I realized it's actually more consumer facing than the backend, it made much more sense as why it became so popular.
In the end, MCP is just like Rest APIs, there isn't need for a paid service for me to connect to 400 Rest APIs now, why do I need a service to connect to 400 MCPs?
All I need for my users is to be able to connect to one or two really useful MCPs, which I can do myself. I don't need to pay for some multi REST API server or multi MCP server.
nothing prevents you from using a proxy server if you want it all from one entry point. idk of any clients hitting hundreds of MCPs anyways, that's a straw man.
unlike web dev where you client cab just write more components / API callers, AI has context limits. if you try to plug (i get it, exaggerated) 500 MCP servers each with 2-10 tools you will waste a ton of context in your sys prompt. you can use a tool proxy (one tool which routes to others) but that will add latency from the processing.
Forcing LLMs to output JSON is just silly. A lot of time and effort is being spent forcing models to output a format that is picky and that LLMs quite frankly just don't seem to like very much. A text based DSL with more restrictions on it would've been a far better choice.
Years ago I was able to trivially teach GPT 3.5 to reliably output an English like DSL with just a few in prompt examples. Meanwhile even today the latest models still have notes that they may occasionally ignore some parts of JSON schemas sent down.
MCP doesn't force models to output JSON, quite the opposite. Tool call results in MCP are text, images, audio — the things models naturally output. The whole point of MCP is to make APIs digestable to LLMs
I'm not sure about that. Newer models are able to output structured outputs perfectly and infact, if you combine it with changesets, you can have insanely useful applications since changesets also provide type-checking.
If you search for MCP integrations, you will find tons of MCP "servers", which basically are entire servers for just one vendor's API (sometimes just for one of their products, eg. YouTube). This is the go to default right now, instead of just one server with 100 modules.
The MCP protocol itself is just to make it easier to communicate with the LLM clients that users can use and install. But, if you're doing backend code, there is no need to use MCP for it.
It is only undesirable if you are to expose your APIs for others to use via a consumer facing client. As a backend developer, I'm writing a backend for my app to consume, not for consumers (like MCP is designed for). So, this is better for me since I'm a 100% Elixir shop anyway.
I am just glad that we now have a simple path to authorized MCP servers. Massive shout-out to the MCP community and folks at Anthropic for corralling all the changes here.
The analogy that was used a lot is that it's essentially USB-C for your data being connected to LLMs. You don't need to fight 4,532,529 standards - there is one (yes, I am familiar with the XKCD comic). As long as your client is MCP-compatible, it can work with _any_ MCP server.
Fascinated to see that the core spec is written in TypeScript and not, say, an OpenAPI spec or something. I suppose it makes sense, but it’s still surprising to see.
Because type script is a language not a language agnostic specification for languages. As someone who uses typescript a lot that probably is irrelevant. But if I’m using Rust, say, then the typescript means I need to reimplement the spec from scratch. With OpenApi, say, I can code generate canonically correct stubs and implement from there.
It is mostly pointless complexity, but I’m going to miss batching. It was kind of neat to be able to say ‘do all these things, then respond,’ even if the client can batch the responses itself if it wants to.
I agree. JSON-RPC batching has always been my "gee, that's neat" feature and seeing it removed from the spec is sad. But, as you said, it's mostly just adding complexity.
Elicitation is a big win. One of my favorite MCP servers is an SSH server I built, it allows me to basically automate 90% of the server tasks I need done. I handled authentication via a config file, but it's kind of a pain to manage if I want to access a new server.
Ansible, puppet, chef, salt, cfengine... There are tons of tools that are more precise and succinct for describing sensitive tasks, such as managing a single or a fleet of remote servers. Using MCP/LLMs for this is... :O
I don't know if you understand the role the LLM is playing here. The mechanism used to execute the command is not the relevant thing. The LLM autonomously executing commands has intelligence, it's not just a shell script. If I ask it to do a task, and it runs into an issue... LLMs like Claude can recognize the problem, and find a way to resolve it. Script failed because of a missing dependency, it'll install it. Need a config change, it'll do it. The SSH mcp is just the interface for the LLM to do the work.
You can give an LLM a github repo link, a fresh VPC, and say "deploy my app using nginx" and any other details you need... and it'll get it done.
The introduction of WWW-Authenticate challenge is so welcome. Now it's much clearer that the MCP server can punt the client to resource provider's OAuth flow and sit back waiting for an `Authorization: Bearer ...`.
I spent the last few days playing with MCP building some wrappers for some datasets. My thoughts:
1) Probably the coolest thing to happen with LLMs. While you could do all this with tool calling and APIs, being able to send to less technical friends a MCP URL for claude and seeing them get going with it in a few clicks is amazing.
2) I'm using the csharp SDK, which only has auth in a branch - so very bleeding edge. Had a lot of problems with implementing that - 95% of my time was spent on auth (which is required for claude MCP integrations if you are not building locally). I'm sure this will get easier with docs but it's pretty involved.
3) Related to that, Claude doesn't AFIAK expose much in terms of developer logs for what they are sending via their web (not desktop) app and what is going wrong. Would be super helpful to have a developer mode where it showed you request/response of errors. I had real problems with refresh on auth and it turned out I was logging the wrong endpoint on my side. Operator error for sure but would have fixed that in a couple of mins had they had better MCP logging somewhere in the webui. It all worked fine with stdio in desktop and MCP inspector.
4) My main question/issue is handling longer running tasks. The dataset I'm exposing is effectively a load of PDF documents as I can't get claude to handle the PDF files itself (I am all ears if there is a way!). What I'm currently doing is sending them through gemini to get the text, then sending that to the user via MCP. This works fine for short/simple documents, but for longer length documents (which can take some time to process) I return a message saying it is processing and to retry later.
While I'm aware there is a progress API, it still requires keeping the connection open to the server (which times out after a while with Cloudflare at least) - could be wrong here though?. It would be much better to be able to tell the LLM to check back in x seconds when you predict it will be done and the LLM can do other stuff in the background (which it will do), but then sorta 'pause execution' until the timer is hit.
Right now (AFIAK!) you can either keep it waiting (which means it can't do anything else in the meantime) with an open connection w/ progress, or you can return a job ID, but then it will just return a half finished answer which is often misleading as it doesn't have all the context yet. Don't know if this makes any sense, but I can imagine this being a real pain for tasks that take 10mins+.
Long-running tasks are an open topic of discussion, and I think MCP intends to address it in the future.
There are a few proposals floating around, but one issue is that you don't always know whether a task will be long-running, so having separate APIs for long-running tasks vs "regular" tool calls doesn't fully address the problem.
It'd be nice to more closely integrate MCP into something like Airflow, with hints as to expected completion time.
Real world LLM is going to be built on non-negligible (and increasingly complicated) deterministic tools, so might as well integrate with the 'for all possible workflows' use case.
Readit News
No reason why you couldn't just use tool calls with OpenAPI specs though. Either way, making all your microservices talk to each other over MCP sounds wild.
If you're using the API and not in a hurry, there's no need for it.
Not familiar with elixir, but is there anything prohibiting you from just making a monolith MCP combining multiple disparate API's/backends/microservices as you were doing previously?
Further, you won't get the various client-application integrations with MCP merely using tool-calling; which to me is the "killer app" of MCP (as a sibling comment touches on).
(I do still have mixed feelings about MCP, but in this case MCP sorta wins for me)
This is what I ended up doing.
The reason I thought I must do it the "MCP way" was because of the tons of YouTube videos about MCP which just kept saying how much of an awesome protocol it is, an everyone should be using it, etc. Once I realized it's actually more consumer facing than the backend, it made much more sense as why it became so popular.
Deleted Comment
With something like https://nango.dev you can get a single server that covers 400+ APIs.
Also handles auth, observability and offers other interfaces for direct tool calling.
(Full disclosure, I’m the founder)
In the end, MCP is just like Rest APIs, there isn't need for a paid service for me to connect to 400 Rest APIs now, why do I need a service to connect to 400 MCPs?
All I need for my users is to be able to connect to one or two really useful MCPs, which I can do myself. I don't need to pay for some multi REST API server or multi MCP server.
unlike web dev where you client cab just write more components / API callers, AI has context limits. if you try to plug (i get it, exaggerated) 500 MCP servers each with 2-10 tools you will waste a ton of context in your sys prompt. you can use a tool proxy (one tool which routes to others) but that will add latency from the processing.
Forcing LLMs to output JSON is just silly. A lot of time and effort is being spent forcing models to output a format that is picky and that LLMs quite frankly just don't seem to like very much. A text based DSL with more restrictions on it would've been a far better choice.
Years ago I was able to trivially teach GPT 3.5 to reliably output an English like DSL with just a few in prompt examples. Meanwhile even today the latest models still have notes that they may occasionally ignore some parts of JSON schemas sent down.
Square peg, round hole, please stop hammering.
For example, in Elixir, we have this library: https://hexdocs.pm/instructor/
It's massively useful for any structured output related work.
This is basically what MCP is. Before MCP, everyone was rolling their own function calling interfaces to every API. Now it’s (slowly) standardising.
That implies a language lock-in, which is undesirable.
- a Typescript library to set up ChatGPT-MCP compatible auth
- Source code for an Express & NextJS project implementing the library
- a URL for the demo of the deployed NextJS app that logs ChatGPT tool calls
Should be helpful for folks trying to set up and distribute custom connectors.
https://github.com/mcpauth/mcpauth
Not sure why people treat MCP like it's much more than smashing tool descriptions together and concatenating to the prompt, but here we are.
It is nice to have a standard definition of tools that models can be trained/fine tuned for, though.
You can give an LLM a github repo link, a fresh VPC, and say "deploy my app using nginx" and any other details you need... and it'll get it done.
1) Probably the coolest thing to happen with LLMs. While you could do all this with tool calling and APIs, being able to send to less technical friends a MCP URL for claude and seeing them get going with it in a few clicks is amazing.
2) I'm using the csharp SDK, which only has auth in a branch - so very bleeding edge. Had a lot of problems with implementing that - 95% of my time was spent on auth (which is required for claude MCP integrations if you are not building locally). I'm sure this will get easier with docs but it's pretty involved.
3) Related to that, Claude doesn't AFIAK expose much in terms of developer logs for what they are sending via their web (not desktop) app and what is going wrong. Would be super helpful to have a developer mode where it showed you request/response of errors. I had real problems with refresh on auth and it turned out I was logging the wrong endpoint on my side. Operator error for sure but would have fixed that in a couple of mins had they had better MCP logging somewhere in the webui. It all worked fine with stdio in desktop and MCP inspector.
4) My main question/issue is handling longer running tasks. The dataset I'm exposing is effectively a load of PDF documents as I can't get claude to handle the PDF files itself (I am all ears if there is a way!). What I'm currently doing is sending them through gemini to get the text, then sending that to the user via MCP. This works fine for short/simple documents, but for longer length documents (which can take some time to process) I return a message saying it is processing and to retry later.
While I'm aware there is a progress API, it still requires keeping the connection open to the server (which times out after a while with Cloudflare at least) - could be wrong here though?. It would be much better to be able to tell the LLM to check back in x seconds when you predict it will be done and the LLM can do other stuff in the background (which it will do), but then sorta 'pause execution' until the timer is hit.
Right now (AFIAK!) you can either keep it waiting (which means it can't do anything else in the meantime) with an open connection w/ progress, or you can return a job ID, but then it will just return a half finished answer which is often misleading as it doesn't have all the context yet. Don't know if this makes any sense, but I can imagine this being a real pain for tasks that take 10mins+.
There are a few proposals floating around, but one issue is that you don't always know whether a task will be long-running, so having separate APIs for long-running tasks vs "regular" tool calls doesn't fully address the problem.
I've written a proposal to solve the problem in a more holistic way: https://github.com/modelcontextprotocol/modelcontextprotocol...
Real world LLM is going to be built on non-negligible (and increasingly complicated) deterministic tools, so might as well integrate with the 'for all possible workflows' use case.