tomhoule (u/tomhoule)

tomhoule commented on Nexus: An Open-Source AI Router for Governance, Control and Observability nexusrouter.com/blog/intr... · Posted by u/mitchwainer

SparkyMcUnicorn · 5 months ago

What's the difference between "MCP Server Aggregation" and the litellm_proxy endpoint described here?

https://docs.litellm.ai/docs/mcp

tomhoule · 5 months ago

The main difference is that while you can get Nexus to list all tools, by default the LLM accesses tools by semantic search — Nexus returns only the relevant tools for the what the LLM is trying to accomplish. Also, Nexus speaks MCP to the LLM, it doesn't translate like litellm_proxy seems to do (I wasn't familiar with it previously).

tomhoule commented on Nexus: An Open-Source AI Router for Governance, Control and Observability nexusrouter.com/blog/intr... · Posted by u/mitchwainer

vid · 5 months ago

There is also https://github.com/maximhq/bifrost which apparently overcomes some performance issues of litellm and is easy to get going.

tomhoule · 5 months ago

Yeah they definitely belong in the same space. Nexus is an LLM Gateway, but early on, the focus has been on MCP: aggregation, authentication, and a smart approach to tool selection. There is that paper, and a lot of anecdotal evidence, pointing to LLMs not coping well with a selection of tools that is too large: https://arxiv.org/html/2411.09613v1

So Nexus takes a tool search based approach to solving that, among other cool things.

Disclaimer: I don't work on Nexus directly, but I do work at Grafbase.

tomhoule commented on Are AI code assistants helpful? If yes, which one is the best? · Posted by u/ips1512

tomhoule · 2 years ago

In my experience with Copilot, the only good thing I can say about it is that it sometimes saves time spent typing.

tomhoule commented on Poll HN: What operating system do you primarily develop on? · Posted by u/dennis-tra

tomhoule · 4 years ago

NixOS for everything. It's the Linux distro I've had the best experience with in ~15 years of daily usage.

tomhoule commented on Prisma – ORM for Node.js and TypeScript prisma.io/blog/prisma-the... · Posted by u/janpio

atleta · 5 years ago

Well, of course I don't know about the internals, but having used Django migrations for a decade now (it used to be a standalone solution called "South" back then), I haven't really run into any inaccuracies and can't really imagine how those could happen. As far as I can see, the main difference is that they are storing and intermediate format (that they can map to SQL unambigously) while you immediately generate the SQL.

Django doesn't try (too hard) to validate your model against the actual DB schema. Because why would it? You either ran all the migrations and then it matches or you didn't and then you have to. (Unless you write your own migrations and screw them up. But that's rare and you can catch it with testing.) While your focus then seems to be to check if the schema (whatever is there in the db) matches the model definition. Based on my experience (as a user) this latter is not really something that I need help with.

Data is actually an issue in development and hard resets + (re)seeding is pretty inconvenient compared to what django provides. E.g. in my current project we're using a db snapshot that we've pulled from production about two years ago (after thorough anonymization, of course). We initialize new dev environments and then it gets migrated. It probably takes about half a minute to run as opposed to about 2 seconds of back migrating 2-3 steps.

tomhoule · 5 years ago

It makes a lot of sense. I have a fair amount of Rails experience with ActiveRecord, and it was also my impression that the database schema drifting in development is rarely a problem, but I now think it's a bit of a fuzzy feeling and discrepancies definitely sneak in. The main sources of drift in development would be 1. switching branches, and more generally version control with collaborators, 2. iterating on/editing of migrations, 3. manual fiddling with the database

One assumption with Prisma Migrate is that since we are an abstraction of the database, and support many of them, we'll never cover 100% of the features (e.g. check constraints, triggers, stored procedures, etc.), so we have to take the database as the source of truth and let users define what we don't represent in the Prisma Schema. On SQL databases, we let you write raw SQL migrations for example, so you have full control if you need it.

tomhoule commented on Prisma – ORM for Node.js and TypeScript prisma.io/blog/prisma-the... · Posted by u/janpio

abaldwin99 · 5 years ago

So future features notwithstanding, is the typical Prisma workflow that if a migration failed during a production deploy the developer would have to manually work out how to fix it while the application is down?

tomhoule · 5 years ago

As of now there is no strong opinionation in the tool — you could absolutely maintain a set of down migrations to recover from bad deployments next to your regular migrations, and apply the relevant one (manually, admittedly) in case of problem.