The names of GPT models are just terrible. o3 is better than 4o, maybe?
The names of GPT models are just terrible. o3 is better than 4o, maybe?
- Encourage folks to use read-only by default in our docs [1]
- Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data [2]
- Write E2E tests to confirm that even less capable LLMs don't fall for the attack [2]
We noticed that this significantly lowered the chances of LLMs falling for attacks - even less capable models like Haiku 3.5. The attacks mentioned in the posts stopped working after this. Despite this, it's important to call out that these are mitigations. Like Simon mentions in his previous posts, prompt injection is generally an unsolved problem, even with added guardrails, and any database or information source with private data is at risk.
Here are some more things we're working on to help:
- Fine-grain permissions at the token level. We want to give folks the ability to choose exactly which Supabase services the LLM will have access to, and at what level (read vs. write)
- More documentation. We're adding disclaimers to help bring awareness to these types of attacks before folks connect LLMs to their database
- More guardrails (e.g. model to detect prompt injection attempts). Despite guardrails not being a perfect solution, lowering the risk is still important
Sadly General Analysis did not follow our responsible disclosure processes [3] or respond to our messages to help work together on this.
[1] https://github.com/supabase-community/supabase-mcp/pull/94
[2] https://github.com/supabase-community/supabase-mcp/pull/96
They did put your disclosure process and messages into an llm prompt, but llm chose to ignore it.
Though what would also help if you had an explanation for why we tend to hear these stories mostly from the US and not from other countries.
It's kind of frustrating how many people there are like "hire a lawyer" and OP is like "I did hire a lawyer".
It is weird that there'd be a minimum use requirement. What about for not yet publicly released projects? Or really, why would trademarking be essentially reserved for bigger companies?
If the project is unreleased or small, why would it be protected in scope of the entire european union?