Readit News> for POST requests, we require that developers build a user confirmation flow to avoid destruction actions
However, at least from what I can see, the docs don't provide much more detail about how to actually implement confirmation. I haven't played around with the plugins API myself, but I originally assumed it was a non-AI-driven technical constraint, maybe a confirmation modal that ChatGPT always shows to the user before any POST. From a forum post I saw [2], though, it looks like ChatGPT doesn't have any system like that, and you're just supposed to write your manifest and OpenAPI spec in a way that tells ChatGPT to confirm with the user. From the forum post, it sounds like this is pretty fragile, and of course is susceptible to prompt injection as well.
[1] https://platform.openai.com/docs/plugins/introduction
[2] https://community.openai.com/t/implementing-user-confirmatio...
Sounds like an easy fix, if it’s possible to detect direct prompt injection attacks then the same techniques can be applied to the data staged for retrieval.
One solution to some indirect prompt injection attacks is proposed in this article, where you "sandbox" untrusted content into a second LLM that isn't given the ability to decide which actions to take: https://simonwillison.net/2023/Apr/25/dual-llm-pattern/
addTodoItem(taintedLLMtranslation, untaintedOriginalEmailAddress)
All of the data that came from remote commands, such as the body of a newly created Todo item, should still be considered tainted and and treated in a similar manner.
These are the exact same security issues for any case of remote API calls with arbitrary execution.
Why not just have a limited set of permissions for what commands can originate from a given email address?
The original email address can be included along with whatever commands were translated by the LLM. It seems easy enough to limit that to only a few simple commands like “create todo item”.
Think of it this way, what commands would you be fine to be run on your computer if they came from a given email address?
Here's a basic How-to:
* Run node with `node --inspect <file>`
* Open Chrome and go to `chrome://inspect`
* After it finds your target, click the inspect button to launch a debugger.
To use with Node.js profiling, do the `node --inspect` and `chrome://inspect` steps, then save the profile as a .cpuprofile file and drag that file into speedscope.
Another thing I've found useful is programmatically starting/stopping the profiler using `console.profile()` and `console.profileEnd()`.
Ithought it was 2022. I have a 12 core machine and my next machine will probably have 22 cores.
But I'm amazed, transpiling 636975 lines in <1 second is nice.
[Edit] What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker? Or does it check type errors? Why would I used it for Typescript when the reason to use TS is to add types to JS to prevent errors?
Is this more like Rust check for continous work and then use tsc from time to time to check for errors?
> What I do not understand is "Sucrase does not check your code for errors." So it's not a type checker?
That's correct, Sucrase, swc, esbuild, and Babel are all just transpilers that transform TypeScript syntax into plain JavaScript (plus other transformations). The usual way you set things up is to use the transpiler to run and build your TS code, and you separately run type checking using the official TypeScript package.
> Like all JavaScript code run in V8, Sucrase runs more slowly at first, then gets faster as the just-in-time compiler applies more optimizations. From a rough measurement, Sucrase is about 2x faster after running for 3 seconds than after running for 1 second. swc (written in Rust) and esbuild (written in Go) don't have this effect because they're pre-compiled, so comparing them with Sucrase gets significantly different results depending on how large of a codebase is being tested and whether each compiler is allowed a "warm up" period before the benchmark is run.
(worse it disables esbuild and swc's multi-threading... https://github.com/alangpierce/sucrase/blob/main/benchmark/b... https://github.com/alangpierce/sucrase/blob/main/benchmark/b...)
fake it till ya make it.
it's like saying "if I disable everything and wait for 5 minutes it's faster"
To be clear, the benchmark in the README does not allow JIT warm-up. The Sucrase numbers would be better if it did. From testing just now (add `warmUp: true` to `benchmarkJest`), Sucrase is a little over 3x faster than swc if you allow warm-up, but it seemed unfair to disregard warm-up for the comparison in the README.
It's certainly fair to debate whether 360k lines of code is a realistic codebase size for the benchmark; the higher-scale the test case, the better Sucrase looks.
> worse it disables esbuild and swc's multi-threading
At some point I'm hoping to update the README benchmark to run all tools in parallel, which should be more convincing despite the added variability: https://github.com/alangpierce/sucrase/issues/730 . In an ideal environment, the results are pretty much the same as a per-core benchmark, but I do expect that Node's parallelism overhead and the JIT warm-up cost across many cores would make Sucrase less competitive than the current numbers.
Maybe I'm not fully understanding the approach, but it seems like if you started relying on third-party MCP servers without the AI layer in the middle, you'd quickly run into backcompat issues. Since MCP servers assume they're being called by an AI, they have the right to make breaking changes to the tools, input schemas, and output formats without notice.