Each person gets a dedicated GPU, so we were worried about costs before. But, let' s just go for it.
Really looking forward to trying this out!
Deleted Comment
Each person gets a dedicated GPU, so we were worried about costs before. But, let' s just go for it.
Really looking forward to trying this out!
On the other hand, the fact that this is even possible is more wild. Instead of replacing JS with a proper statically-typed language, we're spending all this effort turning a preprocessor's type system into a turing-complete metalanguage. Pretty soon we'll be able to compile TypeScript entirely using types.
Examples include basically any PaaS, IaaS, or any company that provides a mission-critical service to another company (B2B SaaS).
If you run a basic B2C CRUD app, maybe it’s not a big deal if you service goes down for 5 minutes. Unfortunately there are quite a few categories of companies where downtime simply isn’t tolerated by customers. (I operate a company with a “zero downtime” expectation from customers - it’s no joke, and I would never use any infrastructure abstraction layer other than AWS, GCP or Azure - preferably AWS us-east-1 because, well, if you know the joke…)
Meaning empirically, downtime seems to be tolerated by their customers up to some point?
> I feel that transforming such a wonderfully written story into a podcast with robot voices might result in a shallow and awkward rendition that doesn’t capture the essence of the original work.
I can definitely agree with that in the general sense! But I have a long commute, and I thought the content of it would still come through well enough, and I've appreciated it for a few other subjects. It's not as good as having professional voice actors read it out, but it still has some value to folks!
Better yet, if someone has already _done_ that Notebook Lm podcast and has a link to it, please share!
Definitely think this sort of idea could become the "serverless" equivalent for ml-using apps. I'm curious what you think re: versioning, consumption from various client languages, observability/monitoring/queueing, etc.? Feels like it could grow into a meaningful platform.
The combined use of faithful-chain-of-thought + mechanistic interpretation of LLM output to 1.) diagnose 2.) understand the source of, and 3.) steer the behavior is fascinating.
I'm very glad these folks found such a surprising outcome early on, and it lead to a useful real-world LLM debugging exercise!