za_mike157 (u/za_mike157)

za_mike157 commented on How to Migrate from OpenAI to Cerebrium for Cost-Predictable AI Inference ritza.co/articles/migrate... · Posted by u/sixhobbits

benterix · a month ago

To people from Cerebrium: why should I use your services when Runpod is cheaper? I mean, why did you decide to set your prices higher than an established company with significant user base?

za_mike157 · a month ago

Hey! Founder of Cerebrium here.

- Runpod is one of the cheapest but it comes at the price of reliability (critical for businesses) - We have more performant cold start performance with something special launching soon here - Iterating on your application using CPUs/GPUs in the cloud takes just 2–10 seconds, compared to several minutes with Runpod due to Docker push/pull. - Allow you to deploy in multiple regions globally for lower latency and data residency compliance - We provide a lot of software abstractions (fire and forget jobs, websockets, batching, etc) where as Runpod just deploys your docker image. - SOC 2 and GDPR compliant

With that all being said - we are working on optimisations to bring down pricing

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

doctorpangloss · a year ago

Why use modal instead of SkyPilot?

za_mike157 · a year ago

I haven't used SkyPilot so I am unfamiliar with the experience and performance.

However, some of the situations you would like to use Cerebrium over Skypilot are: - You don't want to manage you own hardware - Reduced costs: With serverless Runtime and low cold starts (unclear if SkyPiolet offers this and what the peformance is like if they do) - Rapid iteration: Unclear of the deployment process on SkyPilot and how long projects take to go live - Observability: Looks like you would just have k8s metrics at your disposal

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

chaosinblood · a year ago

which ui framework do you use? it's very nice

za_mike157 · a year ago

I think we used this UI kit: https://minimals.cc/

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

abraxas · a year ago

Paperspace lets you bring your own containers and will scale them automatically. I don't know whether that would qualify as "serverless".

za_mike157 · a year ago

I guess then the next question would be how quickly can they start executing your container from cold start when a workload comes in? Typically we see companies on around 30-60s

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

mdaniel · a year ago

Being a toml-n00b, why is this quoted? https://github.com/CerebriumAI/examples/blob/85815f8e09e9e77...

Related to that, it seems the syntax isn't documented https://docs.cerebrium.ai/cerebrium/environments/config-file...

za_mike157 · a year ago

Do you mean why the individual file names aren't quoted?

You can see an example config file at the bottom of that link you attached - agreed we should probably make it more obvious

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

risyachka · a year ago

Yeah Runpods cold start is definitely not 250ms, not even close. Maybe for some models idk but a huggingface model 8B params takes like 30 seconds to cold start in their serverless "flash" configuration.

za_mike157 · a year ago

Thanks for confirming! Our cold start, excluding model load is 2-4 seconds typically for HF models.

The only time it gets much longer when companies have done a lot with very specific CUDA implementations

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

tmshapland · a year ago

We use Cerebrium for our Mixpanel for Voice AI product (https://voice.canonical.chat). Great product. So much easier to set up and more robust than other model hosting providers we've tried (especially AWS!). Really nice people on the the team, too.

za_mike157 · a year ago

Thanks Tom! Excited to to support you and the team as you grow

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

spmurrayzzz · a year ago

> Yes RunPod does have cheaper pricing than us however they don't allow you to specify your exact resources but rather charge you the full resource (see example of A100 above) so depending on your resource requirements our pricing could be competitive since we charge you only for the resources you use.

I may be misunderstanding your explanation a bit here, but Runpod's serverless "flex" tier looks like the same model (it only charges you for non-idle resources). And at that tier they are still 2x cheaper for A100, at your price point with them you could rent an H100.

za_mike157 · a year ago

Ah I see they recently cut their pricing by 40% so you are correct - sorry about that. It seems we are more expensive compared to their new pricing

za_mike157 commented on Launch HN: Cerebrium (YC W22) – Serverless Infrastructure Platform for ML/AI · Posted by u/za_mike157

benjamaan · a year ago

Congrats and thank you! We’ve been a happy customer since early on. Although we don’t have much usage, our products are mostly R&D, having Cerebrium made it super easy to launch cost effectively on tight budgets and run our own models within our apps.

The support is next level - team is ready to dive into any problem, response is super fast, and has helped us solve a bunch of dev problems that a normal platform probably won’t.

Really excited to see this one grow!!

za_mike157 · a year ago

Thank you - appreciate the kind words! Happy to continue supporting you and the team.

u/za_mike157

KarmaCake day58February 6, 2021View Original