Calling them "the" company behind Stable Diffusion is pushing it. Just Stable Diffusion the model, but not the idea, maybe.
As far as I know they pretty much just paid for servers to train a big shiny model on that was based on research they had no hand in. Throwing money at researchers after they came up with something good, just to let them build a big shiny version of it, does not retroactively make their accomplishments yours.
Basically they hold no rights to anything relevant, no patents, no secret sauce, nothing. Them going under after exhausting their money will hardly have any effect.
Stable diffusion is just the "brand" of the diffusion generative model, just like Midjourney.
If you look at the stable-diffusion repo, it says that SD is based off a colab with StabilityAI and Runway.
Most of the value from these models is the training + dataset. The architecture is open source, and we've had flavours of it for several years. SD has some improvements on how it handles diffusions, but most architecture out there use about the same, but with wildly different results.
Someone from Runway (Patrick Esser) is actually listed as an author on the paper.[1]
The datasets used are provided by LAION.
Here is a honest summary of Stability.Ai's involvement[2]:
> In their project, the LMU scientists had the support of the start-up Stability.Ai, on whose servers the AI model was trained. “This additional computing power and the extra training examples turned our AI model into one of the most powerful image synthesis algorithms,” says the computer scientist with a smile.
We’ve had, what, over two years of “basically free” image generation? (Dall-E 1 was Jan 21.) All the major players (OpenAI, Microsoft, Stability, Midjourney) offer free image generation and you only pay for large amounts.
There is a ton of momentum behind the general public’s belief and perception that image generation is free for small uses and cheap for large uses. Most of the other players can afford to keep those losing prices for a long time, too. I think it’s going to be an uphill battle to charge for image generation at amounts that turn a profit. I wouldn’t rule out a more creative way of monetizing it, but the obvious routes look unlikely.
Midjourney has stopped offering free generation. I made all of three or four images and now it is saying they don't have capacity for free trials anymore.
Not having enough capacity is not the same thing as running out of money, though. If the number of users is increasing rapidly (and there's every reason to believe that it is) it could just be that they're not able to spin up new GPUs fast enough to keep up with the increasing demand.
I've noticed that they've recently rolled out some new features (e.g., "/describe") that were at first limited to the higher tiers, but after a short time of evaluating the increased GPU load, were enabled for the lower tiers as well.
To me, that seems to indicate "can't install GPUs fast enough" rather than "running out of money", but of course it's just an indication.
I believe their stated reason for getting rid of the free trial accounts (temporarily, they said) was that people were using bots to register hundreds of sockpuppet accounts to get the freebies. Perhaps they'll come back when and if they can deploy some effective anti-bot measures.
Just a question but is it ever profitable to base a company around an open source product the way SD is? Why would anyone pay money to use the company's model when some guys on Reddit is distributing a similar product, albeit with lower performance, for free, that can be run locally?
What would be the incentive for a person, not a company, to pay Stability AI company instead of downloading and doing a bit of setup to have their own uncensored model?
None of these companies are really directly concerned about profitability. Right now the leaders in ML models are going to be defined by those who have the most capital available to train newer and better models. And the quickest way to get that capital is largely getting investors on board. From that perspective it’s quite rational to release something publicly in a way that is likely to raise the profile of your company. Especially in a world where your main competitors will be training their own models anyways.
Investors make for spectacular customers for a few years. You can sell them dreams! Many companies struggle when they need to pivot away from this model
Research for the model was done by the CompVis at LMU Munich.
Curating the data set was supported by Laion, a German non profit and Eluther, also a non profit.
I do not know the details, but based on the fact that some of the orgs involved in the project literally exists to democratise ai, I believe that many of the stake holders in that project were adamant about open sourcing it.
I think there's a lot of "open core" stuff like Sentry, Tailscale (I think?), or Gitlab. Where "enough" of the house is given away in theory for you to do it yourself (but why would you do that?)
Metabase is another company I knew where they do this, but I don't know how successful they are (their hosting costs start at "way too high considering how easy the thing is to spin up", which was great for me at $PREV_JOB I suppose but)
Perhaps the business case is for "we want the Open Source product, but we're in a compliance-oriented business and need to have someone selling us a support and maintenance contract behind it."
I'd suspect the moment you touch anything with external auditing, it looks better to say "we've got a fully paid up support contract from the vendor" than "we're running v1.23.456-ubuntu-patch-357-with-chives-and-salsa that we downloaded last week."
We should outlaw government spending on closed source, non public domain software. That would create a huge market for open source companies to thrive. Would be a win win for everyone (except the current closed vendors who provide government with crap software and get sucker taxpayers to foot the bill).
The issue is not "Microsoft Office vs Libre office" but rather "how much does it cost to necessary up to the support necessary for the government office in order to support Libre office?"
Things like "ok, we can't use one drive - need a different tool," "can't use Sharepoint, need to use a different tool," and so on.
The market is there. RHEL and similar are well established.
In order to make this a "we should do this" either government IT funding needs to be significantly increased (that is difficult in the current political climate in most places) or the support offerings and staff needed for the average user (using Windows, Sharepoint, Word, Excel, Teams, and Project) needs to be competitive with the pricing that Microsoft offers.
That should be "simple" - make a company that offers the same level of support as Microsoft does for a packaged suite of software that includes easy installations, appropriately locked down desktops, call center, and so on.
And if that can't be found at the same price that Microsoft offers - then we return to the "increase government funding."
Saying "we should outlaw government spending on closed source" misses a lot of the tools out there that are needed to keep things running. Is there a FOSS (with support contracts for the stack) alternative to Cerner or Epic? SAP? ArcGIS? And that's not even getting deep at all into the niche SaaS tools that some pieces use for specific problems.
The market is there and state and local governments would likely jump at the opportunity to switch if there's a company that can offer the same functional stack with the same support for the same price or cheaper built on top of FOSS. Otherwise... persuade those state and local governments to staff up to the necessary levels to be able to hire people able to customize and support the FOSS to fit their needs and be prepared to financially support that decision.
I agree, but realistically that’s tough. Sometimes the government just needs a tool like everyone else. And something that works and is available today off the shelf is the best you’ll get.
Sure it’d be great for government acquisitions to subsidize open source, but at what cost?
I only see something like SD to be attractive for personal uses, not corporate. That is why I mentioned a person, not a company, as a customer.
If your product is tested and guaranteed to certain standards, like MySQL and RedHat, then that is something a company may pay for. But a user doesn't really have that high standard so they can be satisfied with just the off brand, derived stuff floating around.
without a stable source of revenue, cutting costs mean nothing. What are they selling, and why isn't there buyers?
A business cannot survive without revenue, and revenue only comes from people who want to buy something from you that they cannot otherwise get else where.
I applaud StabilityAI for releasing SD for free. They could've done what openAI did, and monetized it (which other diffusion model services are doing). By releasing it for free, stabilityAI contributed to the common good. Unfortunately, if there's nothing else that can be sold as a product, they cannot be sustainable.
An alternative, which i'm not too big a fan of, is collectivization of AI models, and make it a global commons for which taxpayers will fund.
They could have at least charged money for it. "I'll give you this 4 GiB model file for $50" might have been a better business model than "lemme give this to you for free!".
It seems like mandating just a tiny bit of usage data back to the model would give SD a massive lead on training data, but I'm not an expert. Maybe that's happening already.
Like, example. I use SD in Blender sometimes as part of the compositor. I have maybe a 10% acceptance rate for SD output: sometimes the water isn't right, or the clouds look goofy, or something keeps getting rendered as an anime pillow for some godforsaken reason. If SD captured my prompt history and some of the final model tweaks between runs, they could ostensibly get really solid HITL test data. Then they could be the curator of that "super model" which they could upsell, maybe along with very high rez stuff, or a higher priority on jobs. Again, not an expert, so who knows. And also, having the model local, that gives you back some of the same benefits, but without the scale.
Not too surprised about funding issues from the casual answer.
I’m not saying it was bad to self fund a project, but having to choosing between your life and fun (and potentially very profitable) projects is not easy.
Note that stability have been funding freelance researcher by providing compute resource such as RWKV[1], Open Assistant, some works by LAION[2] and lucidrains[3]
In a gold rush, sell shovels. And Nvidia's stock price bump reflects that.
This time around though, the means of production are available to anybody with a credit card, and AWS/GCP/Azure throw credits at startups that apply, just to have them locked into their cloud. Break free of the chains of work, with generative pretrained transformers!
Readit News
As far as I know they pretty much just paid for servers to train a big shiny model on that was based on research they had no hand in. Throwing money at researchers after they came up with something good, just to let them build a big shiny version of it, does not retroactively make their accomplishments yours.
Basically they hold no rights to anything relevant, no patents, no secret sauce, nothing. Them going under after exhausting their money will hardly have any effect.
If you look at the stable-diffusion repo, it says that SD is based off a colab with StabilityAI and Runway.
Most of the value from these models is the training + dataset. The architecture is open source, and we've had flavours of it for several years. SD has some improvements on how it handles diffusions, but most architecture out there use about the same, but with wildly different results.
The datasets used are provided by LAION.
Here is a honest summary of Stability.Ai's involvement[2]:
> In their project, the LMU scientists had the support of the start-up Stability.Ai, on whose servers the AI model was trained. “This additional computing power and the extra training examples turned our AI model into one of the most powerful image synthesis algorithms,” says the computer scientist with a smile.
[1]: https://ommer-lab.com/research/latent-diffusion-models/
[2]: https://www.lmu.de/en/newsroom/news-overview/news/revolution...
Which is their business model.
Provide compute to people who can’t afford compute so the only people doing AI research aren’t doing so behind closed doors.
Now, it seems, giving away your product isn’t all that profitable and they need to “pivot” to find a way to keep the business running.
Judging from the interview posted elsewhere in the discussion they make no claim to be inventing anything but just wanted to democratize AI research.
There is a ton of momentum behind the general public’s belief and perception that image generation is free for small uses and cheap for large uses. Most of the other players can afford to keep those losing prices for a long time, too. I think it’s going to be an uphill battle to charge for image generation at amounts that turn a profit. I wouldn’t rule out a more creative way of monetizing it, but the obvious routes look unlikely.
I've noticed that they've recently rolled out some new features (e.g., "/describe") that were at first limited to the higher tiers, but after a short time of evaluating the increased GPU load, were enabled for the lower tiers as well.
To me, that seems to indicate "can't install GPUs fast enough" rather than "running out of money", but of course it's just an indication.
I believe their stated reason for getting rid of the free trial accounts (temporarily, they said) was that people were using bots to register hundreds of sockpuppet accounts to get the freebies. Perhaps they'll come back when and if they can deploy some effective anti-bot measures.
What would be the incentive for a person, not a company, to pay Stability AI company instead of downloading and doing a bit of setup to have their own uncensored model?
I do not know the details, but based on the fact that some of the orgs involved in the project literally exists to democratise ai, I believe that many of the stake holders in that project were adamant about open sourcing it.
In this case they were actually the first to the punch(along with openAI)!
Metabase is another company I knew where they do this, but I don't know how successful they are (their hosting costs start at "way too high considering how easy the thing is to spin up", which was great for me at $PREV_JOB I suppose but)
I'd suspect the moment you touch anything with external auditing, it looks better to say "we've got a fully paid up support contract from the vendor" than "we're running v1.23.456-ubuntu-patch-357-with-chives-and-salsa that we downloaded last week."
None of which are profitable
Things like "ok, we can't use one drive - need a different tool," "can't use Sharepoint, need to use a different tool," and so on.
The market is there. RHEL and similar are well established.
In order to make this a "we should do this" either government IT funding needs to be significantly increased (that is difficult in the current political climate in most places) or the support offerings and staff needed for the average user (using Windows, Sharepoint, Word, Excel, Teams, and Project) needs to be competitive with the pricing that Microsoft offers.
That should be "simple" - make a company that offers the same level of support as Microsoft does for a packaged suite of software that includes easy installations, appropriately locked down desktops, call center, and so on.
And if that can't be found at the same price that Microsoft offers - then we return to the "increase government funding."
Saying "we should outlaw government spending on closed source" misses a lot of the tools out there that are needed to keep things running. Is there a FOSS (with support contracts for the stack) alternative to Cerner or Epic? SAP? ArcGIS? And that's not even getting deep at all into the niche SaaS tools that some pieces use for specific problems.
The market is there and state and local governments would likely jump at the opportunity to switch if there's a company that can offer the same functional stack with the same support for the same price or cheaper built on top of FOSS. Otherwise... persuade those state and local governments to staff up to the necessary levels to be able to hire people able to customize and support the FOSS to fit their needs and be prepared to financially support that decision.
Sure it’d be great for government acquisitions to subsidize open source, but at what cost?
We were a bootstrap so were immediately profitable.
If your product is tested and guaranteed to certain standards, like MySQL and RedHat, then that is something a company may pay for. But a user doesn't really have that high standard so they can be satisfied with just the off brand, derived stuff floating around.
The Stable Diffusion models are proprietary freeware, not open source.
I wonder if, then, Stability AI wanted to sell license exceptions—namely, the ability to use the software for amoral and (mildly) immoral uses.
Dead Comment
Nobody benefits from their failure.
If Runway and Stability can cut costs they will become cherished institutions.
without a stable source of revenue, cutting costs mean nothing. What are they selling, and why isn't there buyers?
A business cannot survive without revenue, and revenue only comes from people who want to buy something from you that they cannot otherwise get else where.
I applaud StabilityAI for releasing SD for free. They could've done what openAI did, and monetized it (which other diffusion model services are doing). By releasing it for free, stabilityAI contributed to the common good. Unfortunately, if there's nothing else that can be sold as a product, they cannot be sustainable.
An alternative, which i'm not too big a fan of, is collectivization of AI models, and make it a global commons for which taxpayers will fund.
...
> cut costs
Unless they can cut costs to zero, something has to change.
Like, example. I use SD in Blender sometimes as part of the compositor. I have maybe a 10% acceptance rate for SD output: sometimes the water isn't right, or the clouds look goofy, or something keeps getting rendered as an anime pillow for some godforsaken reason. If SD captured my prompt history and some of the final model tweaks between runs, they could ostensibly get really solid HITL test data. Then they could be the curator of that "super model" which they could upsell, maybe along with very high rez stuff, or a higher priority on jobs. Again, not an expert, so who knows. And also, having the model local, that gives you back some of the same benefits, but without the scale.
https://youtu.be/YQ2QtKcK2dA
Not too surprised about funding issues from the casual answer.
I’m not saying it was bad to self fund a project, but having to choosing between your life and fun (and potentially very profitable) projects is not easy.
[1] https://github.com/BlinkDL/RWKV-LM
[2] https://huggingface.co/laion/CLIP-ViT-L-14-laion2B-s32B-b82K
[3] https://github.com/lucidrains/gigagan-pytorch#appreciation
This time around though, the means of production are available to anybody with a credit card, and AWS/GCP/Azure throw credits at startups that apply, just to have them locked into their cloud. Break free of the chains of work, with generative pretrained transformers!