Readit NewsIf you already have your own images, you can use the Replicate model directly: https://replicate.com/jagilley/controlnet-scribble -- you can upload your image using the Replicate web UI or do it programmatically using the model's HTTP API.
It seems this does not work on Firefox? I could only draw on about half the canvas and it was pretty buggy. Dont support the chrome monoculture!
Sorry about the trouble. The Firefox incompatibility is the result of a bug in the underlying npm package we're using to render the drawing tool and canvas.
The issue is being tracked here: https://github.com/replicate/scribble-diffusion/issues/17#is...
We may need to wait for a fix to that, or consider swapping out the package we use for scribbling on a canvas.
Thanks for your kind words and feedback. I love seeing all these links to your scribbles.
I'm an engineer at Replicate, which is a place to run ML models in the cloud. [0] We built Scribble Diffusion as an open-source app [1] to demonstrate how to use Replicate.
This is all built on ControlNet [2], a brilliant technique by Lvmin Zhang and Maneesh Agrawala [3] for conditioning diffusion models on additional sources, in this case human scribbles. It also allows for controlling Stable Diffusion using other inputs like pose estimation, edge detection, and depth maps.
ControlNet has only existed for three weeks, but people are already doing all kinds of cool stuff with it, like an app [4] that lets you pose a stick figure and generate DreamBooth images that match the pose. There are already a bunch of models [5] on Replicate that build on it.
I see a few bits of feedback here about issues with the Scribble Diffusion UI, and I'm tracking them on the GitHub repo. If you want to help out, please feel free to open an issue or pull request.
[0] https://replicate.com
[1] https://github.com/replicate/scribble-diffusion
[2] https://github.com/lllyasviel/ControlNet
[3] https://arxiv.org/abs/2302.05543
[4] https://twitter.com/dannypostmaa/status/1630442372206133248
[5] https://replicate.com/explore?query=controlnet