Readit NewsI firefox there's something wonky going on with the scribble area; the bottom of the image doesn't seem to show up until after you've drawn a lot more in the top of the image.
Yeah sorry about that. Tracking the issue here: https://github.com/replicate/scribble-diffusion/issues/17#is...
Will be nice to be able to upload a picture/sketch instead of having to sketch something out in the scratchpad. I tend to sketch block diagrams from time to time and hate having to use some tool to draw/drag/align things.
Very cool idea though!
Hey there. Scribble Diffusion author here.
If you already have your own images, you can use the Replicate model directly: https://replicate.com/jagilley/controlnet-scribble -- you can upload your image using the Replicate web UI or do it programmatically using the model's HTTP API.
Looks awesome, I think this is a great mix with prompt engineering.
It seems this does not work on Firefox? I could only draw on about half the canvas and it was pretty buggy. Dont support the chrome monoculture!
Hey, Scribble Diffusion author here.
Sorry about the trouble. The Firefox incompatibility is the result of a bug in the underlying npm package we're using to render the drawing tool and canvas.
The issue is being tracked here: https://github.com/replicate/scribble-diffusion/issues/17#is...
We may need to wait for a fix to that, or consider swapping out the package we use for scribbling on a canvas.
Thanks for your kind words and feedback. I love seeing all these links to your scribbles.
I'm an engineer at Replicate, which is a place to run ML models in the cloud. [0] We built Scribble Diffusion as an open-source app [1] to demonstrate how to use Replicate.
This is all built on ControlNet [2], a brilliant technique by Lvmin Zhang and Maneesh Agrawala [3] for conditioning diffusion models on additional sources, in this case human scribbles. It also allows for controlling Stable Diffusion using other inputs like pose estimation, edge detection, and depth maps.
ControlNet has only existed for three weeks, but people are already doing all kinds of cool stuff with it, like an app [4] that lets you pose a stick figure and generate DreamBooth images that match the pose. There are already a bunch of models [5] on Replicate that build on it.
I see a few bits of feedback here about issues with the Scribble Diffusion UI, and I'm tracking them on the GitHub repo. If you want to help out, please feel free to open an issue or pull request.
[0] https://replicate.com
[1] https://github.com/replicate/scribble-diffusion
[2] https://github.com/lllyasviel/ControlNet
[3] https://arxiv.org/abs/2302.05543
[4] https://twitter.com/dannypostmaa/status/1630442372206133248
[5] https://replicate.com/explore?query=controlnet