Readit News logoReadit News
Posted by u/seanisom 2 years ago
3D + 2D: Testing out my cross-platform WASM graphics engine
I used to work at Adobe on the infrastructure powering big applications like Photoshop and Acrobat. One of our worst headaches was making these really powerful codebases work on desktop, web, mobile, and the cloud without having to completely rewrite them. For example, to get Lightroom and Photoshop working on the web we took a winding path through JavaScript, Google’s PNaCl, asm.js, and finally WebAssembly, all while having to rethink our GPU architecture around these devices. We even had to get single-threaded builds working and rebuild the UI around Web Components. Today the web builds work great, but it was a decade-long journey to get there!

The graphics stack continues to be one of the biggest bottlenecks in portability. One day I realized that WebAssembly (Wasm) actually held the solution to the madness. It’s runnable anywhere, embeddable into anything, and performant enough for real-time graphics. So I quit my job and dove into the adventure of creating a portable, embeddable WASM-based graphics framework from the ground up: high-level enough for app developers to easily make whatever graphics they want, and low-level enough to take full advantage of the GPU and everything else needed for a high-performance application.

I call it Renderlet to emphasize the embeddable aspect — you can make self-contained graphics modules that do just what you want, connect them together, and make them run on anything or in anything with trivial interop.

If you think of how Unity made it easy for devs to build cross-platform games, the idea is to do the same thing for all visual applications.

Somewhere along the way I got into YC as a solo founder (!) but mostly I’ve been heads-down building this thing for the last 6 months. It’s not quite ready for an open alpha release, but it’s close—close enough that I’m ready to write about it, show it off, and start getting feedback. This is the thing I dreamed of as an application developer, and I want to know what you think!

When Rive open-sourced their 2D vector engine and made a splash on HN a couple weeks ago (https://news.ycombinator.com/item?id=39766893), I was intrigued. Rive’s renderer is built as a higher-level 2D API similar to SVG, whereas the Wander renderer (the open-source runtime part of Renderlet) exposes a lower-level 3D API over the GPU. Could Renderlet use its GPU backend to run the Rive Renderer library, enabling any 3D app to have a 2D vector backend? Yes it can - I implemented it!

You can see it working here: https://vimeo.com/929416955 and there’s a deep technical dive here: https://github.com/renderlet/wander/wiki/Using-renderlet-wit.... The code for my runtime Wasm Renderer (a.k.a. Wander) is here: https://github.com/renderlet/wander.

I’ll come back and do a proper Show HN or Launch HN when the compiler is ready for anyone to use and I have the integration working on all platforms, but I hope this is interesting enough to take a look at now. I want to hear what you think of this!

hardwaresofton · 2 years ago
A talk given by OP which is a fantastic intro with 2 successful demos across 2 platforms :)

https://www.youtube.com/watch?v=CkV-nWFXvbs

Disclaimer: I currently work at a company in the WebAssembly space that was involved with this conference

boomskats · 2 years ago
Great talk! Same one I was referring to in my comment below.

Dead Comment

reactordev · 2 years ago
Skip the PAL step and just go right into SetupRuntime with the arguments. Non-gfx devs don’t know about these things and adding extra steps in your API is unnecessary. Since PAL isn’t used anywhere else. Other than that, I would highly recommend getting on the WebGPU train using wgpu-native or dawn. (IPal should be a member of IRuntime and is ripe for removal for WebGPU context).

Keep it up! Bookmarked.

seanisom · 2 years ago
Great suggestion, appreciate it. wgpu is coming!
ronyeh · 2 years ago
Awesome project. What are you planning for text and font support? Some graphics engines don’t support all the ways you might want to display text. Will we be able to load OTF or WOFF2 files and display arbitrary strings? :-)
seanisom · 2 years ago
Thanks! I haven't looked deeply into font yet, but I've always been partial to HarfBuzz for shaping, so will probably build on top of that. It also has an experimental Wasm shaper which certainly served as a bit of inspiration for the design of this.
spuzvabob · 2 years ago
There is a well maintained Wasm build of harfbuzz: <https://github.com/harfbuzz/harfbuzzjs> with both OpenType and AAT shapers support, which should be enough but you can also provide your own shaper implementation in Wasm yes.

We're successfully using Wasm harfbuzz to render text in a web-based design tool with relatively high usage so there should be no issues integrating it :)

ronyeh · 2 years ago
Nice! Looking forward to your alpha release. (And eventual HarfBuzz integration.)
DrScientist · 2 years ago
Second that - while there are a lot of graphics libs out there - text rendering support always seems to be lagging.

Decent support there would be a differentiator in my view.

iFire · 2 years ago
We've been doing work in Godot Engine trying to get wasm working.

How did you overcome the shared array buffer accessibility problem on safari vs access to ad networks which is important for online games?

I called it single threads vs regular builds.

Hope to help make sure there's a diverse set of rendering kernels for everyone.

Edited: Link to our work at making portable 3d graphics on the web with an editor. https://editor.godotengine.org/releases/latest/

iFire · 2 years ago
We also collaborate with https://github.com/thorvg/thorvg.

We were impressed by your work, https://github.com/rive-app and https://graphite.rs/

Keavon · 2 years ago
Thanks! If perhaps we spoke at GDC (taking a guess given the context), it was nice meeting you! (Keavon from Graphite here.)
seanisom · 2 years ago
Big fan of Godot! I think it has done wonders to make graphics more accessible.

From an Adobe perspective - it doesn't. If you go to photoshop.adobe.com in Safari, you will see the answer. Things can work in a single-threaded build, but that is not production code.

I can't speak for the Safari team, but I do see this getting traction soon with the current priorities for Wasm. Seems like now the most common answer is just to use Chrome.

spxneo · 2 years ago
whew game engine running on WASM + WebGPU would finally be what it takes to power browser based AAA titles. We shouldn't have to download executables via garden walled ecosystems that take a huge chunk of devs revenues
jms55 · 2 years ago
I don't see WASM/WebGPU changing anything when it comes to gaming, as an industry, personally. 3d visualizations and interactive websites? Yeah definitely a nice improvement over WebGL 2, if years late. The OP's experience with Adobe is a great example of this.

WebGPU is pretty far behind what AAA games are using even as of 6 years ago. There's extra overhead and security in the WebGPU spec that AAA games do not want. Browsers do not lend themselves to downloading 300gb of assets.

Additionally, indie devs aren't using Steam for the technical capabilities. It's purely about marketshare. Video games are a highly saturated market. The users are all on Steam, getting their recommendations from Steam, and buying games in Steam sales. Hence all the indie developers publish to Steam. I don't see a web browser being appealing as a platform, because there's no way for developers to advertise to users.

That's also only indie games. AAA games use their own launchers, because they don't _need_ the discoverability from being on Steam. So they don't, and avoid the fees. If anything users _want_ the Steam monopoly, because they like the platform, and hate the walled garden launchers from AAA companies.

EDIT: As a concrete example of the type of problem's WASM for games face, see this issue we discovered (can't unload memory after you've loaded it, meaning you can never save memory by dropping the asset data after uploading assets to the GPU, unless you load your assets in a very specific, otherwise suboptimal sequence): https://github.com/bevyengine/bevy/issues/12057#issuecomment...

(I work on high end rendering features for the Bevy game engine https://bevyengine.org, and have extensive experience with WebGPU)

jasonjmcghee · 2 years ago
Can't https://bevyengine.org/ do this?

AFAIK https://wgpu.rs/ makes this possible with Rust.

---

But this is very different than what was demonstrated in the vimeo video.

MenhirMike · 2 years ago
We already have that ability, both Unity and Unreal can run in the browser (though it looks like Unreal moved their HTML5 stuff into a public plugin). I think Godot also works if you don't use C# and stick to GDScript, or use Godot 3.x instead of the current 4.x.

The problem isn't the tech to run the game, it's the marketplace - how do you actually sell the games without losing the huge customer base that buys through Steam and platform-specific stores? If you're popular enough you probably still get a lot of customers, but I doubt it's anywhere near what Steam does for you.

Oh, piracy and anti-cheat are also a problem, because you just can't have a AAA game without Denuvo and Kernel Backdoors anymore (greeting to the Apex Legends players out there!).

There are probably still a few issues that would have to be solved on the game engine side, but I'm willing to say that the game engine is not the problem with browser-based games.

astlouis44 · 2 years ago
Here's links to Unreal Engine 5 running in the browser, for anyone interested:

https://news.ycombinator.com/item?id=39920890

https://simplystream.com/demos

seanisom · 2 years ago
This right here. The web is the OS of the future - the standards are getting there, the tools are just starting to catch up.
tambourine_man · 2 years ago
I think something like this is inevitable and possibly great. The problems I see are the niceties of my platform of choice that I may loose when everything is rendered in a canvas tag.

For instance, I grab individual elements of the UI all the time for sharing screenshots on macOS (like an individual menu, in a transparent png with drop shadow). I have several text shortcuts that work everywhere except on electron apps. Or, for example, how would accessibility work?

Like I said, I think a future where a cross platform open source web stack becomes the standard for UI development is kind of inevitable. I just hope it’s a great stack, with the best ideas from Mac, Windows, Gnome, KDE, etc, and not a lowest common denominator, which is usually the case.

Anyway, this is extremely cool and I’ll keep an eye on the project.

satvikpendem · 2 years ago
Have you read this article by the lead developer of Flutter, Ian Hickson [0]? It describes using WASM just as you describe to have a fully cross platform UI framework, which is a concept that Flutter uses.

[0] https://docs.google.com/document/d/1peUSMsvFGvqD5yKh3GprskLC...

seanisom · 2 years ago
Thanks - that link does not appear to be open access, anyways I don't think I've seen it. I'm familiar with Flutter at a high-level (Kevin Moore gave a great talk on it at Wasm I/O), and I think other than requiring users to work in Dart, it is probably one of the most powerful ways to do cross-platform UI today.

Worth noting that their original GPU backend was Skia, and now they are retooling around Flutter GPU (Impeller)[0], which is kind of designed similarly as an abstract rendering interface over platform-specific GPU APIs.

[0] https://github.com/flutter/flutter/wiki/Flutter-GPU

satvikpendem · 2 years ago
I edited the link to be public, let me know if that still works.

I think the ideal in that article is that people can write components in whatever languages they want, and when they compile to WASM, they can all interoperate. It reminds me of all of those compile-to-Javascript languages for writing micro-frontends, although there is not as much interoperability from a React boundary to say, a ClojureScript boundary.

By the way, what are you building as a solo founder for YC? Is it related to this project? For this project, I'm curious to see how exactly WASM interoperates with the GPU directly, bypassing the platform specific APIs. Do you still have to write GPU-specific parts for each of the GPU manufacturers? I wonder if there would be an open standard called WASM-GPU in the future that abstracts over these but doesn't necessarily touch any of the OS directly.

nmfisher · 2 years ago
Related - I’ve written a Flutter package to wrap the Filament PBR rendering package and I hacked together a WASM implementation so I could build 3D apps in Flutter for web.

It’s still just experimental (I’m waiting for some upstream Dart fixes to land around WASM FFI, and shared memory support would be nice in Flutter too) but I think it’s promising. Bundle size is a bit of an issue at the moment too.

https://github.com/nmfisher/flutter_filament

seanisom · 2 years ago
This is awesome! I'm not fluent with Flutter/Dart but would like to dig in to how the build / Wasm packaging works.

The state of shared memory for Wasm is not great, although raw SharedArrayBuffers work ok in a browser for running multiple guests. Getting multi-memory properly working through llvm is likely a better solution.

We've got a bundle size issue as well even with -O3. I thought it was due to the amount of templated glm simd code we run, but now am convinced its deeper than that into Emscripten. Haven't been able to look into deeply yet.

sheepscreek · 2 years ago
nchmy · 2 years ago
nope
mendyberger · 2 years ago
This is the public link https://t.co/3xeGnKhwYr
zengid · 2 years ago
This is super neat and I am very interested!

I'm in a rush so I can't look to closely now but I have a few questions (and please forgive any stupid questions, I'm not a graphics dev, just a hobbyist):

What's the runtime like? Is there an event loop driving the rendering? (who calls the `render` on each frame? are there hooks into that? ) FFI story? Who owns the window pointer?

I'm interested in audio plugins, and VSTs (etc) have a lot of constrains on what can be done around event loops and window management. JUCE is pretty much the de-facto solution there, but it's pretty old and feels crufty.

PaulDavisThe1st · 2 years ago
The limits on what audio plugins can do is not a function of the drawing toolkit, but the fact that they do not own the event loop if the GUI is run in-process with the host. And as long as they do, they will never own the evelop loop. In addition (and mostly related to this) the top level window they appear in is owned by the host, which also inherently limits the plugin's role in window management.

If you want more, use the capability built into LV2, AU and VST3 for out-of-process GUIs for a plugin (LV2 has had this for more than a decade). CLAP has, I think, abandoned plans to support this based on lack of uptake elsewhere.

I'd hardly call JUCE "pretty old", but then I'm a lot older than JUCE. And it's likely only crufty if you're more used to other styles of GUI toolkits; in terms of the "regular" desktop GUI toolkits, it's really bad at all.

zengid · 2 years ago
Hey Paul, thanks for sharing!

Yes I think JUCE is great, It's very well made, but it drives you into a very narrow path of either using everything in the library, or leaving you to fend for yourself (which I admit may be a normal experience for C++ devs). For instance, the ValueTrees frequently used for UI state are very powerful, but they're not very type safe (or thread safe), and they feel clunky compared to more contemporary reactive state management patterns like signals.

I'm sure folks who use ValueTrees are happy, but I don't see much advancement to that pattern being shared in the JUCE forums. If y'all have some better tricks over in the Ardour project I'd love to know! (BTW, I'm a fan of y'all's work. I really enjoyed reading some of the development resources, like the essay on handling time [0]).

[0] https://ardour.org/timing.html

seanisom · 2 years ago
Great questions!

The host app owns the event loop. I don't foresee that changing even once we re-architect around WebGPU (allowing the Wasm guest to control shaders), as the host app is responsible for "driving" the render tree, including passing in state (like a timer used for animations). The host app owns the window pointer, as renderlets are always designed to be hosted in an environment (either an app or a browser). Open to feedback on this, though.

FFI is coming with the C API soon!

I don't know much about audio but I see a ton of parallels - well-defined data flow across a set of components running real-time, arbitrary code. Simulations also come to mind.

zengid · 2 years ago
Thank you for the reply! I'm excited to watch as this project progresses, and I wish you the best of luck!
pdntspa · 2 years ago
> JUCE is pretty much the de-facto solution there,

Is it though? iPlug nee wdl-ol nee iPlug2 seems pretty good too. JUCE stuff has a pretty distinct and slightly obnoxious look and feel that takes a fair bit of effort to strip out

boomskats · 2 years ago
Not much to add, just wanted to say I thought your presentation at wasm I/O in Barca a couple of weeks ago was amazing and it's great to see this work getting some attention!
mambru · 2 years ago
Barca -> rowing boat

Barça -> the football club

Barna -> cute form of Barcelona

boomskats · 2 years ago
I guess a lot of the English-speaking world has (mis)appropriated the anglicised name of the football club?

Still, for what it's worth, b7a is my favourite city so far.