Evan Wallace (former CTO) was a WASM pioneer IMO, he has plenty of excellent explorations and works shared on GitHub. I don't know Evan, but I've greatly appreciated stumbling into his works when exploring SharedArrayBuffer and what was once bleeding edge browser performance.
Absolutely! Evan and the early team laid a strong foundation when Figma was ported to WASM + WebGL. It happened before my time at Figma, but check out these earlier posts from Evan and Jamie on the transition, perf testing, and graphs back from 2017-2018:
That's kind of wild to me to see an app like Figma that's a commercial success, but also a demonstration of technical excellency. My brain automatically assumes that you can't pick both commercial success and technical excellency at the same time and must compromise on the latter to build more features. But maybe there is hope and users actually love the benefits of a better tech stack (performance, consistency, stability) and maybe it's easier to build features with an excellent technical foundation.
Your assumption was correct most of the time. 99% of tech startups are not innovating technically. They are applying existing tech to new problems. In these cases, quality engineering is low priority.
But there is that other 1% that is making new things possible. And if they are extremely lucky, they’ll build new tech and figure out how to turn it into a product. Figma is an example of what can happen when that mix hits.
Most technical innovations, though, will be licensed/sold/etc. to other companies which fall into the first group.
All that said, it turns out customers love software that works well. Startups (and most software companies, really) undervalue quality because it’s hard and not strictly necessary when there’s no direct competition.
> My brain automatically assumes that you can't pick both commercial success and technical excellency at the same time
The problem is your anchor point. Your idea of what software can and should be is based on? Gmail? Salesforce? The GTAV loading screen? The iOS warning that you’re running out of icloud storage space?
Silicon Valley software expectations are tainted by recency bias and private equity excrement.
The current ceiling to aspire to is literally “CI passes”
Really awesome work! If the authors happen to drop by, I'd be curious if some of the work in their pipeline might be able to help for other types of GUI applications with Wasm. I'm also very curious to see what role, if any the Wasm engine's performance plays in the overall story, and be able to distill out any key workloads that can help Wasm engines improve.
WASM gave Figma a lot of speed by default for a lot of perf-sensitive code like rendering, layouts, applying styles and materializing component instances, our GUI code is mostly React and CSS.
WASM engine performance has not been a problem for us, instead we are constantly looking forward improvements in the devex department: debugging, profiling and modularization.
One of the largest challenges of the platform we face today is the heap size limit. While Chrome supports up to 4GB today, that's not yet the case for all browsers. And even with that, we are still discovering bugs in the toolchain (see this recent issue filed by one of our engineers) https://github.com/emscripten-core/emscripten/issues/20137
The challenge of the perf-testing at scale in our company is helping developers to detect perf regressions when they don't expect them - accidental algorithmic errors, misused caches, over-rendering React components, dangerously inefficient CSS directives, etc.
+1 to heap size limit speaking as a PM over here at Figma. We regularly see files with hundreds of thousands of layers, and trying to keep those enormous files under 2GB of memory for usage across all supported browsers is a constant struggle.
Would love to see that cap get raised across the board, it'd enable us and others to do so much more.
Seeing a 20% margin of error for some of their tests (due to VMs, noisy neighbors), makes me curious how others solve this problem. Dedicated hardware seems like a good investment, otherwise you need to constantly overcompensate with multiple runs.
Hey, I am one of the authors of the article and the systems described.
The default 20% margin of error is indeed pretty wide, and it is intended to catch large obvious regressions (e.g. an algorithm accidentally becoming quadratic instead of being linear)
As we described in the blog post, we have the second system based on the real hardware. This system is on-demand. If an engineer has a suspect commit or an experimental change that might affect performance, they would schedule a 30min-1hr run on that queue, where we run selected benchmarks 9-15 times each on laptops of various strength. In this configuration, the margin of error is closer to 2-3% from our observations so far. To get more confidence, you would want to run even more trials, typically we advise 9 iterations, though.
We also do all our daily benchmarking on those laptops too.
Edit: in addition to preventative testing, we also track production metrics in a similar way as described by the sibling comment
At Framer (framer.com) we actually had a bit lower variance (I think it was closer to 15%) after spending three months of solid engineering, ending up with a dedicated Mac Mini running Chrome.
Apart from the admin overhead (things got stuck on OS updates) we ended up abandoning the setup because the variance was too big to get anything useful out of running tests for every pr.
The most reliable way for us to monitor performance today is to count slow frames (>16ms) across all actual users and divide them by total time spent in app. It’s a rough proxy, but pretty accurate at showing us when we mess up.
Often CPU steals are visible in cloud environments. This could be useful for finding some noisy neighbor behaviors, and deciding to either adjusting expectations or rerun.
But things like IO, GPU or memory contestation also could be responsible. There are some fancy new-ish extensions for controlling memory throughput. Intel has Memory Bandwidth Allocation controls in their Resource Director Technology, which is a suite of capabilities all designed for observing & managing cross system resources. There's also controls available for setting up cache usage/allocation.
Yeah, I think I/O has bit us before. Running on AWS, you may randomly get a huge spike in I/O latency (maybe Amazon adjusted the physical hardware, drive failed, etc) and have no idea why.
Is Figma fast? I see huge lag on loading files. Sometimes images don't load for 10+ seconds after the file opens. Based in EU we often don't get thought about in testing.
The answer to "is x fast?" is always "compared to what?"
Having been a long-time user of Figma's somewhat-trailing (at this point) competitor, Sketch, speed is surprisingly one of Figma's most immediately-apparent advantages despite being web-based rather than MacOS-native.
At a previous job, we had a Sketch file that contained more or less an entire B2B app, and at hundreds of megabytes, it took tens of seconds to load (not asynchronously, either, blocking the UI until the whole thing was in memory). A similar everything-file at a more recent job where we used Figma was like night and day: something like three seconds until the file was usable, and perhaps a few more seconds for any big images to load.
Then there's Figma's upstart competitor, Penpot. In my initial explorations, it felt about as responsive as Figma, but when I loaded one of their tutorial files – not even a mega-B2B-app file – everything slowed considerably. The load time wasn't bad, but the frame rate for simply scrolling around the artboards dropped like a rock. While I'm bullish on Penpot, they have a long optimization road ahead of them.
The most telling point to me is when people outside of product, design, or engineering use figma for the first time.
In my experience they dive in and don't think about it, it's just natural to use. Not to say the UX is ideal, because imho it sucks hard, but no one complains about *having to use a webapp when they open figma.
Last time I checked, Penpot was creating/destroying actual SVG elements in the dom. I get the sentiment of wanting to use open technology, but for the average user Figma's "copy as svg" is probably open enough. And switching to canvas based rendering would probably improve performance a lot.
Sounds like y'all might have a lot of large images or raster files?
That I have files with enormous thousands of components and Figma doesn't miss a beat. If you're able to get vector versions of those, then the speed shouldn't be an issue. Obviously if there are a lot of photos, that might be tough.
Figma is anything but fast. Whenever I need to open a file it takes a minute to load on my half a gigabit connection. It's strange how much people are willing to put up with slow software nowadays.
How fast is it after opening the file? I’d imagine there are trade-offs being made where some aspect of performance are sacrificed to improve others but I haven’t used Figma (sadly, it looks really nice)
It does pretty well after the initial loading. But the loading is excruciating for me because I need to open different files again and again throughout the day, and it takes the same amount of time to load every file every time.
Like many other commenters, I found this article surprising. Figma is anything but fast for me. Do you performance-test your software on any Linux distributions?
Figjam on Firefox on Ubuntu is painfully slow and laggy. On a 32-core Ryzen machine, drawing lines on an empty file visibly lags. Doing the same thing in Excalidraw, on the other hand, is extremely responsive.
we test on macs, windows and linux laptops, it is very surprising that drawing 1 rectangle is painfully slow.
Sometimes it happens when your browser does not enable hardware acceleration or when your Linux distro does not know how to switch to the discrete GPU.
We won't be able to tell without getting more of your hw specs and debug information, feel free to reach out to the Figma support or email me at skim@figma.com - this is exactly the type of issues that elud us when looking at prod metrics in aggregation.
Readit News
https://github.com/evanw
https://www.figma.com/blog/webassembly-cut-figmas-load-time-...https://www.figma.com/blog/figma-faster/
From https://madebyevan.com/figma/
But there is that other 1% that is making new things possible. And if they are extremely lucky, they’ll build new tech and figure out how to turn it into a product. Figma is an example of what can happen when that mix hits.
Most technical innovations, though, will be licensed/sold/etc. to other companies which fall into the first group.
All that said, it turns out customers love software that works well. Startups (and most software companies, really) undervalue quality because it’s hard and not strictly necessary when there’s no direct competition.
The problem is your anchor point. Your idea of what software can and should be is based on? Gmail? Salesforce? The GTAV loading screen? The iOS warning that you’re running out of icloud storage space?
Silicon Valley software expectations are tainted by recency bias and private equity excrement.
The current ceiling to aspire to is literally “CI passes”
WASM gave Figma a lot of speed by default for a lot of perf-sensitive code like rendering, layouts, applying styles and materializing component instances, our GUI code is mostly React and CSS.
WASM engine performance has not been a problem for us, instead we are constantly looking forward improvements in the devex department: debugging, profiling and modularization.
One of the largest challenges of the platform we face today is the heap size limit. While Chrome supports up to 4GB today, that's not yet the case for all browsers. And even with that, we are still discovering bugs in the toolchain (see this recent issue filed by one of our engineers) https://github.com/emscripten-core/emscripten/issues/20137
The challenge of the perf-testing at scale in our company is helping developers to detect perf regressions when they don't expect them - accidental algorithmic errors, misused caches, over-rendering React components, dangerously inefficient CSS directives, etc.
Would love to see that cap get raised across the board, it'd enable us and others to do so much more.
The default 20% margin of error is indeed pretty wide, and it is intended to catch large obvious regressions (e.g. an algorithm accidentally becoming quadratic instead of being linear)
As we described in the blog post, we have the second system based on the real hardware. This system is on-demand. If an engineer has a suspect commit or an experimental change that might affect performance, they would schedule a 30min-1hr run on that queue, where we run selected benchmarks 9-15 times each on laptops of various strength. In this configuration, the margin of error is closer to 2-3% from our observations so far. To get more confidence, you would want to run even more trials, typically we advise 9 iterations, though.
We also do all our daily benchmarking on those laptops too.
Edit: in addition to preventative testing, we also track production metrics in a similar way as described by the sibling comment
I.e I'm curious if there's a cloud provider managing them for you or you guys keep them in a closet somewhere.
I worked on a really perf sensitive system and for perf tests we would run the last x commits each time to get rid of the busy vm syndrome.
It meant that the margin of error could be much less.
You might want to consider it as a mid way step between vm’s and scheduling on laptops (those poor laptop batteries!)
Ed
Apart from the admin overhead (things got stuck on OS updates) we ended up abandoning the setup because the variance was too big to get anything useful out of running tests for every pr.
The most reliable way for us to monitor performance today is to count slow frames (>16ms) across all actual users and divide them by total time spent in app. It’s a rough proxy, but pretty accurate at showing us when we mess up.
Often CPU steals are visible in cloud environments. This could be useful for finding some noisy neighbor behaviors, and deciding to either adjusting expectations or rerun.
But things like IO, GPU or memory contestation also could be responsible. There are some fancy new-ish extensions for controlling memory throughput. Intel has Memory Bandwidth Allocation controls in their Resource Director Technology, which is a suite of capabilities all designed for observing & managing cross system resources. There's also controls available for setting up cache usage/allocation.
Having been a long-time user of Figma's somewhat-trailing (at this point) competitor, Sketch, speed is surprisingly one of Figma's most immediately-apparent advantages despite being web-based rather than MacOS-native.
At a previous job, we had a Sketch file that contained more or less an entire B2B app, and at hundreds of megabytes, it took tens of seconds to load (not asynchronously, either, blocking the UI until the whole thing was in memory). A similar everything-file at a more recent job where we used Figma was like night and day: something like three seconds until the file was usable, and perhaps a few more seconds for any big images to load.
Then there's Figma's upstart competitor, Penpot. In my initial explorations, it felt about as responsive as Figma, but when I loaded one of their tutorial files – not even a mega-B2B-app file – everything slowed considerably. The load time wasn't bad, but the frame rate for simply scrolling around the artboards dropped like a rock. While I'm bullish on Penpot, they have a long optimization road ahead of them.
In my experience they dive in and don't think about it, it's just natural to use. Not to say the UX is ideal, because imho it sucks hard, but no one complains about *having to use a webapp when they open figma.
That I have files with enormous thousands of components and Figma doesn't miss a beat. If you're able to get vector versions of those, then the speed shouldn't be an issue. Obviously if there are a lot of photos, that might be tough.
Figjam on Firefox on Ubuntu is painfully slow and laggy. On a 32-core Ryzen machine, drawing lines on an empty file visibly lags. Doing the same thing in Excalidraw, on the other hand, is extremely responsive.
we test on macs, windows and linux laptops, it is very surprising that drawing 1 rectangle is painfully slow.
Sometimes it happens when your browser does not enable hardware acceleration or when your Linux distro does not know how to switch to the discrete GPU.
We won't be able to tell without getting more of your hw specs and debug information, feel free to reach out to the Figma support or email me at skim@figma.com - this is exactly the type of issues that elud us when looking at prod metrics in aggregation.
Is your GPU doing any work? What are the results from here? https://pmndrs.github.io/detect-gpu/