My question is what's next? Without sacrificing (much) memory. We know from TruffleRuby that given enough warmups and lots of memory it could be multiple times faster than CRuby. And compared to Ruby 3.3 YJIT may be 2-3x faster. But it seems the communities as a whole cares a lot about memory usage.
We know in order to make JIT more useful we need to move some of the Gems from C to Ruby. But other than that do we have anything on the table that will significantly speed up Ruby Rails?
> But it seems the communities as a whole cares a lot about memory usage.
Matz very much sees Ruby as still a general purpose language that still supports scripting as a first tier concern. From that vantage bootup and memory are hugely important. Also providers like Heroku (I work there) charge for memory and the primary way to get parallelism with the GIL/GVL is using multiple processes where memory use can multiply rapidly (literally).
> speed up Ruby on Rails?
You can look at other communities to get a preview of what might come. Ruby now has “object shapes” like node.js which will benefit the community more once people start using it, enabling warnings and fixing hotspots where instance variables are not declared optimally.
Personally, I would like to see rails embrace the power of database backed constraints and fallback to using validation queries only when the database throws an error on commit. You can already saturate a CPU with processes and threads, the biggest bottleneck (IMHO) for most rails apps is still the database. Counterintuitively, pushing those constraints to the database can mean a net decrease in DB computing needed and more overall throughput.
Aside from that I think JIT will help do more with fewer resources. Which makes sense when the main YJIT sponsor has a huge server bill and being able to slash it in half more than justifies such long term gambles.
I’m curious why you think constraints are such a problem? I’ve worked on quite a few large rails apps, and whilst you’re not wrong, the extra traffic from checking constraints has always been a tiny negligible part of overall db traffic.
There is still lots of optimizations to make. On the JIT side, there is still no inlining, which opens the door to much more aggressive optimizations.
But also on memory as well. There was a lot of improvements done on the Ruby garbage collector for 3.3, but there's still a lot of room for improvement, and a few fine folks at Shopify are looking into making the GC swappable so we'd be able to use mttk GCs with MRI.
As for the community caring about memory usage, I wonder how much of that has to do with the default Heroku dynos. Because it's an issue for small scale deployment. Once you scale more, you recoup a lot with CoW etc, so it's much less of a worry.
Not to mention the warm-up time - the graph in the referenced article is measured in seconds, with max > 20!
Can attest, just pulled TruffleRuby (both native and JVM), it is over ten seconds to start a hello world script cold. As in, script being run from command line as needed.
These are gains that are only going to make sense for the long-running process crowd.
It does make sense in the web server context to optimize for memory first over compute. This could explain the dynamic you called out. Furthermore, the propensity for many popular gems to have native portions makes the trade off even more useful.
Regarding JIT being faster, types are the next effective tool IMO.
The two drivers I see are:
- JIT has more stack frames to optimize away, allowing more optimization - you addressed this one
- JIT has more information to limit possible outcomes, therefore finding more opportunities
I used to work with Ruby and Rails and I really found Ruby to be a very comfortable, terse, and expressive language, but I found it very slow after trying some compiled languages (I think I compared service startup and some simple computationally heavy stuff like Advent of Code or Project Euler solutions) and I just couldn’t go back to Ruby after that. Also I found the Rails (framework) stacktraces and source code very hard to follow.
However, that was quite some time ago. Does anyone have any more recent opinions? How is it to work on a large modern Ruby or Rails codebase? Also, I saw some type annotations examples a few years back and thought they looked pretty ugly - is that how I would feel after some months using them, too?
I work at Shopify (not the team that does YJIT) so I am working on a very large, modern Rails codebase. (my opinions are my own)
I think rails code is the easiest code to follow in any code base that I have ever worked on. The enforcement in the framework of the code structure, models containing the object and schema relationships, focus on message passing, skinny controllers. Its very very easy to debug rails code and work in code that is relatively foreign due to the consistency across the code base. Some of this might be discipline at Shopify, I do think we have a very good code review culture. But I have been places that rest controllers just led to a spaghetti of "Operations" and "Processors" that just change state everywhere and a bunch of microservice network calls.
Ruby speed hasn't been an issue. We focus on latency and speed quite a bit and work on keeping request times low. The speed of ruby running is a good trade off for the speed of development I think. And most slow running requests really seem to end up due to network io or database calls that I see.
Sorbet uses a kind of type annotation, its very ugly and I do not like it. I really would prefer something more built into the language. Like def foo(Integer x) -> Integer. The big issue is method arguments I think since Ruby already uses :x, x:, x =, and a lot of the other symbols that indicate types so its hard to add it in an optional fashion.
Just curious, what's your opinion on YARD comments for typing? I have a little more experience with that, than I do Sorbet (or RBS as well) and I found it to be at least tolerable. Though something like your `-> Integer` would be amazing though if there was somehow a way to make it work with the syntax of Ruby.
In my experience most Ruby apps (Rails in particular) develop this idea that Ruby is the bottleneck when in reality most of the time it's something else entirely (mostly DB or network IO). Of course you found compiled languages faster! that's by definition. You also tried Ruby on CPU sensitive code bases ("Advent of Code or Project Euler solutions"), that's textbook "ruby is slower than C". Rails pays dividends in the web and in large code bases with many engineers.
Weird, because if anything I keep hearing people say Ruby performance and GVL don't matter because it's all IO anyway.
Which is true to a certain extent only. When you first start optimizing a Rails app, it's true that bad queries and N+1 is where most time is wasted, but from my experience once you clear these out, the IO/CPU ratio really isn't that high, generally in the 40-60% IOs range.
> Also, I saw some type annotations examples a few years back
You probably saw the third party tool Sorbet. I agree it's ugly and extremely limited. Ruby's native typing solution, RBS, is written in a separate file. It's a bit better but... it's written in a separate file...
Most major companies seem to still primarily rely on Sorbet. Yeah I'd say the typing scene in Ruby is still one of it's greatest limitations
Since Ruby 3 I would recommend trying again. It is much faster now.
Back in the Ruby 2 days we had tools like Zeus to preload our development application because loading was so slow (10-20 seconds). These days we just load the whole app (2 seconds)
These benchmarks show a 2x speed up. While I haven't see that, I have seen at least a 50% increase in performance for our applications.
Obviously as an interpreted language, it's never going to be as fast as something like C, Rust, or Go. Traditionally the ruby maintainers have not designed or optimized for pure speed, but that is changing, and the language is definitely faster these days compared to a decade ago.
If you like the ruby syntax/language but want the speed of a compiled language, it's also worth checking out Crystal[^1]. It's mostly ruby-like in syntax, style, and developer ergonomics.[^2] Although it's an entirely different language. Also a tiny community.
I moved from ruby 2.7 to 3.2 for a rails app and was hopeful that it would lead to large speedups like shopify claims it did for them, but was bummed to find it did basically nothing. Anyone else running a large rails app have a similar or different experience?
Very interesting blog post! My best guess as to how shopify is able to achieve such large increases in speed is they have a lot more actual ruby code than most codebases (https://shopify.engineering/ruby-yjit-is-production-ready).
1. How did you evaluate performance? Did you let it run for a while in production? YJIT's improvements will be most visible over time.
2. If your web app's response times are dominated by database queries, then YJIT will do nothing for you. Even re-writing your app in assembly language won't help. All languages are equally fast when sitting around waiting for the database to response. ;-)
That said, if your response times are dominated by db query wait times, maybe the async queries in Rails 7.x can offer you some very nice improvements. However, you will probably need to restructure (not rewrite, exactly, but restructure) your code to take advantage. Not the whole app, just the hot spots.
We saw memory go up when moving from 3.1 to 3.2, but we also removed compiling Ruby with jemalloc and used the defaults. We have seen performance increase by about 10% for our web applications. Our forking background processes run about the same. YJIT appears to perform for repeated requests – ie first run will not expose any benefit, ie your tests won't run faster.
[edit, the memory increase was small just a few percent IRC]
Sorry to hear you had that experience we saw 30% speedup moving from 2.6 to 3.2 and the 95th in many cases was 40%... Did you happen to enable yjit and if so which app server are you running?
I benchmarked a good sized app (ran a black-box QA suite multiple times) while monitoring performance and req/response times with Prometheus/Grafana with Ruby 3.2.
With YJIT enabled memory usage ballooned and performance dipped below non-YJIT Ruby 3.2, IIRC the difference was a good 10% degradation. Granted, it's an API only service so no HTML is being generated, only JSON and that could be the culprit.
Suffice it to say, we didn't enable YJIT for 3.2. Maybe 3.3 is indeed different, but both the faster and less memory claims are really suspicious to me.
Interesting. Do you think html generation is one of the things that makes the numbers look so good for 3.2 YJIT? We run a mostly json api only app. I think json serialization is notoriously slow in ruby so I was hoping yjit would speed it up.
I think Ruby has at this point caught up to modern languages on everything except typing. Unfortunately the typing story is so poor that it still makes Ruby an incredibly difficult choice to justify in 2023.
Well, in ruby everything is an object that is a member of some class. Those classes have specific properties that only allow for specific methods to be applied on them. Ie, integers are members of the numeric::integer class, and if you want to use that integer as part of a string in output, you do have to convert it to a string first using a method defined as such.
So, what exactly do you see as missing in ruby's typing?
For what use-case? Because I've been developing in ruby for over 10 years now and it has been the optimal choice for a ton of companies I've worked for. The lack of static typing is a feature for a lot of people.
I get the need to improve Ruby's speed and memory usage, but there's much to be desired. I really enjoy Rails and Ruby in general, but the performance is terrible.
I'd like to see how Ruby 3.3 stacks up to other languages and runtimes.
In 2023 if performance is a functional requirement, Ruby is still not the answer IMO. For companies like Shopify, there's really no other alternative at this point, but their "fast" is only fast compared to the performance they've had 1-2 years ago.
Exactly, ruby performance is like someone beating their personal best time jogging around the neighborhood and then writing an article about it. It's something positive in their own ecosystem, but is not competitive when people need results.
Readit News
We know in order to make JIT more useful we need to move some of the Gems from C to Ruby. But other than that do we have anything on the table that will significantly speed up Ruby Rails?
Matz very much sees Ruby as still a general purpose language that still supports scripting as a first tier concern. From that vantage bootup and memory are hugely important. Also providers like Heroku (I work there) charge for memory and the primary way to get parallelism with the GIL/GVL is using multiple processes where memory use can multiply rapidly (literally).
> speed up Ruby on Rails?
You can look at other communities to get a preview of what might come. Ruby now has “object shapes” like node.js which will benefit the community more once people start using it, enabling warnings and fixing hotspots where instance variables are not declared optimally.
Personally, I would like to see rails embrace the power of database backed constraints and fallback to using validation queries only when the database throws an error on commit. You can already saturate a CPU with processes and threads, the biggest bottleneck (IMHO) for most rails apps is still the database. Counterintuitively, pushing those constraints to the database can mean a net decrease in DB computing needed and more overall throughput.
Aside from that I think JIT will help do more with fewer resources. Which makes sense when the main YJIT sponsor has a huge server bill and being able to slash it in half more than justifies such long term gambles.
But also on memory as well. There was a lot of improvements done on the Ruby garbage collector for 3.3, but there's still a lot of room for improvement, and a few fine folks at Shopify are looking into making the GC swappable so we'd be able to use mttk GCs with MRI.
As for the community caring about memory usage, I wonder how much of that has to do with the default Heroku dynos. Because it's an issue for small scale deployment. Once you scale more, you recoup a lot with CoW etc, so it's much less of a worry.
Oh wow. Had no idea there was no inlining. That is potentially another 2-3x performance.
[0] https://www.graalvm.org/latest/reference-manual/ruby/#aim
[1] https://www.graalvm.org/latest/reference-manual/ruby/Polyglo...
[2] https://medium.com/graalvm/graalvm-ten-things-12d9111f307d
But uses 5-10x more memory.
Still has way more upside left to gain.
https://railsatscale.com/2023-11-07-yjit-is-the-most-memory-...
Can attest, just pulled TruffleRuby (both native and JVM), it is over ten seconds to start a hello world script cold. As in, script being run from command line as needed.
These are gains that are only going to make sense for the long-running process crowd.
Regarding JIT being faster, types are the next effective tool IMO.
The two drivers I see are:
- JIT has more stack frames to optimize away, allowing more optimization - you addressed this one
- JIT has more information to limit possible outcomes, therefore finding more opportunities
Just curious, how well do we know this, e.g., theory, experiments, it's being done, or ?
It'd be fun if RiiR[uby] were a small part of the overall memory saftey glacier (e.g., https://www.cisa.gov/case-memory-safe-roadmaps)
However, that was quite some time ago. Does anyone have any more recent opinions? How is it to work on a large modern Ruby or Rails codebase? Also, I saw some type annotations examples a few years back and thought they looked pretty ugly - is that how I would feel after some months using them, too?
I think rails code is the easiest code to follow in any code base that I have ever worked on. The enforcement in the framework of the code structure, models containing the object and schema relationships, focus on message passing, skinny controllers. Its very very easy to debug rails code and work in code that is relatively foreign due to the consistency across the code base. Some of this might be discipline at Shopify, I do think we have a very good code review culture. But I have been places that rest controllers just led to a spaghetti of "Operations" and "Processors" that just change state everywhere and a bunch of microservice network calls.
Ruby speed hasn't been an issue. We focus on latency and speed quite a bit and work on keeping request times low. The speed of ruby running is a good trade off for the speed of development I think. And most slow running requests really seem to end up due to network io or database calls that I see.
Sorbet uses a kind of type annotation, its very ugly and I do not like it. I really would prefer something more built into the language. Like def foo(Integer x) -> Integer. The big issue is method arguments I think since Ruby already uses :x, x:, x =, and a lot of the other symbols that indicate types so its hard to add it in an optional fashion.
Which is true to a certain extent only. When you first start optimizing a Rails app, it's true that bad queries and N+1 is where most time is wasted, but from my experience once you clear these out, the IO/CPU ratio really isn't that high, generally in the 40-60% IOs range.
You probably saw the third party tool Sorbet. I agree it's ugly and extremely limited. Ruby's native typing solution, RBS, is written in a separate file. It's a bit better but... it's written in a separate file...
Most major companies seem to still primarily rely on Sorbet. Yeah I'd say the typing scene in Ruby is still one of it's greatest limitations
Back in the Ruby 2 days we had tools like Zeus to preload our development application because loading was so slow (10-20 seconds). These days we just load the whole app (2 seconds)
These benchmarks show a 2x speed up. While I haven't see that, I have seen at least a 50% increase in performance for our applications.
If you like the ruby syntax/language but want the speed of a compiled language, it's also worth checking out Crystal[^1]. It's mostly ruby-like in syntax, style, and developer ergonomics.[^2] Although it's an entirely different language. Also a tiny community.
[1]: https://crystal-lang.org/ [2]: https://crystal-lang.org/reference/1.10/crystal_for_rubyists...
Isn't the point of YJIT that it's a ruby compiler.
In most WebApps the first performance bottleneck people tend to hit is the DB: missing indexes, n+1 queries, etc.
1. How did you evaluate performance? Did you let it run for a while in production? YJIT's improvements will be most visible over time.
2. If your web app's response times are dominated by database queries, then YJIT will do nothing for you. Even re-writing your app in assembly language won't help. All languages are equally fast when sitting around waiting for the database to response. ;-)
That said, if your response times are dominated by db query wait times, maybe the async queries in Rails 7.x can offer you some very nice improvements. However, you will probably need to restructure (not rewrite, exactly, but restructure) your code to take advantage. Not the whole app, just the hot spots.
https://www.shakacode.com/blog/rails-7-1-active-record-api-f...
[edit, the memory increase was small just a few percent IRC]
With YJIT enabled memory usage ballooned and performance dipped below non-YJIT Ruby 3.2, IIRC the difference was a good 10% degradation. Granted, it's an API only service so no HTML is being generated, only JSON and that could be the culprit.
Suffice it to say, we didn't enable YJIT for 3.2. Maybe 3.3 is indeed different, but both the faster and less memory claims are really suspicious to me.
So, what exactly do you see as missing in ruby's typing?
YJIT is the most memory-efficient Ruby JIT - https://news.ycombinator.com/item?id=38265773 - Nov 2023 (29 comments)
Also:
Ruby 3.3's YJIT Runs Shopify's Production Code 15% Faster - https://news.ycombinator.com/item?id=37579926 - Sept 2023 (154 comments)
Running Ruby 3.2's YJIT in Production at Discourse - https://news.ycombinator.com/item?id=35895309 - May 2023 (1 comment)
Ruby 3.2’s YJIT is Production-Ready - https://news.ycombinator.com/item?id=34413012 - Jan 2023 (286 comments)
Our Experience Porting the YJIT Ruby Compiler to Rust - https://news.ycombinator.com/item?id=31344065 - May 2022 (89 comments)
Ruby YJIT Ported to Rust - https://news.ycombinator.com/item?id=31094130 - April 2022 (88 comments)
Rust YJIT is complete – it passes all the CRuby tests - https://news.ycombinator.com/item?id=31090084 - April 2022 (1 comment)
Ruby: Porting YJIT to Rust - https://news.ycombinator.com/item?id=29971360 - Jan 2022 (48 comments)
Benchmarking CRuby, MJIT, YJIT, JRuby, and TruffleRuby - https://news.ycombinator.com/item?id=29824076 - Jan 2022 (29 comments)
Merge YJIT: an in-process JIT compiler - https://news.ycombinator.com/item?id=28938446 - Oct 2021 (24 comments)
YJIT: Building a New JIT Compiler for CRuby - https://news.ycombinator.com/item?id=28874283 - Oct 2021 (47 comments)
Proposal to Merge YJIT into Ruby - https://news.ycombinator.com/item?id=28691048 - Sept 2021 (77 comments)
YJIT: Building a New JIT Compiler Inside CRuby - https://news.ycombinator.com/item?id=27371977 - June 2021 (1 comment)
I'd like to see how Ruby 3.3 stacks up to other languages and runtimes.
Deleted Comment