I remember having mixed feelings about Sorbet when I first joined Stripe in late 2018, but by the time I left, I found it indispensable. Especially after the VS Code extension was released internally... holy crap, that made such a huge difference (vs having CI fail 20 mins after pushing up a PR because you forgot to run the typechecker script ahead of time, ugh).
This article also made me laugh, because it reminded me of one of my small pet peeves about the Ruby codebase at Stripe: the fact that you would often find `merchant`, `account`, `invoice`, etc used as method parameters that represented the _ID_ of the resource rather than the resource itself. So Sorbet definitely helped with that, but it also could've been nice to just write `invoice_id` instead... :P
This is exactly how I felt when I was first forced to use TypeScript instead of JavaScript, but I can't even tell you the number of hours it has saved me. Now, years later, I can't stand using regular JavaScript, and would never recommend it for any project that will be beyond a toy.
I use Elm but not Typescript. I also write plain javascript intensively. I've seen languages adding static type layer after the fact they are dynamic type. They never end well, lots of edge case here and there, it becomes a problem rather than helper. Static type lang needs to think from start how they will compose nicely, what's pure, what's effect etc. However, the ship has sailed, I don't actually want to debate on this, because web tech is becoming a cult. (the last phrase added for downvote bait, lets do that!)
.Net CodeContracts were this for me, but that was never fast. C# is a statically-typed language, so what kind of types could you further add to it? As a trivial example, maybe you want to ensure that `invoice_id` consists of at least 20 characters. CodeContracts would then statically attempt to prove that assertion for you. They had meticulously annotated the entirety of the .Net stdlib, and those annotations were spot-on.
This obviously helps with things like `null`.
It completely changed the way I code. You have to think a little bit more about how you structure your code if you plan to hand it off to a theorem prover. I unlearned several bad habits (I unlearned even more with Rust).
> the fact that you would often find `merchant`, `account`, `invoice`, etc used as method parameters that represented the _ID_ of the resource rather than the resource itself
I've encountered a few Rails projects in the wild that do this. One solution is to make liberal use of the `to_param` method. This method converts objects to strings that are intended for use in URLs. Of particular note, it's the identity function for strings and numbers, but returns `.id.to_s` for ActiveRecord models. Using this within definitions makes your function polymorphic for whether it accepts a model or an id.
If you do this widely, would probably be best to monkey-patch in your own `to_id` method.
I also joined Stripe in 2018, and thought Sorbet was a waste of time. I quickly changed my mind when I realized how many incidents it prevented. Now I want types for :allthethings!
Is there anything you can think to say to convince the old you? I have a few friends who haven’t yet seen the typing light.
I also think Stripe’s API (external) should not be moving ids and objects. Given some payload in which ‘account_id’ is always present and ‘account’ may be the object (using ‘expand’ IIRC?) or not makes a lot more sense to me.
My experience has been that the people opposed to types won't be convinced to start liking them by anything you can tell them or have them read. In all of the cases where I've seen Sorbet be adopted, the process looked like this:
1. Ambitious team who wants types does work to get the initial version passing in CI. Importantly, it's only checking at `# typed: false`, which basically only checks for missing constants and syntax errors.
2. That initial version sits silently in the codebase over a period of days or weeks. If new errors are introduced, it pings the enthusiastic Sorbet adoption team; they figure out whether it caught a real bug or whether the tooling could be improved. It does not ping the unsuspecting user yet.
3. Repeat until the pings are only high-signal pings
4. Turn Sorbet on in enforcing mode in CI. It's still only checking at `# typed: false` everywhere, but now individual teams can start to put `# typed: true` or higher in the files they care about.
5. Double check that at this point it's easy to configure whatever editor(s) your team uses to have Sorbet in the editor. Sorbet exposes an LSP server behind the `--lsp` flag, and publishes a VS Code extension for people who want a one-click solution.
6. Now the important part: show them how good Sorbet is, don't tell them. Fire up Sorbet on your codebase, delete something, and watch as the error list populates instantly. Jump to definition on a constant. Try autocompleting something.
In my experience trying to bring static types to Ruby users, seeing is really believing, and I've seen the same story play out in just about every case.
One final note: be supportive. Advertise one place for people to ask questions and get quick responses. Admit that you will likely be overworked for a bit until it takes off. But in the long run as it spreads, other teammates will start to help out with the evangelism as the benefits spread outward.
I haven't yet seen the typing light, so maybe my perspective might spawn some replies that could help.
I understand that types are helpful, especially within the context of learning new code (both as a new dev, and more easily understanding new code written by your team), but my hesitation to "jump ship" to them comes from weighing all the "boilerplate" code necessary to get them working versus the tangible benefits I feel they'd give.
To that end, maybe enumerating over the benefits for your friends (ideally more specifically than just "fewer bugs" or "easier to read/write") might tilt the scale more towards using them.
The article shows some good examples of problems that types solve, but many of them could also be "fixed" with better naming schemes, standardizations, and/or refactors -- and then wouldn't also be littered with type annotations (which take up 50% of the code in their example, [1], where I've also added a couple other easy-to-understand "solutions" with less cruft.
IMO, most of the typing syntax in their examples is distracting from the actual code -- especially when just writing better or more standardized code (which you still need to do with types) would make the typing syntax, again IMO, less necessary.
What are some examples of where types have saved you? I mean, I can imagine some, but are there any good blog posts or have an example yourself?
I've moved on from Ruby to Elixir and we use typespecs at my current job, but I still never do in any of my own code. Elixir does have a way to do type hinting which I do use and appreciate when appropriate. So I'm not "against" types and like the idea in theory, but I've never really felt the pain.
The most tangible argument I've bought is that types help out in a huge codebase. I can definitely see this. This is really solving a social problem, of course, because on smaller teams, it's much easier to enforce coding standards, especially if you follow XP and pair all the time. For example, in large dynamic codebases I've worked on, calling something `object` when it really means `object_id` would never fly. Variables and functions always have to be full words describing exactly what they do. Functions are only ever allowed to return one type. And of course everything is thoroughly tested. Of course I understand that not all these things are practiced everywhere and nor do they need to be. But for me that has always made for incredibly readable code.
I'm writing this just to give the perspective of a dynamic weenie and maybe looking to be convinced a little more? I also know three incredibly good programmers with 20-50 years experience each who all wrote in typed languages for years and then couldn't be happier not to have types when they moved to Ruby. So ya, I dunno what I'm expecting here, but mostly offering my perspective.
But the TL;DR: if I join your team and you want me to use types, I'll do it! But I really feel there are ways around it but those ways may not be for everyone (and sometimes perhaps impractical).
hmmm, I dunno, this was longer than I meant it to be, haha.
Remember, this are devs that worked on codebases that have millions lines of Ruby code (Stripe) and most probably have to handle loads of state in the application layer, so their context/anecdata might be different for your friends. I prefer dynamically typed languages for most stuff but at the scale of Stripe I would also like to have a type checker.
With Ruby 3 releasing with RBS, for new projects, whats the current advised path? Native RBS or Sorbet? Can they co-exist and if so, is there a point to using both?
Sorbet is something I've been interested in using for a couple years and finally got a round to actually trying it out. I tried to use Sorbet with ruby 3.1.1 but unfortunately it didn't "just work" which I think is crucial for mass adoption. I want to give the benefit of the doubt and say its my local env that causing issues with Sorbet but in a fresh `rails new test_app --api` project, I'd expect `srb init` to work without errors... maybe I need to give it another go, curious on your thoughts above tho! :)
We discovered a bug in `srb init` for Ruby 3.1 recently that a teammate of mine is working on fixing at the moment. It's likely that if you tried again in a few days it'll have been fixed. Sorry about that, totally agree that the out-of-box experience should just work.
I wrote up an FAQ about the state of Ruby 3 and RBS here:
The tl;dr is that RBI files (not RBS files) will probably always be the preferred way to declare types for third party code (because it will always support exactly the same set of features that Sorbet does). We have some people in the community look into teaching Sorbet to read the RBS format, but the existing parsers for RBS files are written in Ruby and are very slow, and there are some ambiguities in the spec that make writing a third party parser that compiles to native code tricky. You can see an attempt to write a fast RBS parser in C++ here[1], but again given that RBI files do everything we need them to right now and we have other features people are asking us for, we haven't prioritized RBS support incredibly highly.
`srb init` has had a lot of problems since Ruby 3.x, and while I haven't tried in a few months it looks like there's recent issues that it still doesn't work (https://github.com/sorbet/sorbet/issues/5332). Is the advice just to use Tapioca instead of `sub init` at this point?
I tried adding this to a new Rails project with no luck. Is there a sample Rails app with Sorbet fully configured (i.e. most gems typed) available on github for reference?
Unfortunately I don't know of an example repo, but I do know that most projects (except Stripe) who use Sorbet use it with Rails. There's a #rails channel on Slack—maybe you'd like to try asking there!
Hey, nice work with Sorbet! I am one of the grad students who worked on RDL, one of the early research projects related to Ruby type systems. What are the next set of challenges that a tool like Sorbet needs to solve? I see you mentioned meta-programming in the blog post, is that something that is handled well by Sorbet? Sorry if this is already handled, I haven't been up to date with the latest features of Sorbet.
Thanks for your work on RDL! The post didn't mention it, but Sorbet still owes most of its type definitions for the Ruby standard library to RDL's original annotations. We just borrowed them and changed the syntax.
Our general approach to metaprogramming at the moment has been two-fold:
- Use ahead-of-time code generation powered either by runtime reflection or ad-hoc static analysis to generate RBI files declaring things that have been metaprogrammed.
- Build type system features, errors, and autocorrects that encourage people to structure their code in ways that doesn't require metaprogramming to solve.
Metaprogramming is definitely still a sticking point, but the existing solutions work ~okay and the rest of the upside Sorbet provides make it worthwile to power through.
Next challenges:
- Make it faster. While the post was talking about how fast it is, it wasn't telling the whole truth. Turns out some type checking operations in a 15 million line codebase are still slow, and we're working on making those faster.
- Add more IDE features. At the beginning of this year I put a lot of work into making Sorbet's parser more tolerant of syntax errors, which helps things like autocompletion work better. We also want to make more code actions, autocorrects, and refactoring tools, to bring Ruby in line with what you'd expect from other typed languages in the IDE experience
- Add more type system features. Shapes and tuples are a huge unimplemented feature still, and people ask about it all the time. There are a handful of other type system features (happy to list them if you're curious) that would also let people write idiomatic Ruby and still have good typing.
Thank you for writing up such a nice overview. I've been busy trying to get tests into various legacy projects I've inherited - definitely interested in tackling types next.
Couple of questions:
> The declare_method call above acted like a decorator on the def call method: it would check that the msg argument given to call was a String and that call returned a String on every invocation
How was this implemented? Dynamically altering Object#send or something along those lines?
How is the story for sorbet and vim/nvim?
Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)?
The mechanism in the post is the same mechanism in use today with Sorbet signatures.
To get the `sig` method in scope, you have to put `extend T::Sig` in that class (or one of its parents). When `sig` is called for the first time in a class, it monkey patches that class to install some overrides of the method_added method. Ruby calls this method_added method for you every time it creates a method (from any means, static or dynamic). Code is here:
Great, honestly better than VS Code for everything except autocompletion. I personally use Sorbet with Neovim.
> Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)?
Not that I know of unfortunately, but I also pay more attention to the type checker and it’s bugs than the tooling people build around it to get real world work done
Overall sorbet has been good and useful, so thank you. But you just have to be careful when you're not using the same version of ruby that stripe uses. I recently got hit by a bug:
# typed: strict
extend T::Sig
sig {params(x: Integer, y: String).void}
def run(x:, y:)
puts(x + 1, y)
end
args = {
x: 1,
y: "Hi"
}
run args
type checks just fine in sorbet, but is an error in ruby 3+. It works though (with errors) in 2.7 and lower which is what Stripe uses from what I gather.
Note that Shopify is much better about keeping up with regular Ruby updates (Stripe takes a pretty conservative approach there), and they're using it with Ruby 3+ with no problem.
You're right that Sorbet does not yet catch these bugs yet, and we'll likely get around to it in the future. It's being tracked in this issue[1].
We haven't quite prioritized this yet partly because it doesn't actually prevent you from using Sorbet in a Ruby 3 codebase, it just won't report all the errors it could (e.g., Sorbet allows using `T.untyped`, which can also cause runtime errors to go unreported).
Now that you've jogged my memory, there used to be some weirdness in our internal representation for method calls that made implementing this feature tricky. But we've since refactored some internal data structures and now it's probably a lot easier. Maybe I should look into how much work this would be again ...
I tried using sorbet through it's LSP in neovim. I would get diagnostics not after every change, but after every change that adds/removes a function definition. Effectively, I would have to delete and add back a function definition, or the "end" keyword, to trigger sorbet to type check my code.
I added Sorbet to my codebase right after reading the article, but it seems to be expecting that I annotate every single one of my gems. Is this accurate? Is there a way around this?
Somewhat. They don't all need super specific types for every method they've defined, but Sorbet does at least need to know all the classes, modules, and constants in use in your codebase, whether those come from code you've written or code inside gems.
But there's tooling (first-party and third-party) that will either download or generate RBI files defining constants that come from gems. `srb init` is the first party solution, and Shopify's `tapioca` gem is the most popular third-party solution[1].
Unfortunately, because Ruby doesn't have import statements at the top of every file, Sorbet can't just do something like silently treat unknown imports as not having a type (like TypeScript and Flow can do), because then it would never be able to tell between "exists but unknown" vs "typo; does not exist" for constant definitions. This definitely makes the adoption process a little tricker compared to other languages, but it's generally a one-time thing once you've got the tooling set up.
Also if you're ever having trouble getting the tooling to work, there's a lot of people chatting about Sorbet daily at https://sorbet.org/slack
Unfortunately while we've developed the Sorbet Compiler in the open, we've mostly focused on internal-only use cases. There are other exciting efforts to improve Ruby performance though! You might be curious to read more about YJIT, the JIT compiler that was recently merged into Ruby 3.1—it’s being primarily developed by a team at Shopify where Ruby performance is the highest priority.
We've been starting to use Sorbet at Figma and honestly it's been pretty cool! Sorbet is definitely not as good at TypeScript (yet?). It's more verbose, doesn't support things like recursive types and records (shapes are experimental), and it doesn't inspire the same confidence TS does but it's definitely worth it to add it to your codebase if it's big enough!
Also, it's fast! I'm in total agreement with the point made in the article. That makes a huge difference in developer UX.
I have some concrete ideas for how to fix shape types to make them not incremental. Just a matter of finding the time to push the prototype over the line, and do a migration on Stripe's codebase to fix or silence the ensuing errors. It's one of the most requested features for sure, and I think once we implement it Sorbet will feel much better to use, especially in smaller projects and scripts where you don't want to have to define `T::Struct` for one-off data structures.
Wow that would be awesome! Shapes/Records are definitely the #1 thing I'm missing from moving over from TypeScript.
I started thinking about this a bit and I came up with the conclusion that the single biggest difference between structs and shapes is really iterating over keys. I spent some time trying to create structs by which you could iterate over all the keys and all the solutions seemed clunky or inelegant.
huh... didn't expect figma to be using any ruby. what do you all use it for there? I'm mainly a ruby programmer lately but I used figma for my last project design and it was really lovely to use so good work!
Sorbet and/or RBS seems like they may be the future given how popular typescript is with JS programmers these days. There are some other projects that assist programmers without relying on formal type definitions in the source or shadow typing files:
Solargraph combines inference and insight from YARD docs (standard for many gems, plus Castwide has written more YARD for the standard library) to make some pretty good guesses. Crucially it has plugins that add the insights from popular gems with static analysis (e.g. reek, rubocop). I maintain solargraph-rails, which parses your Ruby to make guesses about (surprise) Rails.
The typeprof gem can help IDE plugins make typing guesses based on your tests. This project is interesting to me because it's going into Ruby 3.1 so I think it reflects awareness from the core ruby team that many programmers are not ready to add types to their code.
Sorbet is very useful, but the ergonomics suck. It’s fucking difficult to write rspec tests. The performance overhead of writing sorbet on rails in a big codebase is so much that we have turned it off. The pre-interpreter type checking is somewhat useful.
The alpha releases are also a big concern. We are stuck on a 300 commit (release) old build and can never upgrade safely.
We have also never been able to get the VSCode extensions to run.
Thanks for Sorbet, but I’d suggest people outside Stripe to look elsewhere.
My problem with Sorbet is how ugly it looks. It's probably not possible to do with Ruby (now) but Crystal does it better[1] while (mostly) retaining the syntax of Ruby. I suppose it's trying for Elixir's @spec, which I like less than Crystal's sigs but still prefer over this.
To be honest I struggle with the frustration of programmers of dynamic languages building their own (mostly half-assed) type checkers, after years and years of dealing with their bogus arguments about why dynamic languages are superior and all you need is unit tests (the same cohort is also now on a misguided crusade against unit tests but that's a whole other story).
At least the JS crowd had the decency to buy into a whole new (far better) language instead of a bolt-on solution.
Young people are coming around ... I used to write code in C, C++ then Java, C# (.NET v1). I could never understand how people could implement large systems with dynamically typed languages such as Ruby or Python. They are great for smallish scripts, but once your codebase (and team) grows, they become a nightmare to maintain.
In my experience, large codebases of those types of languages have a lot of "magic" thing happen. There's a lot of implicit stuff that one has to guess or spend time "following the code" to understand what it is doing.
And I say this after having built a major lending platform from scratch in Ruby, including a major Machine Learning scoring system in Python, having to maintain with a good sized payment system in pure JavaScript, and nowadays dealing with a major trading/liquidity system in Ruby.
They are fun languages, but once the code and systems start to scale, static typing really helps. For that reason I've seen a lot of these endeavours try to move to TypeScript or other typed languages.
I've worked with Ruby + Sorbet, and also with Java. I would rather write Ruby + Sorbet than Java right now. Ruby is a really nice language.
Though Java still has some great strengths, especially the 8+ functional programming features and the concurrency library is great. If I could use Rails with Java it might be a different story though, since I hate Spring.
If I put together my top ten language features it would include documentation (both for the core language but also the doc framework for coders), legible stack traces, tight IDE integration (this typically requires strong+strict typing), easy debugging, first class build tools for perfectly reproducible builds and dependency management, and easy version management. Java ticks all those boxes. In fact, I think the thing that launched Java in the first place was not nonsense like checked exceptions, it was javadoc. In terms of syntax and semantics, I've never felt any more or less comfortable coding in anything in the Algol family. Anyone who's been coding long enough knows that 98% of the complexity in building software is people. And anything that helps you understand other people's code is 10X more valuable than anything that helps you code faster.
I started my career in Java and although I loved IntelliJ and was perplexed by how poor RubyMine seemed by comparison (circa 2012) that's not enough for me to go back from Ruby to Java.
What I'm realising now is that Java has (had?) a really clunky type system and that there are other languages that do types better, so I shouldn't use Java as my reason for avoiding them.
I was asked a version of this question by a colleague at work, namely: "if types are so great, why didn't Python/Ruby/JS include them from the start (ie. early '90s)?"
That's because the theory of gradual type systems was only worked out in the '00s. Before that, you could have a static or dynamic type system, not anything in between. Common Lisp did have type annotations, but they were hints for optimization, without any guarantees. They were also local to subroutines only. Dylan[3] is an example of an early implementation of the idea, but Dylan was several years late and, without being able to compete with Java, died without ever being widely used.
The proper theory was first established by J. Siek[1] and W. Taha in 2006. It's distinct from nominal static typing which uses a single top type (like Object in Java) or generics, and obviously it's different from both purely static and dynamic typing. It took almost a decade for the idea to start gaining practical implementations - I think the original was a made for Scheme, and one of the first implementations was Typed Scheme for PLT Scheme, which continues on as Typed Racket[2] today. Typed Racket is unique in that it enforces the types even on the untyped side, by wrapping values and exports in contracts.
The idea proved to be useful in practice, and started being adopted in various (non-Scheme) dynamically typed languages, starting with TypeScript for JS and Hack for PHP. On the other hand, some statically typed languages also became gradually typed, most notably C#. The implementations continued to improve, shrinking the parts of their respective languages that could not be statically typed. In dynamic languages there are still features that cannot be practically expressed in static type systems - most metaprogramming and code generation falls into this category - but they are generally "good enough" for day to day coding.
Gradual typing is useful in the same way static type systems are useful: it can prevent certain kinds of errors by marking known-invalid expressions without the need to run the code (so, for example, can help you find errors even in code that's not covered by tests); it helps in writing tooling for the language (eg. go to definition, find references); it helps make the code clearer for the reader (no need to break into a debugger to see what kind of value a given identifier refers to); in some implementations it may also help in optimizing the runtime performance, but that's rare. The "gradual" aspect makes it easier to adopt when the codebase grows larger - the bigger the codebase, the more useful static types are, but by the time the codebase grows large enough to justify static typing it's too big to rewrite in a different, statically typed language.
In short: writing small projects or prototypes in a dynamically typed language is faster while maintenance and expansion of large projects is easier in statically typed one. Gradual typing lets you go from one to the other without a huge cost of a full rewrite.
> In dynamic languages there are still features that cannot be practically expressed in static type systems - most metaprogramming and code generation falls into this category
Actually, both metaprogramming and code generation blur the phase distinction between compile and run time - but this makes them a great fit for dependent typing systems, which do pretty much the same thing. So there's no reason why more refined static systems could not express these idioms.
Readit News
This article also made me laugh, because it reminded me of one of my small pet peeves about the Ruby codebase at Stripe: the fact that you would often find `merchant`, `account`, `invoice`, etc used as method parameters that represented the _ID_ of the resource rather than the resource itself. So Sorbet definitely helped with that, but it also could've been nice to just write `invoice_id` instead... :P
Makes me nostalgic though, good times!
This obviously helps with things like `null`.
It completely changed the way I code. You have to think a little bit more about how you structure your code if you plan to hand it off to a theorem prover. I unlearned several bad habits (I unlearned even more with Rust).
If you have a type system then you should leverage it, not bolt on something extra
I've encountered a few Rails projects in the wild that do this. One solution is to make liberal use of the `to_param` method. This method converts objects to strings that are intended for use in URLs. Of particular note, it's the identity function for strings and numbers, but returns `.id.to_s` for ActiveRecord models. Using this within definitions makes your function polymorphic for whether it accepts a model or an id.
If you do this widely, would probably be best to monkey-patch in your own `to_id` method.
I also think Stripe’s API (external) should not be moving ids and objects. Given some payload in which ‘account_id’ is always present and ‘account’ may be the object (using ‘expand’ IIRC?) or not makes a lot more sense to me.
1. Ambitious team who wants types does work to get the initial version passing in CI. Importantly, it's only checking at `# typed: false`, which basically only checks for missing constants and syntax errors.
2. That initial version sits silently in the codebase over a period of days or weeks. If new errors are introduced, it pings the enthusiastic Sorbet adoption team; they figure out whether it caught a real bug or whether the tooling could be improved. It does not ping the unsuspecting user yet.
3. Repeat until the pings are only high-signal pings
4. Turn Sorbet on in enforcing mode in CI. It's still only checking at `# typed: false` everywhere, but now individual teams can start to put `# typed: true` or higher in the files they care about.
5. Double check that at this point it's easy to configure whatever editor(s) your team uses to have Sorbet in the editor. Sorbet exposes an LSP server behind the `--lsp` flag, and publishes a VS Code extension for people who want a one-click solution.
6. Now the important part: show them how good Sorbet is, don't tell them. Fire up Sorbet on your codebase, delete something, and watch as the error list populates instantly. Jump to definition on a constant. Try autocompleting something.
In my experience trying to bring static types to Ruby users, seeing is really believing, and I've seen the same story play out in just about every case.
One final note: be supportive. Advertise one place for people to ask questions and get quick responses. Admit that you will likely be overworked for a bit until it takes off. But in the long run as it spreads, other teammates will start to help out with the evangelism as the benefits spread outward.
I understand that types are helpful, especially within the context of learning new code (both as a new dev, and more easily understanding new code written by your team), but my hesitation to "jump ship" to them comes from weighing all the "boilerplate" code necessary to get them working versus the tangible benefits I feel they'd give.
To that end, maybe enumerating over the benefits for your friends (ideally more specifically than just "fewer bugs" or "easier to read/write") might tilt the scale more towards using them.
The article shows some good examples of problems that types solve, but many of them could also be "fixed" with better naming schemes, standardizations, and/or refactors -- and then wouldn't also be littered with type annotations (which take up 50% of the code in their example, [1], where I've also added a couple other easy-to-understand "solutions" with less cruft.
IMO, most of the typing syntax in their examples is distracting from the actual code -- especially when just writing better or more standardized code (which you still need to do with types) would make the typing syntax, again IMO, less necessary.
[1] https://sorbet.run/#%23%20typed%3A%20true%0Aextend%20T%3A%3A...
I've moved on from Ruby to Elixir and we use typespecs at my current job, but I still never do in any of my own code. Elixir does have a way to do type hinting which I do use and appreciate when appropriate. So I'm not "against" types and like the idea in theory, but I've never really felt the pain.
The most tangible argument I've bought is that types help out in a huge codebase. I can definitely see this. This is really solving a social problem, of course, because on smaller teams, it's much easier to enforce coding standards, especially if you follow XP and pair all the time. For example, in large dynamic codebases I've worked on, calling something `object` when it really means `object_id` would never fly. Variables and functions always have to be full words describing exactly what they do. Functions are only ever allowed to return one type. And of course everything is thoroughly tested. Of course I understand that not all these things are practiced everywhere and nor do they need to be. But for me that has always made for incredibly readable code.
I'm writing this just to give the perspective of a dynamic weenie and maybe looking to be convinced a little more? I also know three incredibly good programmers with 20-50 years experience each who all wrote in typed languages for years and then couldn't be happier not to have types when they moved to Ruby. So ya, I dunno what I'm expecting here, but mostly offering my perspective.
But the TL;DR: if I join your team and you want me to use types, I'll do it! But I really feel there are ways around it but those ways may not be for everyone (and sometimes perhaps impractical).
hmmm, I dunno, this was longer than I meant it to be, haha.
Have you considered using pre-commit?
Sorbet is something I've been interested in using for a couple years and finally got a round to actually trying it out. I tried to use Sorbet with ruby 3.1.1 but unfortunately it didn't "just work" which I think is crucial for mass adoption. I want to give the benefit of the doubt and say its my local env that causing issues with Sorbet but in a fresh `rails new test_app --api` project, I'd expect `srb init` to work without errors... maybe I need to give it another go, curious on your thoughts above tho! :)
I wrote up an FAQ about the state of Ruby 3 and RBS here:
https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-will...
The tl;dr is that RBI files (not RBS files) will probably always be the preferred way to declare types for third party code (because it will always support exactly the same set of features that Sorbet does). We have some people in the community look into teaching Sorbet to read the RBS format, but the existing parsers for RBS files are written in Ruby and are very slow, and there are some ambiguities in the spec that make writing a third party parser that compiles to native code tricky. You can see an attempt to write a fast RBS parser in C++ here[1], but again given that RBI files do everything we need them to right now and we have other features people are asking us for, we haven't prioritized RBS support incredibly highly.
Sorbet works completely fine without RBS files!
[1] https://github.com/Shopify/rbs_parser
https://github.com/sorbet/sorbet/pull/5533
https://sorbet.org/slack
Our general approach to metaprogramming at the moment has been two-fold:
- Use ahead-of-time code generation powered either by runtime reflection or ad-hoc static analysis to generate RBI files declaring things that have been metaprogrammed. - Build type system features, errors, and autocorrects that encourage people to structure their code in ways that doesn't require metaprogramming to solve.
Metaprogramming is definitely still a sticking point, but the existing solutions work ~okay and the rest of the upside Sorbet provides make it worthwile to power through.
Next challenges:
- Make it faster. While the post was talking about how fast it is, it wasn't telling the whole truth. Turns out some type checking operations in a 15 million line codebase are still slow, and we're working on making those faster.
- Add more IDE features. At the beginning of this year I put a lot of work into making Sorbet's parser more tolerant of syntax errors, which helps things like autocompletion work better. We also want to make more code actions, autocorrects, and refactoring tools, to bring Ruby in line with what you'd expect from other typed languages in the IDE experience
- Add more type system features. Shapes and tuples are a huge unimplemented feature still, and people ask about it all the time. There are a handful of other type system features (happy to list them if you're curious) that would also let people write idiomatic Ruby and still have good typing.
Lots left to do!
Couple of questions:
> The declare_method call above acted like a decorator on the def call method: it would check that the msg argument given to call was a String and that call returned a String on every invocation
How was this implemented? Dynamically altering Object#send or something along those lines?
How is the story for sorbet and vim/nvim?
Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)?
The mechanism in the post is the same mechanism in use today with Sorbet signatures.
To get the `sig` method in scope, you have to put `extend T::Sig` in that class (or one of its parents). When `sig` is called for the first time in a class, it monkey patches that class to install some overrides of the method_added method. Ruby calls this method_added method for you every time it creates a method (from any means, static or dynamic). Code is here:
https://github.com/sorbet/sorbet/blob/master/gems/sorbet-run...
> How is the story for sorbet and vim/nvim?
Great, honestly better than VS Code for everything except autocompletion. I personally use Sorbet with Neovim.
> Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)?
Not that I know of unfortunately, but I also pay more attention to the type checker and it’s bugs than the tooling people build around it to get real world work done
You're right that Sorbet does not yet catch these bugs yet, and we'll likely get around to it in the future. It's being tracked in this issue[1].
We haven't quite prioritized this yet partly because it doesn't actually prevent you from using Sorbet in a Ruby 3 codebase, it just won't report all the errors it could (e.g., Sorbet allows using `T.untyped`, which can also cause runtime errors to go unreported).
Now that you've jogged my memory, there used to be some weirdness in our internal representation for method calls that made implementing this feature tricky. But we've since refactored some internal data structures and now it's probably a lot easier. Maybe I should look into how much work this would be again ...
[1]: https://sorbet.run/talks/StrangeLoop2018/#/28
Is that supposed to be how it works?
But there's tooling (first-party and third-party) that will either download or generate RBI files defining constants that come from gems. `srb init` is the first party solution, and Shopify's `tapioca` gem is the most popular third-party solution[1].
Unfortunately, because Ruby doesn't have import statements at the top of every file, Sorbet can't just do something like silently treat unknown imports as not having a type (like TypeScript and Flow can do), because then it would never be able to tell between "exists but unknown" vs "typo; does not exist" for constant definitions. This definitely makes the adoption process a little tricker compared to other languages, but it's generally a one-time thing once you've got the tooling set up.
Also if you're ever having trouble getting the tooling to work, there's a lot of people chatting about Sorbet daily at https://sorbet.org/slack
[1] https://github.com/Shopify/tapioca
Also, it's fast! I'm in total agreement with the point made in the article. That makes a huge difference in developer UX.
I started thinking about this a bit and I came up with the conclusion that the single biggest difference between structs and shapes is really iterating over keys. I spent some time trying to create structs by which you could iterate over all the keys and all the solutions seemed clunky or inelegant.
Solargraph combines inference and insight from YARD docs (standard for many gems, plus Castwide has written more YARD for the standard library) to make some pretty good guesses. Crucially it has plugins that add the insights from popular gems with static analysis (e.g. reek, rubocop). I maintain solargraph-rails, which parses your Ruby to make guesses about (surprise) Rails.
The typeprof gem can help IDE plugins make typing guesses based on your tests. This project is interesting to me because it's going into Ruby 3.1 so I think it reflects awareness from the core ruby team that many programmers are not ready to add types to their code.
solargraph: https://github.com/castwide/solargraph solargraph-rails: https://github.com/iftheshoefritz/solargraph-rails typeprof: https://www.youtube.com/watch?v=UTMj51j9yEg
The alpha releases are also a big concern. We are stuck on a 300 commit (release) old build and can never upgrade safely.
We have also never been able to get the VSCode extensions to run.
Thanks for Sorbet, but I’d suggest people outside Stripe to look elsewhere.
Still, if it helps it helps.
[1] https://crystal-lang.org/reference/1.3/tutorials/basics/60_m...
Edit: Perhaps I spoke too soon https://blog.appsignal.com/2021/01/27/rbs-the-new-ruby-3-typ...
At least the JS crowd had the decency to buy into a whole new (far better) language instead of a bolt-on solution.
In my experience, large codebases of those types of languages have a lot of "magic" thing happen. There's a lot of implicit stuff that one has to guess or spend time "following the code" to understand what it is doing.
And I say this after having built a major lending platform from scratch in Ruby, including a major Machine Learning scoring system in Python, having to maintain with a good sized payment system in pure JavaScript, and nowadays dealing with a major trading/liquidity system in Ruby.
They are fun languages, but once the code and systems start to scale, static typing really helps. For that reason I've seen a lot of these endeavours try to move to TypeScript or other typed languages.
Though Java still has some great strengths, especially the 8+ functional programming features and the concurrency library is great. If I could use Rails with Java it might be a different story though, since I hate Spring.
What I'm realising now is that Java has (had?) a really clunky type system and that there are other languages that do types better, so I shouldn't use Java as my reason for avoiding them.
That's because the theory of gradual type systems was only worked out in the '00s. Before that, you could have a static or dynamic type system, not anything in between. Common Lisp did have type annotations, but they were hints for optimization, without any guarantees. They were also local to subroutines only. Dylan[3] is an example of an early implementation of the idea, but Dylan was several years late and, without being able to compete with Java, died without ever being widely used.
The proper theory was first established by J. Siek[1] and W. Taha in 2006. It's distinct from nominal static typing which uses a single top type (like Object in Java) or generics, and obviously it's different from both purely static and dynamic typing. It took almost a decade for the idea to start gaining practical implementations - I think the original was a made for Scheme, and one of the first implementations was Typed Scheme for PLT Scheme, which continues on as Typed Racket[2] today. Typed Racket is unique in that it enforces the types even on the untyped side, by wrapping values and exports in contracts.
The idea proved to be useful in practice, and started being adopted in various (non-Scheme) dynamically typed languages, starting with TypeScript for JS and Hack for PHP. On the other hand, some statically typed languages also became gradually typed, most notably C#. The implementations continued to improve, shrinking the parts of their respective languages that could not be statically typed. In dynamic languages there are still features that cannot be practically expressed in static type systems - most metaprogramming and code generation falls into this category - but they are generally "good enough" for day to day coding.
Gradual typing is useful in the same way static type systems are useful: it can prevent certain kinds of errors by marking known-invalid expressions without the need to run the code (so, for example, can help you find errors even in code that's not covered by tests); it helps in writing tooling for the language (eg. go to definition, find references); it helps make the code clearer for the reader (no need to break into a debugger to see what kind of value a given identifier refers to); in some implementations it may also help in optimizing the runtime performance, but that's rare. The "gradual" aspect makes it easier to adopt when the codebase grows larger - the bigger the codebase, the more useful static types are, but by the time the codebase grows large enough to justify static typing it's too big to rewrite in a different, statically typed language.
In short: writing small projects or prototypes in a dynamically typed language is faster while maintenance and expansion of large projects is easier in statically typed one. Gradual typing lets you go from one to the other without a huge cost of a full rewrite.
[1] https://wphomes.soic.indiana.edu/jsiek/what-is-gradual-typin...
[2] https://docs.racket-lang.org/ts-reference/index.html
[3] https://opendylan.org/index.html
Actually, both metaprogramming and code generation blur the phase distinction between compile and run time - but this makes them a great fit for dependent typing systems, which do pretty much the same thing. So there's no reason why more refined static systems could not express these idioms.
see for SBCL:
http://www.sbcl.org/manual/index.html#Handling-of-Types
Types are useful when they don't get in the way. As are method contracts.