Very excited for this, looks like it's built on top of pretty solid technical foundations (iirc, it uses salsa from rust for incremental type checking). I've found mypy pretty terrible. Pyright is okay (but requires node).
When I tried pyright I immediately bailed on the requirement for Node, which seems absolutely bananas. Mypy is at least written in its own dogfood of python.
There is third, pytype (by google), which I found pretty good but it rarely gets mentioned. However, like the others it is slow, so I hope this one is fast and supports all the pytype features (especially being able to type-check un-annotated code).
I've avoided Pyright explicitly for that reason. I have a severe dislike of Node, and don't want it installed on my computer for any reason. I'm aware that this is a self-limiting position.
Anyway, agreed that this is very exciting news. Poetry was great, and then I found uv. It's... wow. It's really good.
Mypy is currently the only one that can handle Django projects (because it has a plugin system and there's a plugin that can import your django code to infer types)
Unfortunately that massively limits its performance too since loading a large django codebase is pretty slow.
I've found pyright to be limited in its ability to infer types, especially compared to Pycharm, which can resolve class inherence across multiple modules quite well. I noticed this when trying to use Zed instead of Pycharm.
If you actually type your code [1], pyright is miles ahead pycharm in every case I've seen. For partially/incorrectly typed code pycharm is often better in practice because it relies on heuristics. (tbf. Some libraries lack or have bad stubs, which can be annoying using pyright (eg. Pandas)).
[1] You don't have to typehint everything for this to work. Pyright infers return types just fine for instance
I started learning Python when it was barely at version 1 precisely because there was no real static type checking. If I wanted that, I would just write the program in C or its variants. I wasn't excited about type hints because developers became super judgy if there weren't any. I just want to know why you're using Python if you want stronger typing and feel the need to change the entire language instead of using another one yourself.
Progressive typing gives you the option to type check when and if you want or need it. You don't _have_ to type. If things get really hairy but you know it's right, you can focus on the code instead of the typing and efficiently complete your task.
There's also something fundamental about taking a dynamic language like Python or JavaScript and adding typing versus taking a static language and adding dynamic typing (e.g.`auto` in C#). The dynamic language allows modifications that are _really_ convenient if not a little hacky that you just can't easily express without big refactors in static codebases. Things like tagging objects with properties, making quick anonymous structs (before that became a thing in modern languages) so you can return tuples of values, other constructs like dynamic functions or whatever. Progressive typing is just so much more expressive
Even "dynamic" isn't a true dynamic in C#, it ends up invoking methods via reflection, and using it for anything other than foreign COM objects is a massive code smell.
Probably because a lot of programmers don't get to choose the language they are using. They have a job developing some existing app that is already written in Python, but they wish it wasn't Python. They don't have an option to rewrite the whole codebase in Rust or whatever, so the best they can do is advocate for Python to evolve to be more like languages they do like.
But most people in our profession get to choose their job, don't they? Why would a Python project hire a rust programmer? Why would someone who hates Python work on a Python project? I feel the same way. Sporadic use of type hints is nice, and I use it sometimes for reference, like a very succinct comment. But if I want a typed language, Go, Dart, C or even Java are right there.
I think this is exactly it. My gut instinct is just "Why the obsession with adding static types to Python, just use a language that already has good static types".
As you say though, not everybody gets to choose their language, and I certainly wouldn't choose Python if I could possibly help it, so if we can add good static types to Python, it's still not a great language, but it's much improved by static types.
Man, I feel the exact opposite. Working without types, even if it's just type hints, feels like I'm stumbling around in the dark. I want to know what stuff is, not just YOLO my way through everything.
Looking at my Python before I began using type hints and data structures such as dataclasses, typevars, etc I was still type-checking a function's argument, but it was very manual and messy. Nowadays, my code is so much cleaner, concise, less error-prone, and much easier to revisit after a long period away. IMO there isn't a single downside to leaning heavily into types, even if they're sorta "fake" like they are in Python.
I personally just love Python type hinting for testing purposes. So I can avoid types while prototyping or playing with architecture, but once I have something I'm happy with, I find that adding types allows me to gain confidence in the program while having fewer tests, and generally making it easier to refactor further.
And full typing is also a great first step for rewriting in another language.
Python's got a pretty incredible library ecosystem, and it's easy to get a project up and running quickly. The big problems with it start to appear as you cross the 10k+ sloc / 3+ dev thresholds; if things are changing quickly, the iteration velocity the duck typing provides can quickly become a big problem.
Type hints fix this by allowing you to ensure your contracts are upheld while your team continues to move. If you don't find yourself needing them, you probably don't have a team that's trying to ship and iterate super quickly, or your project is still at the one-off script scale.
C is basically untyped but makes you type out the types. This is not what modern type checking is; it’s usually the other way around - you specify types where you want them checked because it gives you confidence that the program is correct.
The entire ML/AI ecosystem is in Python for the most part these days, so if anyone wants to get involved it means needing to touch Python to some degree. So, many developers might get handed projects that require working in Python, but would really prefer not to give up all the nice things that come from a modern statically typed language.
Personally, I'm one of those people that got converted over to statically typed languages, as I spent a large part of my early career in startups using Ruby. I don't think I could ever go back after using Go, TypeScript, and Rust over the past decade or so.
I guess there's something about Python that just makes it really easy to read and write for me. I haven't found another language that fits the vibe. From my experience, the resentment towards stronger typing comes from developers misusing it.
I used to prefer dynamically typed languages for most tasks, and saw attempts at statically typing everything as some kind of [compulsion][1].
At work, we were writing a bunch of tests and tooling in python, and the team lead at the time insisted that we type things and lint with mypy strict as if it were a compiler.
I said to my teammate "this is so stupid, if you want to know the types of the parameters, you read the documentation comment right there in the definition. If you want to know more, you read the test. If you want to know even more, you read the source."
My teammate replied, "YOU will write a function with good documentation, thorough tests, and a comprehensible definition, but as an organization grows, a shrinking percentage of code will be written that way. These automated checks prevent the worst from coders who care less than you do."
Moving from Java to Python for the preferred backend language many many years ago was a relief: building prototypes faster, no need to mention obvious types (int, bool, string) anywhere. In some places using (data) classes and I had robust programs. When doing type mistakes, running the program or having unit tests exactly told me where I had it wrong - no need for type annotations to enjoy programming, right?
But now over the last years, I happen to prefer changing programs a lot without running them a lot. I often do code refactorings over multiple commits, being highly concentrated, without running the code once. And then only running the code only 1 hour or 2 later. Such a pleasant activity! Just writing, reading, writing, thinking code. No ugly and long stack traces, no need to input user data into input fields, just reading and changing the program code, with the editor in full screen.
And guess what I highly started to appreciate, after many years of Python! Type annotations! With type annotations a LSP would immediately underline any mistake I would accidentally make, and that made me feel much more comfortable and secure about my changes, over several commits, touching 100s of lines without ever running the program. I suddenly learned to understand, after many years of dynamic type less programming with Python, that I can get more easily and for much longer into a flow state when I add annotations and run a LSP live over my code while I work on it.
Yeap, type provides a really good baseline for a documentation. A good documentation is really needed when debugging dynamically typed language python.
Python’s typing is so different from C’s that there should be a different word for it. Typing in C is about storage. Move the stack pointer 8 bytes to make room for my integer. Move it another 8 to make room for a pointer to a character. When I deference it, write just one byte.
Python typing is about machine checkable properties of your program. Consider a function foo that takes a dictionary keyed by uuids, reads from it, and returns some kind of information. We can check all sorts of properties without ever running the code. Accidentally construct a dictionary keyed by strings instead of uuids? Rejected, wrong argument type. Accidentally modify the dictionary in the body of foo()? Rejected, if you wrote Mapping[UUID, Any] for the argument type. Mapping alone does not allow you to assume mutability. So many of the errors I make in Python, I can declare my intention not to make, and then check to see if I’ve committed them.
> because developers became super judgy if there weren't any
Who? People on your team or randoms complaining about your open source project? If it's the latter, you can tell them to get bent or write their own type stubs if it bothers them. Type hints are fully optional in Python.
Sometimes you need to write a script that is too large for a reasonable bash script but too small to make a full program. In those instances, I tend to reach for Python because of its massive standard library. I often don't need any additional dependencies. But when I use python I absolutely want types. It makes IDE-features like auto-completion so significantly more effective when your editor actually knows what type every variable is. And of course adding types provides the obvious benefits to code clarity, and the additional error-checking that type checking is.
Also its super hard to find non-crypto-currency rust jobs, but python jobs are pretty common.
There's a lot of value in some typing. APIs defined with Pydantic types save you time. Refactoring is...not amazingly well supported, at least in my install of VSCode - perhaps PyCharm is better - but it does occasionally help.
> What I want to do is build software that vertically integrates with our open source tools, and sell that software to companies that are already using Ruff, uv, etc. Alternatives to things that companies already pay for today.
> An example of what this might look like (we may not do this, but it's helpful to have a concrete example of the strategy) would be something like an enterprise-focused private package registry. A lot of big companies use uv. We spend time talking to them. They all spend money on private package registries, and have issues with them. We could build a private registry that integrates well with uv, and sell it to those companies. [...]
I'm not sure that business model exists personally but good luck to them. It's trivial to spin up an index on AWS or Azure. Usually for a company it's hard for minions dealing with this stuff to get approval to spend money on a new supplier, but easy to set up something with your existing cloud provider.
the packaging one is an odd example because it’s hard to imagine a company that both needs an enterprise packaging solution external to their cloud provider, and is python only
i have a similar concern for the Pydantic folks commercializing via observability platform; how big is the market for python-only observability?
i think both cos will either look for money elsewhere, or find themselves having to deal with at least a few other language ecosystems. Even python+typescript would 10x the TAM or more
Honestly, they should do that, it would force their competitors to actually improve their own, and if they don't well, there you go, really great funding your way.
These questions are often hard to answer for people unfamiliar with how VC works, but mostly because VC is not intuitive from a 'normal business' perspective.
Essentially, VC's know very that what they're funding is risky and may go completely caput. However the VC model explicitly aims to fund companies that have the potential of becoming huge and becoming very profitable.
Both the founders and VC's behind Astral are not stupid, they know that it could fail or only have moderate success... However, if they succeed in building the next generation go-to ecosystem for python development (which is arguably the world's most popular programming language), we could imagine a future where they spin off some product and it does incredibly well!
It's not that that future's unimaginable. It's more that it would be nice to have an easy path to knowing how these tools will be funded. They will suck time and effort out of the ecosystem of Python tools, which is fine, except it's not fine if they vanish or decay in X years' time because of funding issues.
I don't know the solution, and I really think these tools look about as great as tools can look, so I'm very keen to use them. I would just like some way to know that they'll be supported for 10 years at least.
If they open source it, who cares? Obviously I would like them to continue maintaining their stuff, and I am sure they are imaginative enough to find some source of income.
All type checkers other than mypy (e.g. pyright, intellij) have ignored the level of plugin support necessary to make django work well, and so they are DOA for any large existing django codebase. Unless ruff decides to support such a dynamic interface as mypy's, it'll fare no better.
There was an effort to create a typechecking plugin interface for dataclass-style transforms for python type checkers, but what was merged was so lacking that one couldn't even make something close to django-stubs with it.
I am not part of the DSF or anything but I can say with a good amount of confidence that the answer to this is “No”.
Moving to a Pydantic-style model would be a whole different ballgame. Hell of a lot of work, and meanwhile we can’t even really get to “let’s have type hints in some of the Django infernals”.
But! I do think someone who wants to have fun could try to build a type hint based Model subclass as a third party lib! There’s a looooot of stuff to resolve in that world but for any proposal like that the general vibe is “prove the feasibility with a third party package first”.
Django's magic is that you can set various class variables as either a value or a method. So typing that becomes incredibly hard, everything is effectively Union typed.
Applaud the effort, but it's weird seeing python (and es6) living more and more on top of native static analyzers/transpilers .. at which point will they become a frontend syntax for ocaml or julia ? :p
as someone working in this space, the idea is that most python code is implicitly statically typed anyway, in that your functions are only ever meant to be called with specific types for their arguments, and only return objects of a specific type. all that static type analysis does is to make that explicit via type annotations, and then verify that you have indeed used types correctly in your code. static compilers then say "well, if we know the static types of everything in this function, we can compile it to more efficient code, skipping all the dynamic lookups and runtime type checks and dispatching". typically there is an escape hatch that will fall back to regular python behaviour if your code is indeed dynamic (e.g. reflecting on a database schema to generate models at runtime), but losing the benefits of static analysis and optimisation.
if you would like to read more on the topic the keyword is "gradual typing"; in particular you might be interested in looking up stanza, a language designed around gradual types from the outset rather than retrofitting them on top of a dynamically typed language.
> at which point will they become a frontend syntax for ocaml or julia ?
At no point.
What probably will happen is that both the JS and the Python reference implementations will have first class support for type systems which will allow them to generate optimized native code for the typed sections of your code while keeping the untyped sections as bytecode or as higher level compiled code (i.e. a bunch of runtime function calls dealing with tagged union objects).
Readit News
Ruff truly can become one tool to rule them all.
A fast Rust based type checker would be amazing!
https://github.com/google/pytype?tab=readme-ov-file#pytype--...
[1] https://github.com/beartype/beartype
This is true! We're also contributing salsa features upstream where we can, e.g. https://github.com/salsa-rs/salsa/pull/603
Anyway, agreed that this is very exciting news. Poetry was great, and then I found uv. It's... wow. It's really good.
Unfortunately that massively limits its performance too since loading a large django codebase is pretty slow.
[1] You don't have to typehint everything for this to work. Pyright infers return types just fine for instance
Here's the github issues filter linked in the screenshot:
https://github.com/astral-sh/ruff/labels/red-knot
And the best answer/description of what the type checker will be:
https://github.com/astral-sh/ruff/discussions/15149
Missed opportunity to call it typy in my opinion
There's also something fundamental about taking a dynamic language like Python or JavaScript and adding typing versus taking a static language and adding dynamic typing (e.g.`auto` in C#). The dynamic language allows modifications that are _really_ convenient if not a little hacky that you just can't easily express without big refactors in static codebases. Things like tagging objects with properties, making quick anonymous structs (before that became a thing in modern languages) so you can return tuples of values, other constructs like dynamic functions or whatever. Progressive typing is just so much more expressive
Assuming you meant C++ - no, auto is not a even a tiny bit dynamic. It's still fully static typing, but with type inference.
The compiler will still prevent you from calling a method which doesn't exist.
As you say though, not everybody gets to choose their language, and I certainly wouldn't choose Python if I could possibly help it, so if we can add good static types to Python, it's still not a great language, but it's much improved by static types.
Looking at my Python before I began using type hints and data structures such as dataclasses, typevars, etc I was still type-checking a function's argument, but it was very manual and messy. Nowadays, my code is so much cleaner, concise, less error-prone, and much easier to revisit after a long period away. IMO there isn't a single downside to leaning heavily into types, even if they're sorta "fake" like they are in Python.
And full typing is also a great first step for rewriting in another language.
Type hints fix this by allowing you to ensure your contracts are upheld while your team continues to move. If you don't find yourself needing them, you probably don't have a team that's trying to ship and iterate super quickly, or your project is still at the one-off script scale.
This is grossly incorrect.
Personally, I'm one of those people that got converted over to statically typed languages, as I spent a large part of my early career in startups using Ruby. I don't think I could ever go back after using Go, TypeScript, and Rust over the past decade or so.
you can obviously still use it untyped to hack together scripts, but I still find myself leaving annotations for clarity.
I've been using it since the mid 2.x days and find the style we were writing back then to be incredibly dated.
I used to prefer dynamically typed languages for most tasks, and saw attempts at statically typing everything as some kind of [compulsion][1].
At work, we were writing a bunch of tests and tooling in python, and the team lead at the time insisted that we type things and lint with mypy strict as if it were a compiler.
I said to my teammate "this is so stupid, if you want to know the types of the parameters, you read the documentation comment right there in the definition. If you want to know more, you read the test. If you want to know even more, you read the source."
My teammate replied, "YOU will write a function with good documentation, thorough tests, and a comprehensible definition, but as an organization grows, a shrinking percentage of code will be written that way. These automated checks prevent the worst from coders who care less than you do."
I had no retort, so there is that.
[1]: https://github.com/dgoffredo/stag?tab=readme-ov-file#why
But now over the last years, I happen to prefer changing programs a lot without running them a lot. I often do code refactorings over multiple commits, being highly concentrated, without running the code once. And then only running the code only 1 hour or 2 later. Such a pleasant activity! Just writing, reading, writing, thinking code. No ugly and long stack traces, no need to input user data into input fields, just reading and changing the program code, with the editor in full screen.
And guess what I highly started to appreciate, after many years of Python! Type annotations! With type annotations a LSP would immediately underline any mistake I would accidentally make, and that made me feel much more comfortable and secure about my changes, over several commits, touching 100s of lines without ever running the program. I suddenly learned to understand, after many years of dynamic type less programming with Python, that I can get more easily and for much longer into a flow state when I add annotations and run a LSP live over my code while I work on it.
Python typing is about machine checkable properties of your program. Consider a function foo that takes a dictionary keyed by uuids, reads from it, and returns some kind of information. We can check all sorts of properties without ever running the code. Accidentally construct a dictionary keyed by strings instead of uuids? Rejected, wrong argument type. Accidentally modify the dictionary in the body of foo()? Rejected, if you wrote Mapping[UUID, Any] for the argument type. Mapping alone does not allow you to assume mutability. So many of the errors I make in Python, I can declare my intention not to make, and then check to see if I’ve committed them.
Who? People on your team or randoms complaining about your open source project? If it's the latter, you can tell them to get bent or write their own type stubs if it bothers them. Type hints are fully optional in Python.
Also its super hard to find non-crypto-currency rust jobs, but python jobs are pretty common.
Ruff is amazing, uv literally fixed all the python packaging issues we have, and now a type checker!!
Can't wait to see it!
> What I want to do is build software that vertically integrates with our open source tools, and sell that software to companies that are already using Ruff, uv, etc. Alternatives to things that companies already pay for today.
> An example of what this might look like (we may not do this, but it's helpful to have a concrete example of the strategy) would be something like an enterprise-focused private package registry. A lot of big companies use uv. We spend time talking to them. They all spend money on private package registries, and have issues with them. We could build a private registry that integrates well with uv, and sell it to those companies. [...]
i have a similar concern for the Pydantic folks commercializing via observability platform; how big is the market for python-only observability?
i think both cos will either look for money elsewhere, or find themselves having to deal with at least a few other language ecosystems. Even python+typescript would 10x the TAM or more
Essentially, VC's know very that what they're funding is risky and may go completely caput. However the VC model explicitly aims to fund companies that have the potential of becoming huge and becoming very profitable.
Both the founders and VC's behind Astral are not stupid, they know that it could fail or only have moderate success... However, if they succeed in building the next generation go-to ecosystem for python development (which is arguably the world's most popular programming language), we could imagine a future where they spin off some product and it does incredibly well!
I don't know the solution, and I really think these tools look about as great as tools can look, so I'm very keen to use them. I would just like some way to know that they'll be supported for 10 years at least.
Django has a lot of magic that makes type checking more difficult than it should be.
I'm not really expecting anything, even just the speed bump will be nice.
All type checkers other than mypy (e.g. pyright, intellij) have ignored the level of plugin support necessary to make django work well, and so they are DOA for any large existing django codebase. Unless ruff decides to support such a dynamic interface as mypy's, it'll fare no better.
We use mypy with [django-stubs](https://github.com/typeddjango/django-stubs) which works ok nowadays.
There was an effort to create a typechecking plugin interface for dataclass-style transforms for python type checkers, but what was merged was so lacking that one couldn't even make something close to django-stubs with it.
Moving to a Pydantic-style model would be a whole different ballgame. Hell of a lot of work, and meanwhile we can’t even really get to “let’s have type hints in some of the Django infernals”.
But! I do think someone who wants to have fun could try to build a type hint based Model subclass as a third party lib! There’s a looooot of stuff to resolve in that world but for any proposal like that the general vibe is “prove the feasibility with a third party package first”.
Tangential, I'm waiting for Edgedb to become next SQL so all ORM moats like Django's are drained.
https://github.com/typeddjango/django-stubs
https://adsharma.github.io/fquery-meets-sqlmodel/
https://github.com/adsharma/fastapi-shopping/blob/main/model...
if you would like to read more on the topic the keyword is "gradual typing"; in particular you might be interested in looking up stanza, a language designed around gradual types from the outset rather than retrofitting them on top of a dynamically typed language.
what are places to discuss gradual typing these days ? I stopped looking a few years ago (after clojure and scheme started projects on that topic)
At no point.
What probably will happen is that both the JS and the Python reference implementations will have first class support for type systems which will allow them to generate optimized native code for the typed sections of your code while keeping the untyped sections as bytecode or as higher level compiled code (i.e. a bunch of runtime function calls dealing with tagged union objects).
Javascript has a uniquely privileged position as the native browser language, and a couple of decades of effort has been expended on making it usable.