Being new to python, the astral stuff is such a relief.
I don’t think experienced python folks realize how much the flexible tooling slows people down, and creates friction for adopters. Setting up my project I tried 3 environment managers, 2 type checkers, 3 formatters/linters, 3 packagers/dependancy/project managers.
I know this is kinda the n+1 issue where astral is just adding another. But it feels like a complete and well designed stack, not a box of parts encouraging me to build my own. Reminds me a bit of go tooling.
I am a convert. I’ll happily jump on their type checker and packager when ready.
>Setting up my project I tried 3 environment managers, 2 type checkers, 3 formatters/linters, 3 packagers/dependancy/project managers.
I've been using Python for right about 20 years now. I've been through various prior iterations of the "standard" packaging tools. But I've never found myself wanting a type checker, formatter or linter; and aside from a brief period with Poetry (in which I used barely any of its functionality), I've had no interest in dependency management tools, either. What I have found myself wanting is fixes for the problems in Pip.
Different people work on different kinds of projects, and approach the language differently, and have radically different needs as a result.
Do you still write type hints, but not check them? Or just use dynamic typing as in Python 2?
Actually, I guess the first case is really type-hints-as-comments, so isn't substantially different from the second.
> I've been using Python for right about 20 years now
Either way, I've found it quite common among long-time Python users to stick with Python's traditional dynamic typing, whereas newcomers to the language tend to really dislike it in favour of strict static typing. (One exception is GvR himself, who seems to be very pro-type-checking.)
As the second group continue to add increasingly complex typing features to the language, I wonder if Python is becoming less desirable for those in the first group.
Working on a team in a larger python application, type checkers and linters are such a time saver. It's so nice not to think about how I'd like to format my code any longer. We've made some tweaks to our ruff rules to suit the team's opinion, and now even when I don't _love_ the way it formats a particular part of code, I just move on and get something productive done instead.
And type checking is so great for both preventing bugs (which it does all the time) and self-documenting code. Can't recommend them enough.
I’ve been using it for around the same time, and never cared much about formatters. Linting is useful but wouldn’t call it essential. But type checking is a non negotiable on any project I lead. It’s not perfect by any means, but beats having to crawl my way through a call stack trying to figure out what the hell a function is expected to take and why it’s getting a None instead.
I have yet to find a single drawback of adopting mypy (or similars) that isn’t completely eclipsed by the benefits.
> I've been using Python for right about 20 years now.
That's actually the biggest issue I've seen when a bit back joined a python-shop. Everyone there had mostly done python their whole careers. Fell in love with language during studies, then applied for jobs using that language. Stuck in the old ways, perhaps python felt nimble and nice 20 years ago compared to other things at the time, but with little progress (from its users) in using modern tooling it felt so arcane coming to this shop. And it wasn't the job's fault, it was a state of the art python stack, just a decade behind what other languages have.
> I know this is kinda the n+1 issue where astral is just adding another.
My bugbear with that XKCD is that even if it's true, it's also how progress is made. By this point I'm so annoyed whenever anyone links it that I kinda just wish Randall would take it down.
I used to use any standard editor but these days I use PyCharm or Visual Studio (not Code) otherwise, I just use Sublime Text or any adjacent reasonable editor.
Elementary OS has the nicest editor that other Distros have not snatched up yet not sure why. Its fast, and fits the bill where KATE and Sublime Text are for me.
Yeah but a global environment is a really bad idea. It’s going to trip up those beginners when second project that requires a different version of pytorch.
People just need a mentor. Could you imagine getting into plumbing and just trying to work out what kind of tools you need by looking at the pipes? Of course not. You'd learn from someone else.
I primarily work on typescript projects, little bit of go. Never enjoyed working with python until I found uv. Glad to see folks rally around astral's tools.
I'm a little sad that Ruff took off as a re-implementation of a whole bunch of work that was already done, rather than as a project to improve the work that was already done.
It was nice to be able to write little extra linters or flake8 plugins in the language I was linting. Plus decades of effort had gone into making those plugins pretty great, finding the right separation of linting codes so that the right things could be linted/ignored per codebase.
I understand why they did it, and they have done it remarkably well, but "rewrite it from scratch" is almost never the best answer and part of me wonders if this result could have been even better achieved another way.
If something can be rewritten from scratch fairly quickly and ends up being much faster, it makes you wonder what we (the general “we”) might have done “wrong” with the “decades of effort” before.
To be clear: there was nothing wrong with the old toolchains (hence the quotes). And of course, Ruff/uv can do what they do because they build on all that earlier knowledge. My point is just that sometimes it's easier to start over than to fix the old, and that fact itself feels kind of sad to me.
> If something can be rewritten from scratch fairly quickly and ends up being much faster, it makes you wonder what we (the general “we”) might have done “wrong” with the “decades of effort” before.
I think the impact of Rust (and Go) shouldn't be underestimated here. Prior to these languages if you wanted fast runtime you were either using C or C++ with all their associated footguns, tricky build systems, and poor support for exposing and consuming libraries. Or you were using something like Java or C# which meant managing another runtime just for tools, which is especially bad for something like UV which manages runtimes - imagine if you were using a python tool to manage your Java versions and a Java tool to manage you Python versions, it would be a mess!
With both Go and Rust you have fast languages without memory safety issues that compile to a single, easily-installable binary and have a broad ecosystem of libraries available that mean that half the time you don't even need to write any difficult code and can just glue some modules together, and the other half of the time you only need to write the code you actually want to write and not all the support code for general pupose stuff like reading config files, file watching, etc.
> there was nothing wrong with the old toolchains (hence the quotes). And of course, Ruff/uv can do what they do because they build on all that earlier knowledge
I don't think that's true. I think the old toolchains really were bad, partly because the Python community was uniquely resistant to building on what worked in other ecosystems, and uv in particular largely succeeds because it's the first tool in this space to finally just ignore the nonsense that had somehow become accepted wisdom in Python-land and apply a little bit of outside experience.
I am quite grateful that they rewrote so many tools that I had been using. Upgrades were super painful with them split into a dozen separate packages, I routinely ran into incompatibilities and had to pin to specific versions to deal with transitive dependencies.
Given the state of python packaging and tooling, I'd say that consolidating the tooling is a big win in and of itself, and with the tremendous speed wins on top...
Whenever Ruff comes up, I reflexively go and check https://github.com/astral-sh/ruff/issues/970, since Pylint is the one tool whose performance I find to be egregiously bad. Looks like Ruff is still very far from being able to replace it.
Rewrite from scratch is exactly what the Python ecosystem needs more of.
The strong push to always contribute to the existing projects rather than starting new ones in the Python community is what caused the previous subpar state of tooling. I think Astral is also picking a good middle-ground where they still aim to build as much as possible on existing Python standards (and help contributing to new ones), so it's still building on what has been done in the past.
There was probably a lot of engineering that went into those designs and that took time.
Having a clean process to rewrite from is what made it so fast. They knew exactly what outcome they wanted and they had access to the internal implementations on how to do it.
Yeah it's somewhat ideological, I think open source software is better for society when built as a community project than built and controlled by a VC funded company.
I don't think you could get flake8 to be as fast as Ruff, but I think you could improve it to be within an order of magnitude, and honestly that's plenty good enough. There are a lot of low hanging fruit, particularly around caching. I'd push back on there being no chance of being "as good as" Ruff, because "good" is not just speed. Ruff is _not complete_, if you want all the features and linters, you still need Flake8, or you have to sacrifice some. It's also not publicly extensible, and not written in Python, both of which are fair choices but I think deviate from the ideal scenario here.
Blame Guido. Until recently when he was bought out by Microsoft, he had been the primary blocker of higher Python performance. There were a bunch of attempts at adding a JIT to Python but Guido was more interested in splitting hairs over syntax than any real heavy lifting. Python could have been as performant as LuaJIT or V8 but their dictator ruined it. Python need more Mike Palls.
I'm very impressed by the recent developer experience improvements in the python ecosystem. Between ruff, uv, and https://github.com/gauge-sh/tach we'll be able to keep our django monolith going for a long time.
Any opinions about the current state of the art type checker?
I'm very happy with pyright. Most bug reports are fixed within a week and new peps/features are added very rapidly usually before pep is accepted (under experimental flag). Enough that I ended up dropping pylint and consider pyright enough for lint purposes as well. The most valuable lints for my work require good multi-file/semantic analysis and pylint had various false positives.
Main tradeoff is this only works if your codebase/core dependencies are typed. For a while that was not true and we used pylint + pyright. Eventually most of our code is typed and we added type stubs for our main untyped dependencies.
edit: Also on pylint, it did work well mostly. tensorflow was main library that created most false positives. Other thing I found awkward was occasionally pylint produces non-deterministic lints on my codebase.
Thanks for linking this. I wasn't surprised Microsoft made their AI auto completion proprietary (they did similar for C# in VSCode). But it grated me that semantic highlighting wasn't open source.
I have always used MyPy but I have noticed some large open source projects adopt Pyright as of late. The folks behind ruff and uv are currently working on a type checker as well but haven't heard when they plan on releasing it.
First thanks for mentioning tach, I wished this tool existed for a long time and I'm happy to give in a try in the following days!
For typechecker, I also vouch for Pyright which is what we use for all our django backends at work.
Just be aware that you will have hard time to typecheck part of your code where you rely heavily on django's magic (magic strings, auto-generated properties/methods, etc...).
In these cases, it's sometimes better to avoid these features entirely or accept that some part of your code will not be typechecked.
Python development tools produced by Astral (https://github.com/astral-sh), primarily uv and ruff. GP is a complaint about how often people submit links to these projects. The resulting discussion tends not to produce any new insight and the most common points IMX are "it's fast because it's written in Rust" (which is fairly specious logic - most problems with existing native Python tools are due to algorithmic problems such as poorly designed caching) or "it avoids bootstrapping problems because it's not written in Python" (it's completely possible to avoid those problems for everything except actually provisioning the Python interpreter, which some people apparently do see as a killer feature).
I first attempted to use ruff for a small project ~2 years ago, and at the time felt that it wasn't quite good enough to replace the: black+isort+whatever linter combo we were using at work.
I've used it a few times since then and now I'm a big proponent of using only ruff. I think most of its value comes from:
1. Being fast (or at least fast enough that it's not annoying).
2. Replaces the linting/formatting combo of multiple tools, reducing the cognitive load for the developer.
I've been using an amalgamation of pyenv, pip-tools, black, isort, etc. for projects and just gave uv and ruff a try. Wow, it really is fast! Skeptical of anything VC-backed but I'll be keeping my eye on it.
There is no "coupling" inherent to native Python tools. They can generally be installed in a separate virtual environment from the one they operate on.
For example, with Pip you simply pass the `--python` option to have it install for a different environment - you can create your venvs `--without-pip`, and share a copy of Pip across all of them. If you use Pipx, you can expose and use Pipx's vendored Pip for this, as I described in a recent blog post (https://zahlman.github.io/posts/2025/01/07/python-packaging-...).
Twine and Build don't care about your project's environment at all - Twine uploads a specified file, and Build by default creates a new environment for the build (and even withough build isolation, it only requires your build environment to contain the build backend, and the build environment doesn't have to be the one you use for the rest of development). Setuptools, similarly, either gets installed by the build frontend into an isolated build environment, or can reside in a separate dedicated build environment. It doesn't actually operate on your project's environment - it only operates on your project's source tree.
Even twine decoupled itself from my python toolchain some time ago [1], through some dependency. Cannot install it unless you are on a system trusting trust in rust™.
Readit News
I don’t think experienced python folks realize how much the flexible tooling slows people down, and creates friction for adopters. Setting up my project I tried 3 environment managers, 2 type checkers, 3 formatters/linters, 3 packagers/dependancy/project managers.
I know this is kinda the n+1 issue where astral is just adding another. But it feels like a complete and well designed stack, not a box of parts encouraging me to build my own. Reminds me a bit of go tooling.
I am a convert. I’ll happily jump on their type checker and packager when ready.
I've been using Python for right about 20 years now. I've been through various prior iterations of the "standard" packaging tools. But I've never found myself wanting a type checker, formatter or linter; and aside from a brief period with Poetry (in which I used barely any of its functionality), I've had no interest in dependency management tools, either. What I have found myself wanting is fixes for the problems in Pip.
Different people work on different kinds of projects, and approach the language differently, and have radically different needs as a result.
Do you still write type hints, but not check them? Or just use dynamic typing as in Python 2?
Actually, I guess the first case is really type-hints-as-comments, so isn't substantially different from the second.
> I've been using Python for right about 20 years now
Either way, I've found it quite common among long-time Python users to stick with Python's traditional dynamic typing, whereas newcomers to the language tend to really dislike it in favour of strict static typing. (One exception is GvR himself, who seems to be very pro-type-checking.)
As the second group continue to add increasingly complex typing features to the language, I wonder if Python is becoming less desirable for those in the first group.
And type checking is so great for both preventing bugs (which it does all the time) and self-documenting code. Can't recommend them enough.
I have yet to find a single drawback of adopting mypy (or similars) that isn’t completely eclipsed by the benefits.
That's actually the biggest issue I've seen when a bit back joined a python-shop. Everyone there had mostly done python their whole careers. Fell in love with language during studies, then applied for jobs using that language. Stuck in the old ways, perhaps python felt nimble and nice 20 years ago compared to other things at the time, but with little progress (from its users) in using modern tooling it felt so arcane coming to this shop. And it wasn't the job's fault, it was a state of the art python stack, just a decade behind what other languages have.
I guess that makes one of us.
https://en.wikipedia.org/wiki/The_Fox_and_the_Grapes
My bugbear with that XKCD is that even if it's true, it's also how progress is made. By this point I'm so annoyed whenever anyone links it that I kinda just wish Randall would take it down.
Introducing n+1 solution/ standard may be the necessary cost of making progress, but it is possible to incur that cost without gaining anything
(at the extreme, trolling is fun activity with net negative on bigger scale)
Elementary OS has the nicest editor that other Distros have not snatched up yet not sure why. Its fast, and fits the bill where KATE and Sublime Text are for me.
The point is, if you are learning to code, you can skip nearly all the ecosystem tools. Their need will arise when the time comes.
I moved to poetry for things a bit ago until I found out about uv recently, it’s so fast I thought it didn’t even do what it asked a couple times.
It was nice to be able to write little extra linters or flake8 plugins in the language I was linting. Plus decades of effort had gone into making those plugins pretty great, finding the right separation of linting codes so that the right things could be linted/ignored per codebase.
I understand why they did it, and they have done it remarkably well, but "rewrite it from scratch" is almost never the best answer and part of me wonders if this result could have been even better achieved another way.
If something can be rewritten from scratch fairly quickly and ends up being much faster, it makes you wonder what we (the general “we”) might have done “wrong” with the “decades of effort” before.
To be clear: there was nothing wrong with the old toolchains (hence the quotes). And of course, Ruff/uv can do what they do because they build on all that earlier knowledge. My point is just that sometimes it's easier to start over than to fix the old, and that fact itself feels kind of sad to me.
I think the impact of Rust (and Go) shouldn't be underestimated here. Prior to these languages if you wanted fast runtime you were either using C or C++ with all their associated footguns, tricky build systems, and poor support for exposing and consuming libraries. Or you were using something like Java or C# which meant managing another runtime just for tools, which is especially bad for something like UV which manages runtimes - imagine if you were using a python tool to manage your Java versions and a Java tool to manage you Python versions, it would be a mess!
With both Go and Rust you have fast languages without memory safety issues that compile to a single, easily-installable binary and have a broad ecosystem of libraries available that mean that half the time you don't even need to write any difficult code and can just glue some modules together, and the other half of the time you only need to write the code you actually want to write and not all the support code for general pupose stuff like reading config files, file watching, etc.
I don't think that's true. I think the old toolchains really were bad, partly because the Python community was uniquely resistant to building on what worked in other ecosystems, and uv in particular largely succeeds because it's the first tool in this space to finally just ignore the nonsense that had somehow become accepted wisdom in Python-land and apply a little bit of outside experience.
Sad, but also liberating. There are surely more projects out there that would benefit from breaking with conventional wisdom to start over.
Given the state of python packaging and tooling, I'd say that consolidating the tooling is a big win in and of itself, and with the tremendous speed wins on top...
(Pyright isn't written in Rust, but adequate performance is still a design goal)
The strong push to always contribute to the existing projects rather than starting new ones in the Python community is what caused the previous subpar state of tooling. I think Astral is also picking a good middle-ground where they still aim to build as much as possible on existing Python standards (and help contributing to new ones), so it's still building on what has been done in the past.
Having a clean process to rewrite from is what made it so fast. They knew exactly what outcome they wanted and they had access to the internal implementations on how to do it.
All that effort was not wasted at all.
Don't you have "decades of effort" to resolve this wonder?
I don't think you could get flake8 to be as fast as Ruff, but I think you could improve it to be within an order of magnitude, and honestly that's plenty good enough. There are a lot of low hanging fruit, particularly around caching. I'd push back on there being no chance of being "as good as" Ruff, because "good" is not just speed. Ruff is _not complete_, if you want all the features and linters, you still need Flake8, or you have to sacrifice some. It's also not publicly extensible, and not written in Python, both of which are fair choices but I think deviate from the ideal scenario here.
Any opinions about the current state of the art type checker?
Main tradeoff is this only works if your codebase/core dependencies are typed. For a while that was not true and we used pylint + pyright. Eventually most of our code is typed and we added type stubs for our main untyped dependencies.
edit: Also on pylint, it did work well mostly. tensorflow was main library that created most false positives. Other thing I found awkward was occasionally pylint produces non-deterministic lints on my codebase.
Besides re-adding features that Microsoft makes exclusive to pylance, it tweaks a number of features that IMO makes pyright work better out the box:
https://docs.basedpyright.com/latest/benefits-over-pyright/p...
For typechecker, I also vouch for Pyright which is what we use for all our django backends at work.
Just be aware that you will have hard time to typecheck part of your code where you rely heavily on django's magic (magic strings, auto-generated properties/methods, etc...).
In these cases, it's sometimes better to avoid these features entirely or accept that some part of your code will not be typechecked.
I've used ruff/uv for sure, but I never pay attention to Astral who is behind it.
I've used it a few times since then and now I'm a big proponent of using only ruff. I think most of its value comes from:
1. Being fast (or at least fast enough that it's not annoying). 2. Replaces the linting/formatting combo of multiple tools, reducing the cognitive load for the developer.
Anyway, big fan.
For example, with Pip you simply pass the `--python` option to have it install for a different environment - you can create your venvs `--without-pip`, and share a copy of Pip across all of them. If you use Pipx, you can expose and use Pipx's vendored Pip for this, as I described in a recent blog post (https://zahlman.github.io/posts/2025/01/07/python-packaging-...).
Twine and Build don't care about your project's environment at all - Twine uploads a specified file, and Build by default creates a new environment for the build (and even withough build isolation, it only requires your build environment to contain the build backend, and the build environment doesn't have to be the one you use for the rest of development). Setuptools, similarly, either gets installed by the build frontend into an isolated build environment, or can reside in a separate dedicated build environment. It doesn't actually operate on your project's environment - it only operates on your project's source tree.
[1] https://github.com/pypa/twine/issues/1015