I did not expect to wake up at 4am seeing my post on front page HN, but here we are nevertheless :D
As the intro mentioned, these started off as 14 small tweets I wrote a month prior to starting my blog. When I finally got that set up, I just thought, "hey, I just spent the better part of two weeks writing these nifty Python tricks, might as well reuse them as a fun first post!"
That's why the flow might seem a little weird (as some pointed out, proxy properties are not really a Python "feature" in of itself). They were just whatever I found cool that day. I tried to find something more esoteric if it was a Friday, and something useful if it was a Monday. I was also kinda improving the entire series as it was going on, so that was also a factor.
Same goes with the title. These were just 14 feature I found interesting while writing Python both professionally and as a hobby. Some people mentioned these are not very "advanced" per se, and fair enough. I think I spent a total of 5 second thinking of a title. Oh well!
The tricks are more advanced than what I've seen in most people's code at some jobs before! The post is refreshing because it sits in the neglected middle ground between the tons of Python-for-beginners content out there and the arcane "I'm a core developer writing about very specific things" content.
Hey, I really enjoyed reading your post. It's straight to the point, the list is very rich and well illustrated with examples. I have been coding in python for the last 20 years and I nowadays rarely like reading these posts as much as I did for yours. Kudos.
I think the background of the post is very well explained on the blog.
But I’m surprised to read you went from idea to posting each day. I had expected maybe a week of preparation before starting execution.
Regardless it’s a great post, regardless of wether one think there is too much typing or too many new features in modern Python or one is a lover of niche solutions to shorten code.
Every time I try to use Python I get this mixed feeling of liking how few guard rails there are between me and the logic I am making and this lingering worry that my code looks like fools attempt to use Python. So much exists as convention or loosely followed rules. Whenever I read articles like this I am wowed by the depth of things I didn't know about Python or how much has changed. It makes something like Go feel like a comfort blanket because I have reasonable certainty code I write won't feel deprecated or left behind a year or two later. Excellent article showing off even more I didn't know.
There are plenty of language features which let you re-use or build your own guardrails in python.
I do with the concept of truthiness and falsiness would be taken out and shot though. It's been responsible for far too many nasty bugs IME and it only cuts out a few extra characters. Not a great trade off.
Instead of comfort I feel constantly annoyed that the language gives me so little that I have to produce more lines and very little in the way of convenience. It's a language that benefits a lot from AI completion, at least. Yeah, old code still looks the same as new code but that is due to a lack of progress on the language though.
I write an unfortunate amount of Go code.. because Go supports 2(?) of the features in this post (structural typing and generics, sort of, if you're feeling generous).
For example, circular imports aren't supported in Go at all. I run into this from time to time even if using an interface sometimes consts or other types are defined in the package as well and the whole thing has to be refactored. No way around it, has to be done.
Circular imports aren't encouraged in Python but Python never leaves you without options. Code can import types only while type checking or move imports out of the module level are quick workarounds. (I hope for typescripts `import type` someday.) That's what gives me a "comfort" feeling in Python, knowing that the language isn't likely to force me to work around the language design.
pick up a book on how to program pythonically and it will feel better, I promise. A lot of people program in python like they do in c++ or rust or javascript and it doesn't feel "right" to them. If you follow the pythonistas take I think you'll appreciate it better.
My own opinion is that Python shall remain Python, and golang, Rust and Typescript each should be whichever they are with their unique philosophy and design.
I am coding in all 4, with some roughly 28 years now, and I don't like what is becoming of Python
There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes.
this looks familiar to me, but from other languages.
response := get_user_input()
I am aware of the factI am in minority, and not trying to change anyone's mind, simply what this voice to be heard from time to time.
All in all, a very comprehensive list of some of the recent introduced features.
There is an older list on SO which readers might also find useful:
Extra stuff like type checking and annotations are definitely not the reason why python became that popular and widely adapted and used, but it certainly doesn't hurt to add and use them.
To be clear, I'm not expecting people to start adding generics to their quick hacked together Python scripts (in fact please don't do that). Instead, if you're building a library or maintaining a larger Python codebase, a lot of these start becoming very useful. A lot of the typing features I mentioned are already used by Python under the hood, and that a lot of Python developers just take for granted.
> There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes.
Python got popular and widely adopted for the same reason PHP did: 1) it was readily available everywhere and 2) it's an easy language for beginners to pick up and get real applications working quickly.
But it turns out that the language features desirable for a quick prototype done by your fresh-out-of-college founding engineer aren't the same as the language features desirable for your team of 100 engineers trying to work together on the same codebase.
Python is at an odd intersection where it's used by everything from large teams building robust backend applications (particularly in the data/ML space), to data scientists hacking on scripts interactively in Jupyter, to ops people writing deployment scripts. You don't need type checking and annotations if you're writing a few one-off scripts, but you'd be crazy to not take advantage of them for a larger application.
A very good piece, that I enjoyed. But, I do get the impression that many of these more esoteric features, when reading a code-base, actually end up obscuring the developer's intention, and impair understanding. Without very competent IDE tooling, trying to work out what's happening in a piece of code is extremely difficult. It's turtles all the way down, and there's a lot more turtles now. A number of the new features, appreciated by many for good reason, tend to work against the Zen of Python which at some level explained the meteoric rise of Python as a bona fide implementation language over the last 30 years - simple, obvious, effective and practical.
I love Guido's balance. You don't need type checking and annotations if you don't want to use them. They are nice when you're playing in more complex systems, however, completely optional.
I have used Python for a long time for glue scripts, build stuff, test systems etc. and I've always thought it was great. I didn't like using it for larger applications though - when I did, I usually got bitten at some point by some effect of the dynamism (regardless of testing). At scale things became more unworkable.
With the type system my opinion on that has changed. We have a pretty large tooling codebase with protocol stacks, GUI, test automation etc. and it's all very maintainable with the type checking.
Up to now python emphasizes to be and remain the dynamically typed monkey patch happy core language it is, with the completely volunteer option to provide type hints and use them to your advantage as you see fit.
So you can hack away and monkey patch to your heart's content and nothing is taken from you. no rust borrow checker. no need to use a type checker. ducks everywhere.
and I'm not aware of features that have to be used, aka incompatible changes to the core language.
I mean it's completely optional. Even if you are to use python as a glue language, with absolutely 0 typing annotations on your side, it's still very very useful to have them in the libraries that you inevitably end up using with python. It's not even like it requires an additional build step, so it's really just free.
It has an effect, and is usually worth including anyway. I used to omit it by default; now I include it by default. Also, you say "nowadays" but it's been almost 13 years now (https://peps.python.org/pep-0420/).
> since 3.13 there is a @deprecated decorator that does what you think it does
Nice find. Probably worth mentioning it comes from the `warnings` standard library.
> the time package has functions for monotonic clocks and others not just time()
There's quite a bit in there, but I question how many people need it.
Anyway, it's always surprising to me how when other people make these lists, such a large fraction is taken up by tricks with type annotations. I was skeptical of the functionality when the `typing` standard library was introduced; I've only grown more and more wary of it, even as people continue to insist to me that it's somehow necessary.
This is a nice list of "things you might not know" that is worth skimming to add to your toolkit.
If you are really interested in "advanced Python", though, I would recommend the book Fluent Python by Ramalho. I have the first edition which is still highly relevant, including the async bits (you just have to translate the coroutines into async syntax). There is a second edition which is more up to date.
I would also recommend checking out the functools[0] and itertools[1] modules in the standard library. Just go and read the docs on them top to bottom.
It's also worth reading the first few sections of Python Data Model[2] and then bookmarking this page.
+1 to itertools especially. Absurdly powerful, and the recipes at the bottom of the doc page are terrific.
The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need.
> The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need.
My favourite example of a similar thing happening to me was when I was asked to reverse the digits in a number. I somewhat jokingly asked if I was assuming base 10, which got some awkward looks so I knew something was up. They weren't impressed at all with my answer of `"".join(reversed(str(123456789)))`. I didn't get the job.
For me, the biggest benefit to python is that it feels like executable pseudocode. The language gets out of the way of your domain level instructions. This is probably why most non-programmers have the easiest time with python.
The more fancy stuff you add to it, the less attractive it becomes. Sure, most of these things have some sort of use, but I reckon most people do not get deep enough into python to understand all these little things.
I disagree. Adding "fancy stuff" by definition does not remove what made it attractive to you in the first place. You even acknowledge this in your second sentence. Something you don't know of is not capable to influence your opinion in any way. And when you know of it -- and dislike it -- just keep doing it the way you did before knowing it.
Lua is like this only moreso, and there's little fear of "progress" as large parts of the community have stuck with Lua5.1.
Of course, Lua is batteries-not-included so there may be the problem of "progress" in external libraries; in practice things like Penlight barely change though.
The only things I’d have probably change about this list is the inclusion of some of the collections.abc containers for type annotations, TypedDict and how it can make working with strictly structured dictionaries not terrible (but if it looks like an class and quacks like a class, make it a class if you can), and Counter (only because I forget it exists every single time I should have used it).
Several comments disliking the walrus operator, like many of the features on this list I also hated it… until I found a good use for it. I almost exclusively write strictly typed Python these days (annotations… another feature I originally hated). The walrus operator makes code so much cleaner when you’re dealing with Optionals (or, a Union with None). This comes up a lot with regex patterns:
if (match := pattern.search(line)) is not None:
print(match.group())
Could you evaluate match on a separate line before the conditional? Sure. But I find this is a little clearer that the intended life of match is within the conditional, making it less tempting to reuse it elsewhere.
Not a Python feature specifically, but I’d also love to see code that uses regex patterns to embrace named capturing groups more often. .group(“prefix”) is a lot more readable than .group(1).
Nitpick about 9.3 Short Circuit Evaluation: both things evaluate differently if you have empty strings. The if-else clause treats empty strings as valid while the or operator will treat them equivalent with None.
Readit News
I did not expect to wake up at 4am seeing my post on front page HN, but here we are nevertheless :D
As the intro mentioned, these started off as 14 small tweets I wrote a month prior to starting my blog. When I finally got that set up, I just thought, "hey, I just spent the better part of two weeks writing these nifty Python tricks, might as well reuse them as a fun first post!"
That's why the flow might seem a little weird (as some pointed out, proxy properties are not really a Python "feature" in of itself). They were just whatever I found cool that day. I tried to find something more esoteric if it was a Friday, and something useful if it was a Monday. I was also kinda improving the entire series as it was going on, so that was also a factor.
Same goes with the title. These were just 14 feature I found interesting while writing Python both professionally and as a hobby. Some people mentioned these are not very "advanced" per se, and fair enough. I think I spent a total of 5 second thinking of a title. Oh well!
But I’m surprised to read you went from idea to posting each day. I had expected maybe a week of preparation before starting execution.
Regardless it’s a great post, regardless of wether one think there is too much typing or too many new features in modern Python or one is a lover of niche solutions to shorten code.
Congratulations for ending up on the front page! (I hope the server hosting your blog is okay!)
One I think you missed is getters and setters on attributes!
I do with the concept of truthiness and falsiness would be taken out and shot though. It's been responsible for far too many nasty bugs IME and it only cuts out a few extra characters. Not a great trade off.
I write an unfortunate amount of Go code.. because Go supports 2(?) of the features in this post (structural typing and generics, sort of, if you're feeling generous).
For example, circular imports aren't supported in Go at all. I run into this from time to time even if using an interface sometimes consts or other types are defined in the package as well and the whole thing has to be refactored. No way around it, has to be done.
Circular imports aren't encouraged in Python but Python never leaves you without options. Code can import types only while type checking or move imports out of the module level are quick workarounds. (I hope for typescripts `import type` someday.) That's what gives me a "comfort" feeling in Python, knowing that the language isn't likely to force me to work around the language design.
I am coding in all 4, with some roughly 28 years now, and I don't like what is becoming of Python
There is a reason why python become that popular and widely adapted and used, and it is not the extra layers of type checking, annotations and the likes.
this looks familiar to me, but from other languages.
I am aware of the factI am in minority, and not trying to change anyone's mind, simply what this voice to be heard from time to time.All in all, a very comprehensive list of some of the recent introduced features.
There is an older list on SO which readers might also find useful:
https://stackoverflow.com/questions/101268/hidden-features-o...
To be clear, I'm not expecting people to start adding generics to their quick hacked together Python scripts (in fact please don't do that). Instead, if you're building a library or maintaining a larger Python codebase, a lot of these start becoming very useful. A lot of the typing features I mentioned are already used by Python under the hood, and that a lot of Python developers just take for granted.
Case in point, the python-opencv (https://github.com/opencv/opencv-python) library has basically no types and it's an absolute pain to work with.
BTW thats a really good SO thread, thanks for linking it!
Python got popular and widely adopted for the same reason PHP did: 1) it was readily available everywhere and 2) it's an easy language for beginners to pick up and get real applications working quickly.
But it turns out that the language features desirable for a quick prototype done by your fresh-out-of-college founding engineer aren't the same as the language features desirable for your team of 100 engineers trying to work together on the same codebase.
Python is at an odd intersection where it's used by everything from large teams building robust backend applications (particularly in the data/ML space), to data scientists hacking on scripts interactively in Jupyter, to ops people writing deployment scripts. You don't need type checking and annotations if you're writing a few one-off scripts, but you'd be crazy to not take advantage of them for a larger application.
With the type system my opinion on that has changed. We have a pretty large tooling codebase with protocol stacks, GUI, test automation etc. and it's all very maintainable with the type checking.
Up to now python emphasizes to be and remain the dynamically typed monkey patch happy core language it is, with the completely volunteer option to provide type hints and use them to your advantage as you see fit.
So you can hack away and monkey patch to your heart's content and nothing is taken from you. no rust borrow checker. no need to use a type checker. ducks everywhere.
and I'm not aware of features that have to be used, aka incompatible changes to the core language.
So what is the critique, exactly?
* did you know __init__.py is optional nowadays?
* you can do relative imports with things like "from ..other import foo"
* since 3.13 there is a @deprecated decorator that does what you think it does
* the new generics syntax also works on methods/functions: "def method[T](...)" very cool
* you can type kwargs with typeddicts and unpack: "def fn(*kwargs: Unpack[MyKwargs])"
* dataclasses (and pydantic) support immutable objects with: "class MyModel(BaseModel, frozen=True)" or "@dataclass(frozen=True)"
* class attributes on dataclasses, etc. can be defined with "MY_STATIC: ClassVar[int] = 42" this also supports abstract base classes (ABC)
* TypeVar supports binding to enforce subtypes: "TypeVar['T', bound=X]", and also a default since 3.13: "TypeVar['T', bound=X, default=int]"
* @overload is especially useful for get() methods to express that the return can't be none if the default isn't None
* instead of Union[a, b] or Optional[a] you can write "a | b" or "a | None" nowadays
* with match you can use assert_never() to ensure exhaustive matching in a "case _:" block
* typing has reveal_type() which lets mypy print the type it thinks something is
* typing's "Self" allows you to more properly annotate class method return types
* the time package has functions for monotonic clocks and others not just time()
anyone know more things?
It has an effect, and is usually worth including anyway. I used to omit it by default; now I include it by default. Also, you say "nowadays" but it's been almost 13 years now (https://peps.python.org/pep-0420/).
> since 3.13 there is a @deprecated decorator that does what you think it does
Nice find. Probably worth mentioning it comes from the `warnings` standard library.
> the time package has functions for monotonic clocks and others not just time()
There's quite a bit in there, but I question how many people need it.
Anyway, it's always surprising to me how when other people make these lists, such a large fraction is taken up by tricks with type annotations. I was skeptical of the functionality when the `typing` standard library was introduced; I've only grown more and more wary of it, even as people continue to insist to me that it's somehow necessary.
It's not optional. Omitting it gets you a namespace package, which is probably not what you want.
> TypeVar supports binding to enforce subtypes: "TypeVar['T', bound=X]",
Using the new generics syntax you mentioned above you can now do:
If you are really interested in "advanced Python", though, I would recommend the book Fluent Python by Ramalho. I have the first edition which is still highly relevant, including the async bits (you just have to translate the coroutines into async syntax). There is a second edition which is more up to date.
I would also recommend checking out the functools[0] and itertools[1] modules in the standard library. Just go and read the docs on them top to bottom.
It's also worth reading the first few sections of Python Data Model[2] and then bookmarking this page.
[0] https://docs.python.org/3/library/functools.html
[1] https://docs.python.org/3/library/itertools.html
[2] https://docs.python.org/3/reference/datamodel.html
The only problem I’ve found is that for interviews, people often aren’t familiar with it, which can lead to you solving whatever puzzle they had in far less time than they intended, and without manually building whatever logic it was they assumed you would need.
My favourite example of a similar thing happening to me was when I was asked to reverse the digits in a number. I somewhat jokingly asked if I was assuming base 10, which got some awkward looks so I knew something was up. They weren't impressed at all with my answer of `"".join(reversed(str(123456789)))`. I didn't get the job.
The more fancy stuff you add to it, the less attractive it becomes. Sure, most of these things have some sort of use, but I reckon most people do not get deep enough into python to understand all these little things.
That breaks down as soon as you need to work with anyone else's code that uses the "fancy stuff".
Of course, Lua is batteries-not-included so there may be the problem of "progress" in external libraries; in practice things like Penlight barely change though.
Several comments disliking the walrus operator, like many of the features on this list I also hated it… until I found a good use for it. I almost exclusively write strictly typed Python these days (annotations… another feature I originally hated). The walrus operator makes code so much cleaner when you’re dealing with Optionals (or, a Union with None). This comes up a lot with regex patterns:
Could you evaluate match on a separate line before the conditional? Sure. But I find this is a little clearer that the intended life of match is within the conditional, making it less tempting to reuse it elsewhere.Not a Python feature specifically, but I’d also love to see code that uses regex patterns to embrace named capturing groups more often. .group(“prefix”) is a lot more readable than .group(1).