This is a great feature for those who want a mix of positional and keyword-only arguments.
I should have mentioned originally (and I've since updated my post) that this and the kw_only= flag both require Python 3.10 and higher, so code that works with older versions can't opt into it yet.
I like keyword-only arguments, but they become tedious too quickly - especially when the variable names already match (fn(x=x, y=y, z=z)). I wish Python had JavaScript’s shorthand property syntax. (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...).
JS's shorthand property syntax is lovely and elegant. Python can't really adopt it though as it clashes with the set syntax (which is such a niche use case it really shouldn't have a special syntax).
You could do something like f(**{x, y, z}) with just that. Not the prettiest, but at least it would be DRY.
But in general Python devs seem to prefer "explicit" ad-hoc syntax for each use case instead of composable "primitives". Which is approaching a kind of C++ situation where relatively few users know the syntax comprehensively.
> But in general Python devs seem to prefer "explicit" ad-hoc syntax for each use case instead of composable "primitives".
I hope you don't consider JS having "composable primitives". To backup my point: there is no anything similar to underscore-like libraries for Python. All is covered by stdlib and syntax.
By the look of it, the feeling I get is that a decent and convenient syntax proposal has been bikeshedded into rejection. Some particular form of keyword arguments at invocation would definitely make quite a few of my scripts more readable.
That's annoying for sure. Though a different problem.
All the kw_only=True argument for dataclasses does is require that you pass any fields you want to provide as keyword arguments instead of positional arguments when instantiating a dataclass. So:
obj = MyDataclass(a=1, b=2, c=3)
Instead of:
obj = MyDataclass(1, 2, 3) # This would be an error with kw_only=True
The problem you're describing in boto3 (and a lot of other API bindings, and a lot of more layered Python code) is that methods often take in **kwargs and pass them down to a common function that's handling them. From the caller's perspective, **kwargs is a black box with no details on what's in there. Without a docstring or an understanding of the call chain, it's not helpful.
Python sort of has a fix for this now, which is to use a TypedDict to define all the possible values in the **kwargs, like so:
from typing import TypedDict, Unpack
class MyFuncKwargs(TypedDict):
arg1: str
arg2: str
arg3: int | None
def my_outer_func(
**kwargs: Unpack[MyFuncKwargs],
) -> None:
_my_inner_func(**kwargs)
def _my_inner_func(
*,
arg1: str,
arg2: str,
arg3: int | None,
) -> None:
...
By defining a TypedDict and typing **kwargs, the IDE and docs can do a better job of showing what arguments the function really takes, and validating them.
Also useful when the function is just a wrapper around serializing **kwargs to JSON for an API, or something.
But this feature is far from free to use. The more functions you have, the more of these you need to create and maintain.
Ideally, a function could type **kwargs as something like:
And then the IDEs and other tooling can just reference that function. This would help make the problem go away for many of the cases where **kwargs is used and passed around.
Boto3 in general is just an utter pain in the ass to work with. It's like they did everything they could to go out of their way to make sure your IDE couldn't figure out how to do anything.
Like, why the hell did they use strings to decide what kind of client you want? Why make us do `s3 = boto3.client('s3')` instead of `s3 = boto3.client.s3()` or something similar? It means my IDE can't figure out what the type of `s3` is.
Everything about it is so unpythonic. Functions and classes in it might use the Python snake_case, but keyword arguments use PascalCase.
I've often considered writing a wrapper around it to provide a sane interface to the AWS API, but the amount of functionality is so vast that it would be a massive undertaking.
I guess some prefer to stick with the stdlib instead of third party libs.
Also, dataclasses feels more straightforward and less "magic" to me (in the sense that it is more or less "just" a way to avoid boilerplate for class definition, while pydantic does way more "magic" stuff like de-/serialization and validation, and adding numerous methods and attributes to the classes).
When I started to implement typedload, when types were just introduced, I supported NamedTuple, and then as more things were added, also attrs, dataclasses, typed dict…
What would be the point to require migrating the whole codebase to use something different to use your library?
On the other hand, if you wrote your code from scratch to use basemodel you're pretty much stuck with pydantic.
You could argue the same thing (forcing kwargs) for all Python functions/methods, although, that would make using your APIs very annoying. The `__init__` method for dataclasses are just another method like any other.
As a general rule of thumb, I only start forcing kwargs once I'm looking at above 4-5 arguments, or if the arguments are similar enough that forcing kwargs makes the calling code more readable. For a small number of distinct arguments, forcing kwargs as a blanket rule makes the code verbose for little gain IMO.
For Objective C, using named parameters is the only way to call methods. I don't think I read many critique about this particular aspect. IMO it's actually a good thing and increases readability quite a bit.
For JavaScript/TypeScript React codebase, using objects as a poor man's named parameters also very popular approach.
Also I'd like to add, that it seems a recent trend to add feature to IDEs, where it'll add hint for every parameter, somewhat simulating named parameters. So when you write `mymethod(value)`, it'll display it as `mymethod(param:value)`.
So may be not very annoying.
The only little thing I'd like to borrow from JavaScript is using "shortcut", so you could replace `x=x` with `x`, if your local variable happened to have the same name, as parameter name (which happens often enough).
To be pedantic, Objective-C doesn't have named parameters. Method names are composed of multiple parts, each corresponding to a parameter. Such design contributes to the method's readability and memorability. In contrast, Python methods have their own names, and parameter names are chosen as an afterthought. While there's no reason why Python methods couldn't be named in accordance with parameter names, unfortunately it hasn't been a part of Python's culture.
I find that anything above 2 arguments benefits from explicit keyword notation. With 4-5 arguments, especially when most of them are of the same type, it can be difficult to tell which is which.
> You could argue the same thing (forcing kwargs) for all Python functions/methods, although, that would make using your APIs very annoying. The `__init__` method for dataclasses are just another method like any other.
While that is self evident at a technical level, it is not quite so from a clarity / documentary perspective: “normal” functions and methods can often hint at their parameters through their naming but it is uncommon for types, for which the composite tends to be much more of an implementation detail.
Of course neither rule is universal e.g. the composite is of prime importance for newtypes, and indeed they often use tuple-style types or have special support with no member names.
Readit News
I should have mentioned originally (and I've since updated my post) that this and the kw_only= flag both require Python 3.10 and higher, so code that works with older versions can't opt into it yet.
Thank you for the tip.
You could do something like f(**{x, y, z}) with just that. Not the prettiest, but at least it would be DRY.
But in general Python devs seem to prefer "explicit" ad-hoc syntax for each use case instead of composable "primitives". Which is approaching a kind of C++ situation where relatively few users know the syntax comprehensively.
I hope you don't consider JS having "composable primitives". To backup my point: there is no anything similar to underscore-like libraries for Python. All is covered by stdlib and syntax.
> for(in) / for(of)
Cough, cough.
edit: nevermind, that PEP was rejected :/
Deleted Comment
“What parameters does this take?” you ask, “why, it takes ‘kwargs’” responds the docs and your IDE.
How incredibly helpful!
All the kw_only=True argument for dataclasses does is require that you pass any fields you want to provide as keyword arguments instead of positional arguments when instantiating a dataclass. So:
Instead of: The problem you're describing in boto3 (and a lot of other API bindings, and a lot of more layered Python code) is that methods often take in **kwargs and pass them down to a common function that's handling them. From the caller's perspective, **kwargs is a black box with no details on what's in there. Without a docstring or an understanding of the call chain, it's not helpful.Python sort of has a fix for this now, which is to use a TypedDict to define all the possible values in the **kwargs, like so:
By defining a TypedDict and typing **kwargs, the IDE and docs can do a better job of showing what arguments the function really takes, and validating them.Also useful when the function is just a wrapper around serializing **kwargs to JSON for an API, or something.
But this feature is far from free to use. The more functions you have, the more of these you need to create and maintain.
Ideally, a function could type **kwargs as something like:
And then the IDEs and other tooling can just reference that function. This would help make the problem go away for many of the cases where **kwargs is used and passed around.Like, why the hell did they use strings to decide what kind of client you want? Why make us do `s3 = boto3.client('s3')` instead of `s3 = boto3.client.s3()` or something similar? It means my IDE can't figure out what the type of `s3` is.
Everything about it is so unpythonic. Functions and classes in it might use the Python snake_case, but keyword arguments use PascalCase.
I've often considered writing a wrapper around it to provide a sane interface to the AWS API, but the amount of functionality is so vast that it would be a massive undertaking.
Also, dataclasses feels more straightforward and less "magic" to me (in the sense that it is more or less "just" a way to avoid boilerplate for class definition, while pydantic does way more "magic" stuff like de-/serialization and validation, and adding numerous methods and attributes to the classes).
If I need something more than dataclasses, I’ll normally go for attrs/cattrs. Dataclasses were originally based on attrs, so it’s not much of a leap.
When I started to implement typedload, when types were just introduced, I supported NamedTuple, and then as more things were added, also attrs, dataclasses, typed dict…
What would be the point to require migrating the whole codebase to use something different to use your library?
On the other hand, if you wrote your code from scratch to use basemodel you're pretty much stuck with pydantic.
Deleted Comment
Deleted Comment
https://peps.python.org/pep-3102/
More recently, Python also added support for positional-only parameters:
https://peps.python.org/pep-0570/
As a general rule of thumb, I only start forcing kwargs once I'm looking at above 4-5 arguments, or if the arguments are similar enough that forcing kwargs makes the calling code more readable. For a small number of distinct arguments, forcing kwargs as a blanket rule makes the code verbose for little gain IMO.
For Objective C, using named parameters is the only way to call methods. I don't think I read many critique about this particular aspect. IMO it's actually a good thing and increases readability quite a bit.
For JavaScript/TypeScript React codebase, using objects as a poor man's named parameters also very popular approach.
Also I'd like to add, that it seems a recent trend to add feature to IDEs, where it'll add hint for every parameter, somewhat simulating named parameters. So when you write `mymethod(value)`, it'll display it as `mymethod(param:value)`.
So may be not very annoying.
The only little thing I'd like to borrow from JavaScript is using "shortcut", so you could replace `x=x` with `x`, if your local variable happened to have the same name, as parameter name (which happens often enough).
While that is self evident at a technical level, it is not quite so from a clarity / documentary perspective: “normal” functions and methods can often hint at their parameters through their naming but it is uncommon for types, for which the composite tends to be much more of an implementation detail.
Of course neither rule is universal e.g. the composite is of prime importance for newtypes, and indeed they often use tuple-style types or have special support with no member names.