Dumpster fire compared to what? Nuget? NPM? It may be that some more recent languages managed to start a saner solution and keep it sane.

But really, it's a hard problem, between cross-platform support, backwards compatibility, security concerns, hosting, most authors being volunteers and so on.

And still, even with "just" pip or even conda I am enjoying the Python experience more than some other packaging solutions I've seen.

If Python’s packaging is a dumpster fire, then what is Perl’s packaging?

    $ poetry init
    $ poetry add django
    $ poetry shell

It's pretty simple. Check in the lock file, and then run

    $ poetry install 

to replicate it.

> - Use pip freeze > requirements.txt as a "lockfile".

There's lots of reasons to not do this anymore, and Dependency Hell is real, and has been for 25 years with RedHat RPM's, etc.

Even if you don't want to rely upon poetry for building in prod, poetry can still export a requirements.txt file for you, so you're not locked into using poetry, but you still get to specify the high level packages you want, and let it solve the dep graph for you.

    pip install pip-chill
    pip-chill > requirements.txt

    pip-chill --no-version > requirements.txt

That probably works for smaller projects without many dependencies, but it’s just going to install the sub-dependency versions that satisfy whatever comes last in the requirements file. The pip docs describe that situation here: https://pip.pypa.io/en/latest/topics/dependency-resolution/

The pip docs also suggest using pip-tools to create lock files. Pip-tools is only for creating lock files (it’s not trying to fix virtualenvs like poetry is), and it works great.

> - Use pip freeze > requirements.txt as a "lockfile".

This is not and has never ever been correct. It makes it infinitely harder to install an application vs the standard `pip install -e .` which works on every package manager and avoids PYTHONPATH issues, as well as being able to publish your application to PyPI for easy installation (as simple as pip install --user app or pipx install app).

PEP621 literally includes how Poetry lists dependencies as a synonym for the PEP's "dependencies" section. Poetry does, in fact, adhere to PEP621.

I 100% agree, I’ve made over 100 projects in the past 8 years of being deep into Python, ranging from enterprise software to little hobby projects. I’ve settled on exactly the setup. It’s great when you have multiple projects on the same machine, different dependencies, versions, maybe even specially forked changed and altered versions of libraries. It’s super versatile and easy to share. No problems running my software at all. I’ve even written a script that each time I commit for git it will quickly generate a new requirements file such that it’s always up to date. Thank you for sharing.

I have never experienced any issues with a virtualenv and pip.

python3 -m venv venv && source venv/bin/activate && pip install -r requirements.txt

Python standard library is great and its a nice languate if you like the syntax but aside from a few constants like Django, Flask, and Pandas, the ecosystem feels like it is slowly turning into a fragmented mess.

PDM author here, if anyone want to know the advantage of __pypackages__ over virtualenv-based tools(venv, virtualenv, poetry, pipenv), here it is: The fact that virtualenvs come with a cloned(or symlinked) interpreter makes it vulnerable when users want to upgrade the host interpreter in-place unless you keep all the old installations in your system, which is what pyenv is doing. You can imagine how many interpreters, including virtualenv-embedded ones are on your machine.

You can regard __pypackages__ as a virtualenv WITHOUT the interpreter, it can easily work with any python interpreter you choose as long as it has the same major.minor version as the packages folder.

Use Poetry. It does everything you need and PDM is a bit too new still.

Just go for virtualenv and pip. Virtualenvwrapper to have handy shell tools.

Poetry is good. We've used it 3 years in production without issue of large amounts of time to solve the dependencies.

    $ poetry init # to init the pyproject.toml file.
    $ poetry add <packages> 
    $ poetry shell # to activate inside the virutalenv.

Lock in the poetry.lock file after any change to your project dependencies, and other people can duplicate the project using after doing a git pull.

    $ poetry install

I tried them all a few months ago. Poetry was the best in my opinion.

Do you suffer from requiring different versions of the same library, or have some other kind of non trivial dependencies?

This will mark me out as a Luddite but I am still quite happy with a 5 line setup.py and “pip install .”

https://news.ycombinator.com/item?id=29446715

  $ cat setup.py
  from setuptools import setup
  setup(
    install_requires=['arrow'],
    packages=['dogclock'],
    scripts=['scripts/dogclock'])

I just started a Python project, using PDM. So far, I like it a lot. If I hit a show-stopper, no big deal, it's pretty easy to switch.

Pipenv is garbage. Don’t use it.

I’d just use pip and requirements files if you can. It’s doubtful that your requirements are sufficiently complex as to require a more complex resolver, although that depends on your ML needs.

Thanks for the kind words on PDM. At time of creating PDM I don't want it to be similar with any other package mangers, so I chose PEP 582, and I thought I can play more new stuff on it.

But as PDM becomes mature, it is acknowleged by the Python packaging people, I also work hard to make PDM fit more people's workflow, fortunately, it has a strong plugin system. You can add virtualenv support(pdm-venv), publish command(pdm-publish) and more. In the future, I would like to see it can eventually push the iteration of PEP 582 and make it finalized.

God those PEP's are exactly what I need

This is understandable. But if it is experimental, please label is as such. No mention of that on the landing page right now.

EDIT: Oh, I should say: If it's meant to take over the world, say so, as well!

The big distinguisher of PDM is that it support PEP 582[0]. That means it works less like Pip and works more like NPM of the JS world. To quote PEP 582:

> This PEP proposes to add to Python a mechanism to automatically recognize a __pypackages__ directory and prefer importing packages installed in this location over user or global site-packages. This will avoid the steps to create, activate or deactivate "virtual environments". Python will use the __pypackages__ from the base directory of the script when present.

Thus, the idea of PDM is that it will create a directory, called `__pypackages__` in the root of your project and in that folder it'll populate all the dependencies for that project. Then, when you run scripts in the root folder of your project, your Python install will see that there's a `__pypackages__` folder and use that folder to look up dependencies.

This style of "dependencies inside the project directory" is similar to how npm of the Javascript ecosystem works, where it creates a `node_modules/` folder in the root of your project and fills that folder with the dependencies for your project. This style of dependency management is different from other package managers such as Poetry (Python), Pip (Python), go (Golang), and cargo (Rust), all of which instead have a sort of "secret folder acting as cache of dependencies at particular versions", a folder that's usually pretty hidden out of the way, in which the package manager automatically manages the acquisition, storage, and versioning/version resolution (Poetry, Go, Cargo, all do this but Pip does not).

That's a very fast and probably wrong rundown on what makes this package manager different from others.

[0] - https://www.python.org/dev/peps/pep-0582/

I think I'm starting to like poetry too!