I believe that you are sincere and truthful in what you say.

Unfortunately, the integrity of employees is no guard against the greed of investors.

Maybe next year investors change the CEO and entire management and they start monetizing the open source tools. There is no way of knowing. But history tells us that there is a non-trivial chance of this happening.

The uncertainly over a future rug pull is always real, but in reality I wonder if the actual reason for people's hesitancy is more than just that. In reality I suspect it's closer to one of simply identity and the ownership model itself. Just the very idea that core tooling you depend on is in the hands of a commercial company is enough to many back off in a way one might not be when the tooling is in the hands of a broader community that one can support on more equal terms.

@woodruffw I love your quote above that commits you to your open source base and I'm rooting for you. But how about an approach that commits you to this sentence in a more rigorous and legal way, and spin off your open source tooling to a separate community-based entity? Of course, upon that you can continue to maintain sufficient representation to make Astral's commercial products the natural progression and otherwise the model remains the same. That would be a significant transfer of control, but it is that very transfer that would get a overwhelming response from the community and could really unblock these great tools for massive growth.

I work a lot with LLVM/Clang and whilst i know Apple and Google are significant contributors I feel confident that LLVM itself exists outside of that yet accept that e.g. Apple's contributions afford them weight to steer the project in ways that match their interests in e.g. Swift and Apple tooling.

It makes sense, but the danger can come when non-paying users unwittingly become dependent on a service that is subsidized by paying customers. What you're describing could make sense if pyx is only private, but what if there is some kind of free-to-use pyx server that people start using? They may not realize they're building on sand until the VC investors start tightening the screws and insist you stop wasting money by providing the free service.

(Even with an entirely private setup, there is the risk that it will encourage too much developer attention to shift to working within that silo and thus starve the non-paying community of support, although I think this risk is less, given Python's enormous breadth of usage across communities of various levels of monetization.)

Conda said all this as well, and solved the same issues you’re trying to - namely precompiled versions of difficult to build packages. It then went commercial.

In the HPC space there are already Easybuild and Spack which make all the compiler tool chain and C and Fortran library dependency stuff very easy. They just haven’t taken off outside as they aim to solve cluster management problems but Spack is easy to self serve.

The entire reason people choose "permissive licenses" is so that it won't last forever. At best, the community can fork the old version without any future features.

Only viral licenses are forever.

> Basically, we're hoping to address this concern by building a separate sustainable commercial product rather than monetizing our open source tools.

jfrog artifactory suddenly very scared for its safety

Will pyx describe a server protocol that could be implemented by others, or otherwise provide software that others can use to host their own servers? (Or maybe even that PyPI can use to improve its own offering?) That is, when using "paid, hosted services like pyx", is one paying for the ability to use the pyx software in and of itself, or is one simply paying for access to Astral's particular server that runs it?

The hosting and administration part is what’s expensive and can’t be free and open source except when someone pays for it. So is npm open source? Whether it is or isn't doesn't matter as much as whether Microsoft continues to pay the bill.

I think the only way you can alleviate people's concerns here is by legally binding the company in a way that would adequately address those concerns. Which I don't recall ever seeing any company attempt in such a scenario, probably because it obviously devalues the company as it restricts what a potential future buyer could do with it. Which is exactly why some people don't buy such pinky promises.

This is the way it's worked for PHP's composer packages, and it's worked well.

> This is a valid concern, but astral just has an amazing track record.

The issue is, track record is not relevant when the next investors take over.

Frankly, it’s weird. You can find this business model all over the open-source world but for some reason Astral in particular is singled out for way more criticism on this than anything else I’ve seen, despite being unambiguously great contributors who have never put a foot wrong as far as I can tell.

Microsoft – who invented embrace, extend, and extinguish – own NPM, but I don’t see people wringing their hands over them in every thread that mentions NPM. But you mention Astral here or on Reddit and people line up to tell you it’s only a matter of time before they fuck people over. Why the disparity?

HN comments default to cynisim.

Pyx represents the server side, not the client side. The analogue in the pre-existing Python world is PyPI.

Many ideas are being added to recent versions of pip that are at least inspired by what uv has done — and many things are possible in uv specifically because of community-wide standards development that also benefits pip. However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original). That has a lot to do with why I'm making PAPER.

For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process. It remembers entire wheels, but finding them in that cache requires knowing the URL from which they were downloaded. Because PyPI organizes the packages in its own database with its own custom URL scheme, pip would have to reach out to PyPI across the Internet in order to figure out where it put its own downloads!

This doesn't generalize: you could have said the same thing about pip versus easy_install, but pip clearly has worthwhile improvements over easy_install that were never merged back into the latter.

Pip is broken and has been for years, they're uninterested in fixing the search. Or even removing the search or replacing it with a message/link to the package index.

imo, if pip's preference is to ship broken functionality, then what is/is not shipped with pip is not meaningful.

I'm assuming by pip you mean pypi (the package registry) - I think you're making the mistake of thinking it has a lot more resources than it does, because of how much it punches above its weight.

Pypi is powered by warehouse which has around 3 developers maintaining it[0]. They're doing an awesome job, but the funding and resource available to them are probably substantially less than Astral could have with a paid offering.

[0] https://github.com/pypi/warehouse/graphs/contributors

It’s not that complex - just try it

HAHAHAH don't even get me started on how bad anaconda is. On how slow the installer + interpreter, how they avoided being a part of the usual pip workflow, bloated environment, cross platform inconsistencies, extremely slow dependency resolution, etc etc etc...

Posit has solved similar problems with their Package Manager as well, the benefit being that it's hosted on-prem, but the user has to build wheels for their desired architecture (if they're not on pypi).

This is very close minded. It's best to avoid statements like that.

I feel like having a working python environment is not a great requirement to managing your python environment.

is this ragebait ?

most of the stuff in the python ecosystem have a core built in C, including the language itself.

Definitely has never seen/used uv

Wait until you find out what language Python is written in.

Why?

+1

This is quite a disappointing self-limitation given the improvements uv brings to the table. You’re missing out on some good stuff.

this is the kind of statement that will get anyone who works for me PIPd

I started with "sudo apt install python" a long time ago and this installed python2. This was during the decades-long transition from python2 to python3, so half the programs didn't work so I installed python3 via "sudo apt install python3". Of course now I had to switch between python2 and python3 depending on the program I wanted to run, that's why Debian/Ubuntu had "sudo update-alternatives --config python" for managing the symlink for "python" to either python2 or python3. But shortly after that, python3-based applications also didn't want to start with python3, because apt installed python3.4, but Python developers want to use the latest new features offered by python3.5 . Luckily, Debian/Ubuntu provided python3.5 in their backports/updates repositories. So for a couple of weeks things sort of worked, but then python3.7 was released, which definitely was too fresh for being offered in the OS distribution repositories, but thanks to the deadsnakes PPA, I could obtain a fourth-party build by fiddling with some PPA commands or adding some entries of debatable provenance to /etc/apt/lists.conf. So now I could get python3.7 via "sudo apt install python3.7". All went well again. Until some time later when I updated Home Assistant to its latest monthly release, which broke my installation, because the Home Assistant devs love the latest python3.8 features. And because python3.8 wasn't provided anymore in the deadsnakes PPA for my Ubuntu version, I had to look for a new alternative. Building python from source never worked, but thank heavens there is this new thing called pyenv (cf. pyenv), and with some luck as well as spending a weekend for understanding the differences between pyenv, pyvenv, venv, virtualenv (a.k.a. python-virtualenv), and pyenv-virtualenv, Home Assistant started up again.

This wall of text is an abridged excursion of my installing-python-on-Linux experience.

There is also my installing-python-on-Windows experience, which includes: official installer (exe or msi?) from python.org; some Windows-provided system application python, installable by setting a checkbox in Windows's system properties; NuGet, winget, Microsoft Store Python; WSL, WSL2; anaconda, conda, miniconda; WinPython...

I felt like python packaging was more or less fine, right up until pip started to warn me that I couldn't globally install packages anymore. So I need to make a billion venvs to install the same ml, plotting libraries and dependencies, that I don't want in a requirements.txt for the project.

I just want packaging to fuck off and leave me alone. Changes here are always bad, because they're changes.

I've walked the same rocky path and have the bleeding feet to show for it! My problem is that now my packaging/environment mental model is so muddled I frequently mix up the commands...

What's wrong with just using virtualenv. I never used anything else, and I never felt the need to. Maybe it's not as shipping l shiny as the other tools, but it just works.

> But the dependency I need is built and packaged differently for different operating systems and flavor of GPU, and now my coworkers can't get the project to install on their laptops.

This is why containers are great IMO.

It's meant to solve the problem of "well it works on my machine"

Even the way you import packages is kinda wack

Coming from the php ecosystem this kind of package manager problems feels crazy.

Maybe Python and js people should just use composer too.

You forgot the wheels and eggs

We actually built a platform that eliminates all these steps, you can now reproduce GitHub repos with 0 manual config in 60% cases. check for more info on https://x.com/KeploreAI We just launched it and waiting for first users to be astonished :), let me know if you have any questions

Man I used python sparingly over the years and I still had to deal with all those package manager changes. Worse than the JS bundling almost?

„across different platforms“

First things first:

Import path, os

I love Python, the ZEN of it, and you really need to accept the fact that there are conventions - quite a lot and that bash or shell scripts are where the magic happens, like environmental variables, if you know how to secure your app.

Even the self thing finally makes sense after years of bewilderment (“Wait: not even Java is that brutal to its users.”)

Lately stumbled over poetry after really getting the gist out of venv and pip.

Still hesitant, because Windows doesn’t play a role.

I've been heartened by the progress that opentofu has made, so I think if it gets enough momentum it could survive the inevitable money grab

I agree, now I just use uv and forget about it. It does use up a fair bit of disk, but disk is cheap and the bootstrapping time reduction makes working with python a pleasure again

from what I hear uv is the "solved" and venv by hand is the old way

Lets be honest here - whilst some experiences are better/worse than others, there doesn't seem to be a dependency management system that isn't (at least half) broken.

I use Go a lot, the journey has been

- No dependency management

- Glide

- Depmod

- I forget the name of the precursor - I just remembered, VGo

- Modules

We still have proxying, vendoring, versioning problems

Python: VirtualEnv

Rust: Cargo

Java: Maven and Gradle

Ruby: Gems

Even OS dependency management is painful - yum, apt (which was a major positive when I switched to Debian based systems), pkg (BSD people), homebrew (semi-official?)

Dependency Management is the wild is a major headache, Go (I only mention because I am most familiar with) did away with some compilation dependency issues by shipping binaries with no dependencies (meaning that it didn't matter which version of linux you built your binary for, it will run on any of the same arch linux - none of that "wrong libc" 'fun'), but you still have issues with two different people building the same binary in need of extra dependency management (vendoring brings with it caching problems - is the version in the cache up to date, will updating one version of one dependency break everything - what fun)

Do I get it right that this issue is within Windows? I've never heard of the issues you describe while working with Linux.. I've seen people struggle with MacOS a bit due to brew different versions of some library or the other, mostly self compiling Ruby.

Have you tried conda? Since the integration of mamba its solver is fast and the breadth of packages is impressive. Also, if you have to support Windows and Python with native extensions, conda is a godsend.

I would recommend learning a little bit of C compilation and build systems. Ruby/Rails is about as polished as you could get for a very popular project. Maybe libyaml will be a problem once in a while if you're compiling Ruby from scratch, but otherwise this normally works without a hassle. And those skills will apply everywhere else. As long as we have C libraries, this is about as good as it gets, regardless of the language/runtime.

Have you tried JRuby? It might be a bit too large for your droplet, but it has the java versions of most gems and you can produce cross-platform jars using warbler.

I'm surprised to hear that. Ruby was the first language in my life/career where I felt good about the dependency management and packaging solution. Even when I was a novice, I don't remember running into any problems that weren't obviously my fault (for example, installing the Ruby library for PostgreSQL before I had installed the Postgres libraries on the OS).

Meanwhile, I didn't feel like Python had reached the bare minimum for package management until Pipenv came on the scene. It wasn't until Poetry (in 2019? 2020?) that I felt like the ecosystem had reached what Ruby had back in 2010 or 2011 when bundler had become mostly stable.

Have you tried Nix?

https://nixos.org

If you have to use, e.g., CUDA Toolkit 11.8, then you need a specific version of VS and its build tools for CUDA's VS integration to work. I don't know why exactly that is and I wish I didn't have to deal with it.

In my experience, Anaconda (including Miniconda, Micromamba, IntelPython, et al.) is still the default choice in scientific computing and machine learning.

Anaconda was a good idea until it would break apt on Ubuntu and make my job that much harder. That became the reason _not_ to use Anaconda in my book.

venv made these problems start to disappear, and now uv and Nix have closed the loop for me.

When was that ever a part of the definition? It was part of the early Unix culture, sure, but even many contemporary OSes didn't ship with compilers, which were a separate (and often very expensive!) piece of software.

OTOH today most Linux distros don't install any dev tools by default on a clean install. And, ironically, a clean install of Windows has .NET, which includes a C# compiler.

There are highs and lows to the system, just like with any system. Pip overall was a great package manager like 15 years ago, and a big step up from easy_install for sure (n.b., I started programming around the time easy_install was going out of fashion, so my point of view is coloured by that timing).

That said, it'd be nice if pip (or some PSF-blessed successor) adopted a model more similar to that offered by poetry (and formerly, pipenv, and now, I guess, uv) at least for package locking. `pip freeze > requirements.txt` isn't fit for purpose in the age of supply chain attacks, unfortunately, and PyPI already offers a reasonably workable backend for this model, whether the community at-large agrees to this or not. There are objective benefits (biggest one being better code integrity guarantees) that outweigh the objective (largely, performance-related) drawbacks.

Do you really think Python’s consensus-driven language development is better than authoritarian?

I am honestly tired of the Python packing situation. I breathe a sigh of relief in language like Go and Rust with an “authoritative” built-in solution.

I wouldn’t mind the 30 different packaging solutions as long as there was authoritative “correct” solution. All the others would then be opt-in enhancements as needed.

I guess a good thought experiment would be if we were to design a packaging system (or decide not to) for a new PL like python, what would it look like?

The combination of

– “client (uv) and server (pyx)” and

– “You can use it to host your own internal packages, or as an accelerated, configurable frontend to public sources like PyPI and the PyTorch index.”

is what really helped me understand what pyx aims to be.

I would also put this list of issues that this fixes higher. It makes it more obvious what the point is. (And also a setuptools update literally broke our company CI last week so I was like "omg yes" at that point.)

Astral doesn't really have a business model yet, it has potential business models.

The issue is that there isn't a clean business model that will produce the kind of profits that will satisfy their VCs - not that there isn't any business model that will help support a business like theirs.

Private package management would probably work fine if they hadn't taken VC money.

It’s not negative, it’s accurate. The playbook is well known and users should be informed.

We're hoping that building a commercial service makes it clear that we have a sustainable business model and that our tools (like uv) will remain free and permissively licensed.

(I work at Astral)

You know what they say: The best time to adopt uv was last year...

I'm all seriousness, I'm all in on uv. Better than any competition by a mile. Also makes my training and clients much happier.

Fortunately for a lot of what uv does, one can simply switch to something else like Poetry. Not exactly a zero-code lift but if you use pyproject.toml, there are other tools.

Of course if you are on one of the edge cases of something only uv does, well... that's more of an issue.

Given how widely popular uv is, I'm pretty sure that in the event of any impactful license change it would immediately get forked.