Why it took 4 years to get a lock files specification

They mostly took inspiration from other languages for UV. Cargo (Rust) was a huge inspiration, but they got stuff from Ruby as well, I believe. There was an episode on "the changelog" about it. Don't remember them saying anything about PEPs, although that might just be me not having listened to the entire thing. However, the Charlie Marsh was extremely insistent on the advantages of being a polyglot and hiring people with diverse programming experiences. So I think it's quite safe to assume that played a bigger role than just PEPs.

I can't speak for the UV team. My 2C on how I would treat the PEPs: If there is an accepted one, and implementing it doesn't go too strongly against your competing design goals, do it for compatibility. This does not imply that the PEP is driving your design, or required to make your software. It is a way to improve compatibility.

> dedicated funding can make a huge impact on a project

Where does Astral's funding come from, anyway?

Rye was already pretty good before it was donated to astral and renamed to uv though...

It's true that where Python offers critical performance it's typically by providing a nice interface to existing compiled code. But people who work through those interfaces are still fundamentally "doing it in Python"; the most important "it" is that which makes a useful system on top of the number-crunching.

But putting that aside, a big part of uv's performance is due to things that are not the implementation language. Most of the actually necessary parts of the installation process are I/O bound, and works through C system calls even in pip. The crunchy bits are package resolution in rare cases (where lock files cache the result entirely), and pre-compiling Python to .pyc bytecode files (which is embarrassingly parallel if you don't need byte-for-byte reproducibility, and normally optional unless you're installing something with admin rights to be used by unprivileged users).

Uv simply has better internal design. You know that original performance chart "installing the Trio dependencies with a warm cache"?

It turns out that uv at the time defaulted to not pre-compiling while pip defaulted to doing it; an apples-to-apples comparison is not as drastic, but uv also does the compilation in parallel which pip hasn't been doing. I understand this functionality is coming to pip soon, but it just hasn't been a priority even though it's really not that hard (there's even higher-level support for it via the standard library `compileall` module!).

More strikingly, though, uv's cache actually caches the unpacked files from a wheel. It doesn't have to unzip anything; it just hard-links the files. Pip's cache, on the other hand, is really an HTTPS cache; it basically simulates an Internet connection locally, "downloading" a wheel by copying (the cached artifact has a few bytes of metadata prepended) and unpacking it anew. And the files are organized and named according to a hash of the original URL, so you can't even trivially reach in there and directly grab a wheel. I guess this setup is a little better for code reuse given that it was originally designed without caching and with the assumption of always downloading from PyPI. But it's worse for, like, everything else.

Yet more proof that confirmation bias exists.

Find something better than Django.

Or matplotlib.

Or PyTorch.

after using both for the past 5 years, I'd say the main difference between peotry and uv is user experience and documentation. Poetry fought me every step of the way w.r.t. pip migration, venv, poetry updates. uv just worked in these areas, which meant I was updating packages much more frequently, which increased my experience and confidence in uv for a virtual cycle.

Ruff was the gateway drug for me, much better than the black/isort/etc combo.

Led me to try uv, which fixed a couple of egregious bugs in pip. Add speed and it’s a no brainer.

I don’t think poetry has these advantages, and heard about bugs early on. Is that completely fair? Probably not. But it’s obvious astral tools have funding and a competent team.

Speed and simplicity. Now I can fetch one binary on a system and in seconds fetch everything needed to run a Python tool or work on a code base.

I can do all that without having to even worry about virtual ends, or Python versions too.

Poetry was in some senses before its time; people were frustrated with pip, but didn't really understand the problems they were encountering, meanwhile the ecosystem had started trying to move away from doing everything with Setuptools for everyone. There's a ton that I'd love to explain here about the history of pyproject.toml etc. but the point is that Poetry had its own idea about what it would mean to be an all-in-one tool... and so did everyone else. Meanwhile, the main packaging people had been designing standards with the expectation of making a UNIX-style tool ecosystem work.

Everyone seems to like uv's answer better, but I'm still a believer in composable toolchains, since I've already been using those forever. I actually was an early Poetry adopter for a few reasons. In particular, I would have been fine sticking with Setuptools for building projects if it had supported PEPS 517/518/621 promptly. 621 came later but Poetry's workaround was nicer than Setuptools' to me. And it was easier to use with the new pyproject.toml setup, and I really wanted to get away from the expectation of using setup.py even for pure-Python projects.

But that was really it. The main selling point point of Poetry was (and is) that they offered a lockfile and dependency resolution, but these weren't really things I personally needed. So there was nothing really to outweigh the downsides:

* The way Poetry does the actual installation is, as far as I can tell, not much different from what pip does. And there are a ton of problems with that model.

* The early days of Poetry were very inconsistent in terms of installation and upgrade procedures. There was at least once that it seemed that the only thing that would work was a complete manual uninstall and reinstall, and I had to do research to figure out what I had to remove for the uninstallation as there was nothing provided to automate that.

* In the end, Poetry didn't have PEP 621 support for about four years (https://github.com/python-poetry/roadmap/issues/3 ; the OP was already almost a year after PEP acceptance in https://discuss.python.org/t/_/5472/109); there was this whole thing about how you were supposed to use pyproject.toml to describe the basic metadata of your project for packaging purposes, but if you used Poetry then you used Masonry to build, and that meant using a whole separate metadata configuration. Setuptools was slow in getting PEP 621 support off the ground (and really, PEP 621 itself was slow! It's hard to justify expecting anyone to edit pyproject.toml manually without PEP 621!), but Poetry was far slower still. I had already long given up on it at that point.

So for me, Poetry was basically there to provide Masonry, and Masonry was still sub-par. I was still creating venvs manually, using `twine` to upload to PyPI etc. because that's just how I think. Writing something like `poetry shell` (or `uv run`) makes about as much sense to me as `git run-unit-tests` would.

More information here: https://github.com/astral-sh/uv/issues/12584

> The biggest limitation is that there's no support for arbitrary entrypoints to the graph, because pylock.toml includes a fixed marker for each package entry rather than recording a graph of dependencies.

It's not a rant. It's a man with several burned fingers and a drifter beard muttering warnings from a corner.

How was the Python 2 to Python 3 migration for you? How many Python libraries are you responsible for packaging? Has that been fun for the past 15-20 years? How heavily do you rely on async?

Almost a decade for Django+major libraries to support Python 3. More than a decade for Django async support. I'm not ungrateful. And, I wish that I believed that throwing money at the problem would solve it.

Python has a large and diverse user base. That brings unique challenges that I don't see in other stacks.

Maybe we are all just ranting into the void when posting on HN. Are you ranting into the void as well? Where should we demark this classification's borders.

I felt the post you replied to was valuable, as it provided a general (but not grounded strictly in the facts of a specific example) outline of how these work.

I will corroborate the details. I learned Django 15 years ago, still use it, and use about 30% of its functionality. I don't think I do anything with it now I couldn't have in the earlier versions.

You got me there for a minute.

Pitch-perfect satire.

In the general case what you say is true. But look at the specific example of PEP 751 (the thread we’re in). Normally you’d designate some person who has to gain consensus and add the feature to python or to the standard library. Even if everyone isn’t on board, they’ll get on board when they upgrade python.

PEP 751 isn’t a python feature, it’s a feature that will be implemented by 3 projects - PDM, pip and uv. Consensus isn’t optional or nice to have, it’s necessary. If any of the 3 maintainers felt their needs weren’t met they wouldn’t have implemented it.

Some projects wait too long for consensus because they prioritise not rushing into a suboptimal solution or hurting people’s feelings. Sometimes it’s ok to just go ahead and implement something. Pep 751 is not one of those projects.

A governance model where “time-based decision thresholds” directly damaged veto power as time passes could be used to harness this sort of BDFL-only power in a consensus culture.

> Python was better with a BDFL.

The problem is that the BDFL also didn't want to think about packaging, certainly not the issues that inspired Conda.

I wouldn't mind seeing a PyPy-like fork done in Rust. Maybe take the opportunity to redesign the standard library, too.

doesn't opening a file with O_CREAT|O_EXCL work in posix?

> Or possibly it's just because a lot of the Python ecosystem doesn't even try and follow semver

Even if they try, semver can only ever be a suggestion of the possibility of compatibility at best because people are not oracles and they misjudge the effects of changes all the time.

That's why other spaces have machine tools for this. There seems to be an overall drift in Python to use more type annotations anyway; making a tool that compares 2 versions of a package isn't rocket science (maybe it already exists?)

Literally everyone posting here is using a system built on compatible interfaces. Stable DLLs on Windows, dylib framework versions on OSX, ELF SOversioning on Linux.

It's clearly not impossible, just a question of priorities and effort, and that makes it a policy decision to do it or not. And I'll lean towards we've been shifting that policy too far in the wrong direction.