A year or so ago, I couldn’t find a step-by-step guide to packaging a Python project that didn’t get bogged down in confusing options and choices, so I wrote my own: pkgsample. After I wrote it, I found the PyPA Packaging Python Projects tutorial, which is very good, so I never made a post here about my sample.
Since then, I’ve shown my sample to people a number of times, and they liked it, so I guess it’s helpful. Here’s what I wrote about it back when I first created it:
• • •
The Python packaging world is confusing. There are decades of history and change. There are competing tools, with new ones arriving frequently. I don’t want to criticize anyone, let’s just take it as a fact of life right now.
But I frequently see questions from people who have written some Python code, and would like to get it packaged. They have a goal in mind, and it is not to learn about competing tools, intricate standards, or historical artifacts. They are fundamentally uninterested in the mechanics of packaging. They just want to get their code packaged.
There are lots of pages out there that try to explain things, but they all seem to get distracted by the options, asking our poor developer to choose between alternatives they don’t understand, with no clear implications.
I’m also not criticzing the uninterested developer. I am that developer! I don’t know what all these things are, or how they compete and overlap: build, twine, hatch, poetry, flit, wheel, pdm, setuptools, distutils, pep517, shiv, etc, etc.
I just want someone to tell me what to do so my code will install on users’ machines. Once that works, I can go back to fixing bugs, adding features, writing docs, and so on.
So I wrote pkgsample to be the instructions I couldn’t find. It’s simple and stripped down, and does not ask you to make choices you don’t care about. It tells you what to do. It gives you one way to make a simple Python package that works right now. It isn’t THE way. It’s A way. It will probably work for you.
Comments
Unfortunately, it seems that the PyPA packaging guide has headed down the path of “confusing options and choices”.
Two years ago (https://web.archive.org/web/20220130185127/https://packaging.python.org/en/latest/tutorials/packaging-projects/), it just used setuptools.
Two days ago (https://web.archive.org/web/20240208104845/https://packaging.python.org/en/latest/tutorials/packaging-projects/#choosing-a-build-backend), it is telling readers to choose one of four build backends, with no guidance other than saying the tutorial is now using Hatchling by default in its own examples.
As a Django developer, I’m particularly interested in how to include non-Python files in my package (e.g. .css files, templates, etc). Two years ago, the tutorial linked to setuptools’ instructions for doing that. Now, it just says “If you want to include additional files, see the documentation for your build backend.”, and leaves it to the reader to figure out where that is in the documentation of the build backend they’re using. They don’t even link to the docs for the build backend the tutorial is using by default in their examples now.
I think you have the right idea in showing one project that works, and not introducing any unnecessary choices.
Hey Ned, thanks for writing this post and the
pkgsampletemplate/guidance! It looks like a very useful launching point.I also found the packaging space challenging and worth having a mildly-opinionated working path for. Just to offer some more knowledge for the common pool here, I want to offer that I wrote Publishing Python Packages for Manning Books on the subject. Like you say, the book doesn’t cover _the_ way, but at least gives _a_ way, with hopefully enough education that people can swap out different parts as they like. It goes through a specific, quick(-ish) path while building some mental models about each of the moving parts along the way. I also have a code companion that is ultimately similar in purpose to
pkgsample.I’d be happy to ship a hard copy to any readers here who would like one (up to the number of them that I still have), free of cost.
Thanks again for sharing your learnings!
This is a nice starting point. If I was creating something like this, I’d recommend Poetry, since it handles dependency locking and virtualenv creation for you. It also allows you to specify your dev dependencies right along side your main deps in pyproject.toml and automatically generates classifiers.
I feel like a major place where people get hung up creating packages is with the virtualenv step, and this sample kind of punts on that.
Of course, using Poetry requires installing the tool, which is another place where people get stuck. Do you use pip, pipx, etc…?
This is a reason I was (am) a proponent of PEP-582. Given that PEP seems dead for now, Poetry or PDM seem like the best, most popular “full service” packaging tools currently.
I’d also go with README.md, as it’s easier to write and seems to be much more common than README.rst. I’d also suggest that CHANGELOG.(md|rst) should be included by default as a best practice.
Hi Ned. Would you use something like setuptools_scm for versioning?
Hi Dane. I love your book. I maintain a few dozen internal package at my employer and your guidance was invaluable. Unfortunately I gave mine to a coworker whose contract for terminated before he could return it and I lost contact with him and, given the circumstances, I didn’t have the heart to looking for it. How should I reach out to you, if you have any copies available? Thanks in advance.
@Golam: I prefer to use simpler tools, and I don’t mind updating things by hand. I don’t use setuptools_scm, though I know many people like it. This sample in particular is meant to be as simple as possible.
Thanks for the feedback, @Golam! You can find me by name pretty readily on LinkedIn or any of Twitter/Mastodon/Bluesky so DM me wherever works best for you and I’ll get you sorted. I should’ve included that in my original comment—I definitely don’t mean to hijack here, @Ned, so thanks for your patience with me!
@Dan Poirier: your PyPUG feedback might be useful @ https://github.com/pypa/packaging.python.org/issues/new. People have diverse opinions and that sometimes results in those changes happening to the guides. Currently, the PyPUG is trying to follow diataxis, and with that, tutorials should demonstrate a happy path, yes. Just recently we rejected an attempt to add even more examples to that list, and another change reshuffling it based on somebody’s perception of popularity 🤷♂️ If you’re up to it, you can help improve it. Perhaps, stating explicitly that the choice is unnecessary would be an option. The current state is the result of people blaming the PyPA for favoritism, which is why we’re trying to communicate that it’s impossible to bless one tool to rule them all. There’s always going to be someone unhappy with something. Anyway, such a topic would better fit a discussion document type, rather than a tutorial.
Add a comment: