Secure maintainer workflow

Monday 21 November 2022

I’m trying to establish a more secure workflow for maintaining public packages.

Like most developers, I have terminal sessions with implicit access to credentials. For example, I can make git commits and push to GitHub without having to type a password.

There are two ways this might be a problem. The first is unlikely: a bad guy gets onto my computer and uses the credentials to cause havoc. This is unlikely mostly because a bad guy won’t get my computer, but also, if it does fall into the wrong hands, it will probably be someone looking to resell the laptop, not use my coverage.py credentials maliciously.

The second way is a more serious concern: I could unknowingly run evil or buggy code that uses my credentials in bad ways. People write bug reports for coverage.py, and if I am lucky, they include steps to reproduce the problem. Sometimes the instructions involve small self-contained examples, and I can just run them without fear. But sometimes the steps are clone this repo, and run this large test suite. It’s impossible to review all of that code. I don’t know what the code will do, but if I want to see and diagnose the problem, I have to run it.

I’m trying to reduce the possibilities for bad outcomes, in a few ways:

1Password: where possible, I store credentials in 1Password, and use tooling to get them into environment variables. I have two shell functions (opvars / unopvars) that find values in a vault based on the current directory, and can set and unset them in the environment.

With this, I can have the credentials in the environment for just long enough to use them. This works well for things like PyPI credentials, which are used rarely and could cause significant damage.

But I still also have implicit credentials in my ~/.ssh directory and ~/.netrc file. I’m not sure the best approach to keep them from being available to programs that shouldn’t have them.

Docker: To really isolate unknown code, I use a Docker container. I start with a base image with many versions of Python: base.dockerfile, and then build on it to create a main image that doesn’t even have sudo. In the container, there are no credentials, so I don’t have to worry about malice or accidents. For involved debugging, I might write another Dockerfile FROM these to reduce the re-work that has to happen when starting over.

What else can I be doing to keep safe?

UPDATE: there is more about this in a follow-on post: Secure maintainer workflow, continued.

Comments

Vince 7:57 AM on 21 Nov 2022

You could piggyback on your 1Password workflow to export extra ssh config for git or even to store keys and add them to your shell session using ssh-agent.

Allison Karlitskaya 8:55 AM on 21 Nov 2022

I spend an awful lot of time thinking about exactly this scenario of typing git pull && make all day long and how dangerous this is.

I’ve also started storing secrets (github tokens, and the like) in Bitwarden (which has a good commandline tool).

Two additional suggestions:

my main developer account is not an admin on my laptop. That means that I can’t run sudo from this account. I assume that anyone who is capable of getting “evil” code into my user account can also capture my keystrokes, and would then be able to run sudo for themselves, so I just prevent this scenario entirely. I have a separate admin account that I use in case I need it. I develop inside of toolbox (which I destroy and rebuild regularly) so I don’t usually need to install extra packages on my system, so I find that I use the administrator account very rarely.
I have a Yubikey Nano permanently inside of one of my USB ports, and I use that for as much as I can. One big use-case is for the SSH key to push to GitHub. Each push requires entering a (short) PIN and physically touching the key. The PIN protects against people who would randomly physically find my Yubikey, and the physical touch (mostly) mitigates evil code in my account. In order not to drive myself crazy when cloning/fetching/pulling, I use HTTPS for read-only operations via the url.pushInsteadOf git configuration option.

[url "ssh://git@github.com/"]
        pushInsteadOf = https://github.com/
        pushInsteadOf = github:

[url "https://github.com/"]
        insteadOf = github:

As a side effect that gives a nice slightly-shorter way to clone from github, like git clone github:nedbat/coveragepy.

Dirkjan Ochtman 8:57 AM on 21 Nov 2022

On my Mac I use Secretive, which keeps my SSH key in the secure enclave and requires authentication with TouchID every time it is used.

Remi Rampin 10:11 AM on 21 Nov 2022

If using docker, you should use both --user <non-0> and --cap-drop ALL. This will prevent processes from getting privileges back, for example by running a setuid binary that you left in there (or was sneaked in by the untrusted input).

Then you don’t need to remove sudo and similar from the image, they won’t work.

Kushal Das 10:48 AM on 21 Nov 2022

1Password can do ssh for you.
Instead of docker please try podman once, it comes with more security as default.

Though my personal favorite option is to move things hardware tokens (read Yubikeys) for ssh and webauthn as much as possible.

Brett Cannon 1:20 AM on 22 Nov 2022

To pile on to the “1Password for SSH” bandwagon, I did this a couple of months ago and it’s worked out. 1Password will prompt you when something tries to access their ssh-agent for the first time, so it won’t spontaneously start sharing.

Another way to lock things down is require all changes to go through a PR. That way if your termiy is compromised the best they can do is push a PR to your repo instead of committing. Same goes for using something like GitHub Actions for pushing to PyPI (going as far as requiring a workflow requiring approval before running).

Secure maintainer workflow

Comments

Add a comment: