Why Python class syntax should be different

Saturday 25 May 2019

If you’ve used any programming language for a long enough time, you’ve found things about it that you wish were different. It’s true for me with Python. I have ideas of a number of things I would change about Python if I could. I’ll bore you with just one of them: the syntax of class definitions.

But let’s start with the syntax for defining functions. It has this really nice property: function definitions look like their corresponding function calls. A function is defined like this:

def func_name(arg1, arg2):

When you call the function, you use similar syntax: the name of the function, and a comma-separated list of arguments in parentheses:

x = func_name(12, 34)

Just by lining up the punctuation in the call with the same bits of the definition, you can see that arg1 will be 12, and arg2 will be 34. Nice.

OK, so now let’s look at how a class with base classes is defined:

class MyClass(BaseClass, AnotherBase):

To create an instance of this class, you use the name of the class, and parens, but now the parallelism is gone. You don’t pass a BaseClass to construct a MyClass:

my_obj = MyClass(...)

Just looking at the class line, you can’t tell what has to go in the parens to make a MyClass object. So “def” and “class” have very similar syntax, and function calls and object creation have very similar syntax, but the mimicry in function calls that can guide you to the right incantation will throw you off completely when creating objects.

This is the sort of thing that experts glide right past without slowing down. They are used to arcane syntax, and similar things having different meanings in subtly different contexts. And a lot of that is inescapable in programming languages: there are only so many symbols, and many many more concepts. There’s bound to be overlaps.

But we could do better. Why use parentheses that look like a function call to indicate base classes? Here’s a better syntax:

class MyClass from BaseClass, AnotherBase:

Not only does this avoid the misleading punctuation parallelism, but it even borrows from the English we use to talk about classes: MyClass derives from BaseClass and AnotherBase. And “from” is already a keyword in Python.

BTW, even experts occasionally make the mistake of typing “def” where they meant “class”, and the similar syntax means the code is valid. The error isn’t discovered until the traceback, which can be baffling.

I’m not seriously proposing to change Python. Not because this wouldn’t be better (it would), but because a change like this is impractical at this late date. I guess it could be added as an alternative syntax, but it would be hard to argue that having two syntaxes for classes would be better for beginners.

But I think it is helpful to try to see our familiar landscape as confused beginners do. It can only help with explaining it to them, and maybe help us make better choices in the future.

Tidelift

Monday 20 May 2019

I’m a firm believer that open source software is woefully under-supported. The value people get from using open source far far far exceeds the resources they collectively put into the open source ecosystem.

There have been attempts to improve this situation, but they usually are some form of internet tip jar that goes nowhere. Businesses won’t put money in tip jars because they don’t know what to contribute to (they have hundreds of open source dependencies), and as infuriating as it is, they wonder what they get for that money (they already got the software!!)

Tidelift is approaching the problem of sustainable open source differently: what help do enterprises need with open source? What services would they be willing to pay for? How can enterprises be connected with open source maintainers to benefit both?

Tidelift logo

They sell the Tidelift Subscription, a collection of tools, information, and assurances to close some of the gaps businesses typically face when using open source.

The people behind Tidelift have deep experience at the intersection of open source and enterprises, having come from Red Hat, Gnome, and Mozilla. They’ve thought a lot about the problem of open source sustainability from both sides, and know what they are doing.

Coverage.py is part of the Tidelift Subscription, which makes me “a Lifter.” I get a small but not insignificant amount of money each month as a result. I want Tidelift to succeed partly for myself, but more importantly, because it could mean that open source is more sustainable overall.

If you are an open source maintainer, take a look at whether you can make money from Tidelift. What they ask of you is pretty much what well-maintained projects already do (good release notes, accurate metadata, points of contact), and they can help with some things that are difficult, like security reporting and license compliance.

If your company uses open source, consider whether the subscription is something you would use. It could help your business, and it would definitely help open source.

Thanks.

Coverage.py 5.0a5: pytest contexts

Monday 13 May 2019

Development of version 5 of coverage.py is going slowly, but it is progressing. The latest alpha is out: coverage.py 5.0a5.

The biggest changes are due to Stephan Richter and Justas Sadzevičius, from Shoobox. They improved the support for recording dynamic contexts, informally known as Who Tests What.

Now third-party code, either as a coverage.py plugin or using the coverage.py API can set the dynamic context.

I’ve added support for this to the pytest-cov plugin, to record the pytest test id as the dynamic context. If you’d like to try it:

pip install coverage==5.0a5
pip install git+https://github.com/nedbat/pytest-cov.git@nedbat/contexts
pytest --cov=. --cov-context

The .coverage data file is now a SQLite database. Coverage.py has no support yet for using the collected context data, but you can examine the raw data in the database:

$ sqlite3 .coverage
SQLite version 3.19.3 2017-06-27 16:48:08
Enter ".help" for usage hints.

sqlite> select * from context;
id          context
----------  --------------------------------------------------
1
2           test_it.py::test_prod1|setup
3           test_it.py::test_prod1|call
4           test_it.py::test_prod1|teardown
5           test_it.py::test_prod2|setup
6           test_it.py::test_prod2|call
7           test_it.py::test_prod2|teardown
8           test_it.py::test_prod3[1-1]|setup
9           test_it.py::test_prod3[1-1]|call
10          test_it.py::test_prod3[1-1]|teardown
11          test_it.py::test_prod3[10-100]|setup
12          test_it.py::test_prod3[10-100]|call
13          test_it.py::test_prod3[10-100]|teardown
14          test_it.py::test_prod3[11-121]|setup
15          test_it.py::test_prod3[11-121]|call
16          test_it.py::test_prod3[11-121]|teardown

sqlite> select * from arc where context_id = 9;
file_id     context_id  fromno      tono
----------  ----------  ----------  ----------
1           9           -14         15
1           9           15          16
1           9           16          17
1           9           17          -14

sqlite> select * from file where id = 1;
id          path
----------  --------------------------------------------------
1           /Users/ned/lab/pytest_func_test/src/product.py

I’m looking for feedback about what kinds of reporting would be useful. Stephan has a pull request to provide some context-based reporting. Does it do what you want? Have you used contexts? What needs to happen before they are ready for everybody?

Startup.py

Tuesday 16 April 2019

Someone recently asked how to permanently change the prompt in the Python interactive REPL. The answer is you can point the PYTHONSTARTUP environment variable at a Python file, and that file will be executed every time you enter the interactive prompt.

I use this to import modules I often want to use, define helpers, and configure my command history.

In my .bashrc I have:

export PYTHONSTARTUP=~/.startup.py

Then my .startup.py file is:

# Ned's startup.py file, loaded into interactive python prompts.
# Has to work on both 2.x and 3.x

print("(.startup.py)")

import collections, datetime, itertools, math, os, pprint, re, sys, time
print("(imported collections, datetime, itertools, math, os, pprint, re, sys, time)")

pp = pprint.pprint

# A function for pasting code into the repl.
def paste():
    import textwrap
    exec(textwrap.dedent(sys.stdin.read()), globals())

# Readline and history support
def hook_up_history():
    try:
        # Not sure why this module is missing in some places, but deal with it.
        import readline
    except ImportError:
        print("No readline, use ^H")
    else:
        import atexit
        import os
        import rlcompleter

        history_path = os.path.expanduser(
            "~/.pyhistory{0}".format(sys.version_info[0])
        )

        def save_history(history_path=history_path):
            import readline
            readline.write_history_file(history_path)

        if os.path.exists(history_path):
            readline.read_history_file(history_path)

        atexit.register(save_history)

# Don't do history stuff if we are IPython, it has its own thing.
is_ipython = 'In' in globals()
if not is_ipython:
    hook_up_history()

# Get rid of globals we don't want.
del is_ipython, hook_up_history

A few things could us an explanation. The paste() function lets me paste code into the REPL that has blank lines in it, or is indented. Basically, I can copy code from somewhere, and use paste() to paste it into the prompt without having to fix those things first. Run paste(), then paste the code, then type an EOF indicator (Ctrl-D or Ctrl-Z, depending on your OS). The pasted code will be run as if it had been entered correctly.

The history stuff gives me history that persists across Python invocations, and keeps the Python 2 history separate from the Python 3 history. “pp” is very handy to have as a short alias.

Of course, you can put anything you want in your own .startup.py file. It’s only run for interactive sessions, not when you are running programs, so you don’t have to worry that you will corrupt important programs.

Cog 3.0

Tuesday 2 April 2019

Cog is a small tool I wrote years ago. It finds snippets of Python in text files, executes them, and inserts the result back into the text. It’s good for adding a little bit of computational support into an otherwise static file. Originally I wrote it to generate boilerplate C code, but now I use it for making all my presentations.

It has sat dormant for a long time. Recently someone asked me if it was maintained, and I huffily answered, “it’s maintained as much as it needs to be.” But they were right to ask: it certainly had the look of an abandoned property.

So I jumped in and did a bunch of work to it. Development moved from Bitbucket to GitHub. I merged a few pull requests. I added Travis and Appveyor CI.

The biggest functional change is that errors during execution now get reasonable tracebacks that don’t require you to reverse-engineer how cog ran your code.

I even used mutmut to add a few more tests.

The result is Cog 3.0, ready for your use! I’ll try to stay on top of it better now, I promise!

Circle of Mad Libs

Monday 11 March 2019

Sometimes you find an unexpected real-world connection even in the geekiest of places. I (nedbat) was hanging out in the #python IRC channel on Freenode, and I recommended to someone that they write a Mad Libs game for a project.

Calvin Spealman (aka ironfroggy) chimed in:

[ironfroggy] didn't you write a madlibs python blog post like... forever ago?
    [nedbat] yes :)  14 years ago I think.
    [nedbat] my son was 13, and he just turned 27...
[ironfroggy] nedbat: fun fact: i read that when my wife was pregnant.
[ironfroggy] my son turns 13 in a few weeks.
[ironfroggy] we make games together now
    [nedbat] :) i like the symmetry

My post from 14 years ago is Programming madlibs, written based on a project I did with my then 13-year-old. To think that Calvin read it on the brink of becoming a father, and now has a son the same age that mine was then, is mind-bending.

It’s kind of like a circle of life or something, but I guess it’s just a circle of Mad Libs, which is still good.

Older:

Mar 2:

Mutmut

Jan 1:

Advice