Finding out what I liked and didn’t like wasn’t hard, since the “sophisticated sentiment analysis” was two regexes: “<nedbat>.* is good” and “<nedbat>.* is bad”!

Without further commentary, here is a sampling of things that I said were bad:

• vertical alignment is bad because it means you might have to change many lines just because one of them got wider.
• eval(input()) is bad.
• blindly following stuff is bad.
• trolling is bad. Find a way to use your brains for good.
• floats for currency is bad.
• any class that you can only instantiate once is bad.
• that sounds like a singleton, which is bad.
• some people say, “you should start by learning assembler,” and i think that is bad advice.
• del is fine. __del__ is bad
• implicit copying is bad
• texture (repetition you can see when you squint) is bad in code
• aligning indents with the opening delimiter is bad.
• this is bad: def main(nums=[1,2,3]).
• monkeypatching is bad
• it is bad to modify a list you are iterating.
• __import__ is bad
• import * is bad stuff
• the python doc search is bad
• and-or is bad
• singletons are hidden global state, and global state is bad
• checking types is bad
• that “:type myparam:” syntax is bad, it’s not readable. use google style instead: https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html#google-vs-numpy
• python is bad at recursion deeper than 1000
• there is a package on pypi called time, which is bad.

And here are some things I said were good:

• “python -m app_name” is good, but recent.
• endswith is good
• xpath is good at selecting nodes in an xml dom tree.
• recursion is good for recursive structures. Iteration is good for iterative structures
• madlibs is good because you can do string manipulation with puerile humor
• learning is good! :)
• python is good for full applications.
• re.sub is good.
• mock(spec=thing) is good
• excitement is good :)
• pip will know how to install into virtualenvs, which is good.
• gist.github.com is good
• argparse is good for simple things
• it is good to cover your tests, though others disagree with me
• textwrap.dedent is good
• requirements.txt is good for recreating environments.
• the csv module is good at writing out dicts as rows.
• this is good for seeing how rst will be formatted: http://rst.ninjs.org/
• colorama is good
• yaml is good
• setUp is good. tearDown is better done as addCleanup
• yield from is good for when you want one generator to produce all the values from another generator.
• duck typing is good when there’s an operation supported across a number of types, and you can just use the operation without worrying about the type.
• learning is good! :)
• Django is good if you like having lots of things handled for you. Flask is good if you like to put together all the pieces yourself.
• tox is good for testing against multiple environments
• for validating email addresses, this is good: [^@ ]+@[^@ ]+\.[^@ ]+ https://nedbatchelder.com/blog/200908/humane_email_validation.html
• the python.org tutorial is good if you have programmed in other languages before.
• “pip install -e .” is good
• the interactive interpreter is good for experimentation, but isn’t good for real development.
• bpaste.net is good
• Think Python is good
• atexit is good.
• python is good, and we are helpful and friendly :)
• pandas is good if you need to manipulate tables of data. If you don’t, then don’t use pandas
• you want to do something for each thing in a list, that’s what a for loop is good for :)
• there’s an old habit of using “:type:blah:” or whatever, which is horrible. Sphinx now supports the “napoleon” style natively, which is good: http://www.sphinx-doc.org/en/1.4.8/ext/napoleon.html
• pytest does cool assert rewriting, which 99.9999% of the time is good magic.
• pudb is good in the terminal
• trying to be efficient is good.
• sql is good for some kinds of data. nosql is good for others.
• .format is good
• a decorator is good for wrapping functions in new functionality.
• the pytest -k option is good at that.
• i would not try to jam everything into setup.py. this feels like something a makefile is good at.
• pytest is good at parameterized tests.
• “if not list:” is good python
• @classmethod is good for alternate constructors, yes.
• rg is good: https://github.com/BurntSushi/ripgrep
• the prompt is good for doing small experiments. Once you have larger programs, put them in .py files, and run them: python myprog.py
• low tech is good tech
• Learning is good.
• numpy is good when you can do whole-matrix operations at once. If you need to iterate over elements and do individual operations, it doesn’t provide any benefit.
• any is good when you have an iterable of true/false
• choice is good. Why should there be only one implementation?
• gist.github.com is good, or paste.pound-python.org
• click is good
• you’re not using a shell, which is good.
• whatever helps you learn is good
• numpy is good when you are doing matrix and array operations. lists are good for ordered collections of things
• obfuscation isn’t something Python is good at.
• the ast module is good for one thing: representing python programs as a tree of nodes. It provides tools for parsing source text into that tree.
• python is good as a first language
• isolation is good, but doing it with mocks can be a problem in itself
• subclassing is good for when SubClass, by its essence is a ParentClass.
• talking is good :)
• coverage.py is good.
• recursion is good for recursive structures (trees). iteration is better for linear structures (lists)
• Jupyter is good for visualizations, graphing, tables, etc. interactive experimentation
• lxml.html is good
• sha256 is good too
• Fluent Python is good, if you like books
• for speed, PyPy is good. Or Cython. if you want to write C code, you can use cffi to call it from Python
• curiosity is good
• collections.Counter is good at counting things, and would do this in O(N).
• .encode makes the conversion explicit, which is good. my_bytes = my_unicode.encode(“utf8”)
• they is good.
• pudb is good
• in python 3, super() is good, but it doesn’t work in python 2.
• the -k option is good for this, or you can define markers.
• virtualenv is good for separating different projects’ needs
• tig is good too
• rst is good at multi-page docs, without assuming it will be html. markdown just shrugs and says, “use html when you have to”
• writing is good just for its own sake.
• bpaste.net is good.
• learning is good
• dependencies are good. using other people’s solutions to your problems is good.
• split has a much better PR agency, but partition is good too
• attribute access is good.
• i won’t say that loops should introduce scopes, but it is good to be able to understand the interplay between scoping and closur-ing
• https://pypi.org/project/appdirs/ is good for answering that question
• automate the boring stuff is good. What kind of software will you be writing?
• iso8601 is good
• prompt_toolkit is good
• numpy is good when you have an array full of data, and you can do one operation that works on all of it at once

At first I thought, “that’s not always a good answer,” but then I started paying attention to how people around me responded, and sure enough, not only is it a good answer, but it’s almost always the answer people give.

It’s especially appropriate these days. Nat has been living with us through these COVID-19 times since the middle of March, and he is frustrated at how limited his days have become.

One of Nat’s favorite things is a three-week calendar showing what’s going to happen. It was a regular routine when he would visit on weekends from his group home: we’d sit down and update his calendar. Every day would be marked with where he’d wake up, what he would do during the day, and where he would go to bed. Unusual activities and special events in particular would be noted and recited. He would often sit and study the calendar, or would ask to review it with us.

When he first moved in for the lock-down, we tried updating his calendar, but the result was too depressingly accurate: every day was the same, and every day was at home. The only special event was Passover, which had been changed from in-person to Zoom:

Soon the weekly calendar was abandoned as uninteresting, and Nat started saying, “April,” meaning, I want it to be April when this will be over and I can go back to my regular life.

Of course, April came without a let-up of the lockdown, so he started saying, “May.” Now that we are in May, he says, “Summer.” We can’t give him a definite answer. The best we can do is to remind him that we have to wait, and that everyone he knows is also at home waiting.

I have been walking with Nat, a long-favored activity of ours. We’re up to about five miles a day, which is good for both of us.

My wife Susan has been handling most of the weekday activities other than the walks, trying to find things for Nat to do. They have been doing a lot of Facebook, a baking project most days, a little street basketball, and chores around the house.

She has taken to calling this “Suzie’s Little Day Program“:

Suzie’s Little Day Program pros and cons:

Pros: 1) Great staff-to-client ratio; 2) Lots of love; 3) Lots of napping; 4) Great treats; 5) Strong exercise component; 6) No ABA Whatsoever.

Cons: 1) Over-reliance on sugar; 2) Over-napping; 3) Not enough variety in peer group; 4) Moody staff; 5) Unreliable hours; 6) No ABA Whatsoever; 7) Often boring AF.

Overall, Nat is taking this very well. He has settled into this underwhelming routine. He dutifully wears his face mask on walks, and now knows to walk far around other people on the sidewalk without me prompting him. He likes getting on Zoom calls with the groups he is part of (MUSE foundation, Special Olympics, his day program), even if it’s just to watch because jumping in is difficult in those chaotic events.

We have gotten used to having him around full-time. There are 12 nearly empty bottles of shampoo in the shower (hard to explain). A few favorite Disney movies are on tight rotation. He keeps us a little more regimented, since ad-hoc is not his style.

I keep a close eye on him. He will not let us know if he starts to feel sick, so we have to be alert for him. He can be very passive, so it’s easy to feel guilty if he is staring into space or napping too much. We feel like we should be filling his time somehow, but it’s not possible to keep him busy ten hours a day.

Luckily, he has been in a calm period overall. There were other times in his life when these days would have been much stormier. We hope that his even temper continues. It’s been seven weeks so far, and we don’t know how much longer we are going to be together like this.

This is just our life right now. We’re all doing what we can. What’s going on? I’d have to say, not much.

•    •    •

This morning I tweeted out a link to my Lato’s unfortunate ligatures blog post. Łukasz Langa tweeted back at me:

OMG! Are images looking like that on my site and I didn’t know? (I’ve since fixed the problem.) Turns out it wasn’t just iPhones: Safari on desktop also showed stretched images like this! Why didn’t anyone tell me?

I could have done more thorough testing to begin with, but bugs will happen no matter what.

If you see something wrong, get in touch to let me know. As a creator, I want to make good things. If you can point me to a problem, you are helping me make better things. Even small things like dropped words or mispellings are good to report (see what I did there?).

I know it might feel like you are being a nit-picker, but it shows me that you care enough about what I do to respond. It’s a good thing. Probably other blog authors feel the same way.

Thanks in advance.

You form words by choosing letters in sequence. The only rule is that the next letter must be on a different side of the square than the previous letter:

Your next word has to start with the last letter of the previous word:

The goal is to use all the letters in a certain number of words. For this puzzle, the challenge is five words.

But if you look at the answers they provide for yesterday’s puzzle, it’s always just two words. So now I’m tormenting myself trying to find two-word solutions. And of course I started thinking about writing a program to find them.

I found a giant list of words and started hacking on some code. I wasn’t sure if I’d need some fancy tree structure for searching the solution space. I figured I would start simpler than that, and maybe it would work.

My strategy was to whittle down the list of words a few steps at a time. First, keep only the words formed from just the 12 letters in the puzzle. Then reduce the list to only those that can be formed following the “not same side” rule. Then find pairs of words that end and start with the same letter, and use all 12 letters:

import collections
import itertools
import sys

def print_sample(label, words, n=10):
    print(f"{label}: {len(words)} words: {', '.join(itertools.islice(words, n))}")

with open("words2.txt") as fwords:
    words = set(w.strip() for w in fwords)

print_sample("All words", words)

sides = sys.argv[1:]
alphabet = set("".join(sides))

words = {w for w in words if set(w) < alphabet}
print_sample("Only the letters", words)

numbered_sides = {c: i for i, side in enumerate(sides) for c in side}

def is_possible(word):
    for first, second in zip(word, word[1:]):
        if numbered_sides[first] == numbered_sides[second]:
            return False
    return True

possible = {w for w in words if is_possible(w)}
print_sample("Possible words", possible)

starts = collections.defaultdict(list)
for word in possible:
    starts[word[0]].append(word)

print("Solutions:")
for word1 in possible:
    last = word1[-1]
    for word2 in starts[last]:
        if set(word1 + word2) == alphabet:
            print(word1, word2)


This code is just stream-of-consciousness coding, intermixing running statements with function I needed. On some previous puzzles, it came up with way too many solutions. For “riu pgh lcs yao” it found 1002 pairs of words! Solutions like “hypacusia argol” are not satisfying...

My giant list of words has 466,551 words. I also have a smaller file with only 45,404 words. I refactored the code to make it more flexible. Now it will try the small word list first, and only go to the second larger list if there are no solutions:

import collections
import itertools
import sys

def print_sample(label, words, n=10):
    print(f"{label}: {len(words)} words: {', '.join(itertools.islice(words, n))}")


class LetterBoxed:
    def __init__(self, sides):
        self.sides = sides
        self.alphabet = set("".join(sides))
        self.numbered_sides = {c: i for i, side in enumerate(self.sides) for c in side}

    def is_possible(self, word):
        for first, second in zip(word, word[1:]):
            if self.numbered_sides[first] == self.numbered_sides[second]:
                return False
        return True

    def solutions(self, words):
        alpha_words = {w for w in words if set(w) < self.alphabet}
        print_sample("Using only the letters", alpha_words)
        possible = {w for w in alpha_words if self.is_possible(w)}
        print_sample("Possible words", possible)

        starts = collections.defaultdict(list)
        for word in possible:
            starts[word[0]].append(word)

        for word1 in possible:
            last = word1[-1]
            for word2 in starts[last]:
                if set(word1 + word2) == self.alphabet:
                    yield (word1, word2)

def main(sides):
    letter_boxed = LetterBoxed(sides)
    for wordfile in ["words.txt", "words2.txt"]:
        with open(wordfile) as fwords:
            words = set(w.strip() for w in fwords)
        print_sample("All words", words)
        solutions = list(letter_boxed.solutions(words))
        if not solutions:
            print("No solutions with these words")
            continue
        print(f"{len(solutions)} solutions:")
        for word1, word2 in solutions:
            print(word1, word2)
        break

if __name__ == "__main__":
    main(sys.argv[1:])


With this, “riu pgh lcs yao” finds five solutions from the short word list, using words I actually know, like “gracious splashy.”

Unfortunately, sometimes the official solution uses words that aren’t in even the enormous word list. A previous puzzle was “tub pxi snq oja”, and the solution offered was “juxtaposition niqab,” which is frustrating. Playing word puzzles inevitably brings you face to face with the differences between accepted word lists.

After I wrote my code, I found:

• Caleb Robinson’s blog post about his solution, which involved the fancier tree structures I avoided.
• Art Steinmetz’s blog post about both generating the puzzle and solving it, in R. He mentions the generalizations of different numbers of sides, and numbers of letters per side. I was amused to realize that my code doesn’t care how many sides or letters per side are used, so it works for any number.

Just for grins, I’m wondering if there’s some crazy way to abuse regexes to do most of the work. Too much time on my hands...

Beginners seem to ask this question when they are feeling daunted by the challenge before them. Maybe they are hoping for a helpful answer, but it seems like most answers will just be a jumping off point for feeling bad about their own progress.

Everyone learns differently. They learn from different sources, at different paces. Suppose you ask this question and someone answers “one month”? Will you feel bad about yourself because you’ve been at it for six weeks? Suppose they say, “ten years”? Now what do you think?

The question doesn’t even make sense in a way. What do we mean by “learn”? If you can write a number guessing game in Python, have you learned Python? Are we talking about basic familiarity, or deep memorization? Does something have to be second nature, or is it OK if you are still looking through the docs for details? “Learned” is not a binary state. There isn’t a moment where you don’t know Python, and then suddenly you do.

And what do we even mean by “Python”? Are we talking about the basic syntax, or do you need to be able to write a metaclass, a descriptor, and a decorator with arguments? Is it just the language, or also the standard library? How many of the 200+ modules in the standard library do you need to be familiar with? What about commonly used third-party libraries? Are we also including the skills needed to write large (10k lines) programs in Python? “Python” is a large and varied landscape, and you will be finding out new things about it for years and years.

Especially since it keeps changing! Python isn’t sitting still, so you will never be done “learning Python.” I have been using Python for more than 20 years, and been deeply involved with it for at least half that time. I thought I knew Python well, then they added “async”. I will have to figure that out one of these days...

Since Python is used in many different domains, the things you need to learn could be completely different from someone else. These days, lots of people are learning Python to get into data science. I don’t do data science. Here are more things I don’t know (taken from a random sampling of “libraries you should know” blog posts): TensorFlow, Scikit-Learn, Numpy, Keras, PyTorch, SciPy, Pandas, Matplotlib, Theano, NLTK, etc. How should I compare my learning to a data scientist’s?

My advice to beginners is: don’t compare your learning to other peoples’. Everyone learns differently, using different materials, at different speeds. Everyone has different definitions of “learn,” and of “Python.” Understand your goals and your learning style. Find materials that work for you. Study, and learn in your own way. You can do it.

Python is not broken. It’s just not a great language for writing pure functional programs. Python seemed broken to my friend in exactly the same way that a hammer seems broken to someone trying to turn a screw with it.

I understand his frustration. Once you have fully embraced the FP mindset, it is difficult to understand why people would write programs any other way.

I have not fully embraced the FP mindset. But that doesn’t mean that I can’t apply some FP lessons to my Python programs.

In discussions about how FP and Python relate, I think too much attention is paid to the tactics. For example, some people say, “no need for map/­filter/­lambda, use list comprehensions.” Not only does this put off FP people because they’re being told to abandon the tools they are used to, but it gives the impression that list com­pre­hensions are somehow at odds with FP constructs, or are exact replacements.

Rather than focus on the tactics, the important ideas to take from FP are strategies, including:

• Write small functions with no side-effects
• Don’t change existing data, make new data
• Combine functions to make larger functions

These strategies all lead to modularized code, free from mysterious action at a distance. The code is easier to reason about, debug, and extend.

Of course, languages that are built from the ground up with these ideas in mind will have great tools to support them. They have tactics like:

• Immutable data structures
• Rich libraries of higher-order functions
• Good support for recursion

Functional languages like Scheme, Clojure, Haskell, and Scala have these tools built-in. They are of course going to be way better for writing Functional programs than Python is.

FP people look at Python, see none of these tools, and conclude that Python can’t be used for functional programming. As I said before, Python is not a great language for writing purely function programs. But it’s not a lost cause.

Even without those FP tools in Python, we can keep the FP strategies in mind. Although list comprehensions are presented as the alternative to FP tools, they help with the FP strategies, because they force you to make new data instead of mutating existing data.

If other FP professionals are like my friend, they are probably saying to themselves, “Ned, you just don’t get it.” Perhaps that is true, how would I know? That doesn’t mean I can’t improve my Python programs by thinking Functionally. I’m only just dipping my toes in the water so far, but I want to do more.

For more thoughts about this:

• Gary Bernhardt: Boundaries, a PyCon talk that discusses Functional Core and Imperative Shell.
• If you want more Functional tools, there are third-party Python packages like:
  • pyrsistent, providing immutable data structures
  • pydash, providing functional tools
  • fnc, providing functional tools

This talk has been through a few iterations. In 2011, I gave a presentation at Boston Python about Getting Started Testing, based on the standard library unittest module. In 2014, I updated it and presented it at PyCon. Now I’ve updated it again, and will be presenting it at Boston Python.

The latest edition, Getting Started Testing: pytest edition, uses pytest throughout. It’s a little long for one evening of talking, but I really wanted to cover the material in it. I wanted to touch on not just the mechanics of testing, but the philosophy and central challenges as well.

I’m sure there are important things I left out, and probably digressions I could trim, but it’ll do. Thoughts welcome.

The video was in Turkish, so I couldn’t tell exactly what Samuel was saying, but I scrolled through the video, and sure enough, it was my entire talk, complete with illustrations by my son Ben.

Looking closer, the title slide had been modified:

(I’ve blurred Samuel’s specifics in this image, and Samuel is not his actual name. This post isn’t about Samuel, and I’m not interested in directing any more negative attention to him.)

Scrolling to the end of the talk, my last slide, which repeated my name and contact details, was gone. In its place was a slide promoting other videos featuring Samuel or his firm.

I felt like I had been forcibly elbowed off the stage, and Samuel was taking my place while trying to minimize my contributions.

In 2018, I did two things for this presentation: I wrote it, and I presented it at PyCon 2018. By far the most work was in the writing. It takes months of thinking, writing, designing, and honing to make a good presentation. In fact, of the two types of work, Samuel valued the writing most, since that is the part he kept. The reason this presentation attracted his attention, and why he wanted to present it himself, was because of its content.

“Originally presented by” is hardly the way to credit the author of a presentation, especially in small type while removing his name and leaving only a GitHub handle.

So I tweeted,

This is my talk from PyCon 2018, in its entirety, with my name nearly removed. It’s theft. I was not asked, and did not give permission.

Samuel apologized and took down the video. There were other tweets claiming that this was a pattern of Samuel’s, and that perhaps the apology would not be followed by changed behavior. But again, this post isn’t about Samuel.

This whole event got me thinking about people re-using my presentations.

I enjoy writing presentations. I like thinking about how to explain things. People have liked the explanations I’ve written. I like that they like them enough to want to show them to people.

But I’ve never thought much about how I would answer if someone asked me if they could present one of my talks. If people can use my talks to help strengthen their local community and up-skill their members, I want them to be able to. I am not interested in people using my talks to unfairly promote themselves.

I’m not sure re-using someone else’s presentation is a good idea. Wouldn’t it be better to write your own talk based on what you learned from someone else’s? But if people want to re-use a talk, I’d like to have an answer.

So here are my first-cut guidelines for re-using one of my talks:

1. Ask me if you can use a talk. If I say no, then you can’t.
2. Don’t change the main title slide. I wrote the presentation, my name should be on it. If you were lecturing about a novel, you wouldn’t hand out copies of the book with your name in place of the author’s.
3. Make clear during the presentation that I was the author and first presenter. A way to do that would be to include a slide about that first event, with links, and maybe even a screenshot of me from the video recording of the first event.
4. I include a bit.ly short-link and my Twitter handle in the footer of my slides. Leave these in place. We live in a social online world. I want to benefit from the connections that might arise from one of my presentations.
5. Keep my name and contact details prominent in the end slide.
6. If your video is posted online, include my name and the point about this being a re-run in the first paragraph of the description.

It would be great if my talks could get a broader reach than I can make happen all by myself. To be honest, I’m still not sure if it’s a good idea to present someone else’s talk, but it’s better to do it this way than the way that just happened.

Sounds great! sys.getsizeof has an appealing name, and the description in the docs seems really good:

sys.getsizeof(object)
Return the size of an object in bytes. The object can be any type of object. All built-in objects will return correct results [...]

But the fact is, sys.getsizeof is almost never what you want, for two reasons: it doesn’t count all the bytes, and it counts the wrong bytes.

The docs go on to say:

Only the memory consumption directly attributed to the object is accounted for, not the memory consumption of objects it refers to.

This is why it doesn’t count all the bytes. In the case of a dictionary, “objects it refers to” includes all of the keys and values. getsizeof is only reporting on the memory occupied by the internal table the dict uses to track all the keys and values, not the size of the keys and values themselves. In other words, it tells you about the internal bookkeeping, and not any of your actual data!

The reason my co-worker’s API responses was all the same size was because they were dictionaries with the same number of keys, and getsizeof was ignoring all the keys and values when reporting the size:

>>> d1 = {"a": "a", "b": "b", "c": "c"}
>>> d2 = {"a": "a"*100_000, "b": "b"*100_000, "c": "c"*100_000}
>>> sys.getsizeof(d1)
232
>>> sys.getsizeof(d2)
232


If you wanted to know how large all the keys and values were, you could sum their lengths:

>>> def key_value_length(d):
...     klen = sum(len(k) for k in d.keys())
...     vlen = sum(len(v) for v in d.values())
...     return klen + vlen
...
>>> key_value_length(d1)
6
>>> key_value_length(d2)
300003


You might ask, why is getsizeof like this? Wouldn’t it be more useful if it gave you the size of the whole dictionary, including its contents? Well, it’s not so simple. Data in memory can be shared:

>>> x100k = "x" * 100_000
>>> d3 = {"a": x100k, "b": x100k, "c": x100k}
>>> key_value_length(d3)
300003


Here there are three values, each 100k characters, but in fact, they are all the same value, actually the same object in memory. That 100k string only exists once. Is the “complete” size of the dict 300k? Or only 100k?

It depends on why you are asking about the size. Our d3 dict is only about 100k bytes in RAM, but if we try to write it out, it will probably be about 300k bytes.

And sys.getsizeof also reports on the wrong bytes:

>>> sys.getsizeof(1)
28
>>> sys.getsizeof("a")
50


Huh? How can a small integer be 28 bytes? And the one-character string “a” is 50 bytes!? It’s because Python objects have internal bookkeeping, like links to their type, and reference counts for managing memory. That extra bookkeeping is overhead per-object, and sys.getsizeof includes that overhead.

Because sys.getsizeof reports on internal details, it can be baffling:

>>> sys.getsizeof("a")
50
>>> sys.getsizeof("ab")
51
>>> sys.getsizeof("abc")
52
>>> sys.getsizeof("á")
74
>>> sys.getsizeof("áb")
75
>>> sys.getsizeof("ábc")
76
>>> face = "\N{GRINNING FACE}"
>>> len(face)
1
>>> sys.getsizeof(face)
80
>>> sys.getsizeof(face + "b")
84
>>> sys.getsizeof(face + "bc")
88


With an ASCII string, we start at 50 bytes, and need one more byte for each ASCII character. With an accented character, we start at 74, but still only need one more byte for each ASCII character. With an exotic Unicode character (expressed here with the little-used \N Unicode name escape), we start at 80, and then need four bytes for each ASCII character we add! Why? Because Python has a complex internal representation for strings. I don’t know why those numbers are the way they are. PEP 393 has the details if you are curious. The point here is: sys.getsizeof is almost certainly not the thing you want.

The “size” of a thing depends on how the thing is being represented. The in-memory Python data structures are one representation. When the data is serialized to JSON, that will be another representation, with completely different reasons for the size it becomes.

In my co-worker’s case, the real question was, how many bytes will this be when written as CSV? The sum-of-len method would be much closer to the right answer than sys.getsizeof. But even sum-of-len might not be good enough, depending on how accurate the answer has to be. Quoting rules and punctuation overhead change the exact length. It might be that the only way to get an accurate enough answer is to serialize to CSV and check the actual result.

So: know what question you are really asking, and choose the right tool for the job. sys.getsizeof is almost never the right tool.

• Paul Tol’s notes, which gives special consideration to color-blindness. He has some visual demonstrations that picked up my own slight color-blindness.
• Cynthia Brewer’s ColorBrewer, with interactive elements so you can create your own palette for your particular needs.

Color Palette Ideas is different: palettes based on photographs, but can also be a good source for ideas.

As an update to my ancient blog post about this same topic, Adobe Color and paletton both have tools for generating palettes in lots of over-my-head ways. And Color Synth Axis is still very appealing to the geek in me, though it needs Flash, and so I fear is not long for this world...