Ned Batchelder's blog https://nedbatchelder.com/blog Ned Batchelder's personal blog. en-US Ned Batchelder's blog https://nedbatchelder.com/blog https://nedbatchelder.com/pix/rss-banner.gif Custom search keywords https://nedbatchelder.com/blog/202205/custom_search_keywords.html 2022-05-26T07:25:47-04:00 Ned Batchelder You can define custom search keywords in your browser to make common searches easier. How to do it isn’t well documented, so I’m showing you how here.

As an example, I have a custom keyword “pypi”, so I can search PyPI for a package name by typing “pypi some name” in my browser’s address bar, and it takes me directly to https://pypi.org/search/?q=some name.

You can create your own custom keywords to have shortcuts to all kinds of searches. You provide a URL with a placeholder %s in it. When you type the keyword in the address bar, the rest of the entry is plugged into the placeholder, and you jump to the resulting URL.

Handy keywords

These are some of the custom search keywords I use. You can use them as-is, adapt them as you need, or create your own. Instructions for creating keywords are further down the page.

py: Search Python docs
https://www.google.com/search?q=site%3Adocs.python.org%20%s
pep: Find a PEP by number
https://peps.python.org/pep-0%s
pypi: Find a PyPI package
https://pypi.org/search/?q=%s
gh: GitHub code search
https://github.com/search?type=Code&q=%s
gpy: Google search for Python things, but not Monty Python
http://www.google.com/search?q=%s+python+-monty
xlate: Translate text, auto-detecting the language
https://translate.google.com/?sl=auto&tl=en&op=translate&q=%s

How to define a keyword in Firefox

Firefox defines custom keywords as bookmarks:

  1. Select Bookmarks - Manage Bookmarks from the menu to open the Library dialog.
  2. In the left-hand sidebar, select Quick Searches. You’ll see a handful of pre-defined searches.
  3. Using the gear drop-down at the top of the dialog, select Add Bookmark...
  4. Enter the URL of the search in the URL field. Use %s as the placeholder.
  5. Put the keyword you want to use in the Keyword field.
  6. You can give the bookmark a name to make it easier to find it later if you need to.
  7. Save the bookmark, and close the Library.
  8. Now you can use your keyword in the address bar.

How to define a keyword in Chrome

Chrome defines custom keywords as search engines in Settings:

  1. Select Preferences or type chrome://settings in the address bar.
  2. In the left-hand sidebar, select Search engine and then Manage search engines and site search.
  3. Scroll down to Site search.
  4. Click Add.
  5. Put a description in the Search engine field.
  6. Enter the URL in the URL field, with %s as a placeholder.
  7. Put the keyword in the Shortcut field.
  8. Click Add, then close the Settings tab.
  9. Now you can use your keyword in the address bar.
]]>
Cairo in Jupyter, better https://nedbatchelder.com/blog/202205/cairo_in_jupyter_better.html 2022-05-15T15:32:15-04:00 Ned Batchelder I finally came up with a way I like to create PyCairo drawings in a Jupyter notebook.

A few years ago I wrote here about how to draw Cairo SVG in a Jupyter notebook. That worked, but wasn’t as convenient as I wanted. Now I have a module that manages the PyCairo contexts for me. It automatically handles the displaying of SVG and PNG directly in the notebook, or lets me write them to a file.

The module is drawing.py.

The code looks like this (with a sample drawing copied from the PyCairo docs):

from drawing import cairo_context


def demo():
    with cairo_context(200, 200, format="svg") as context:
        x, y, x1, y1 = 0.1, 0.5, 0.4, 0.9
        x2, y2, x3, y3 = 0.6, 0.1, 0.9, 0.5
        context.scale(200, 200)
        context.set_line_width(0.04)
        context.move_to(x, y)
        context.curve_to(x1, y1, x2, y2, x3, y3)
        context.stroke()
        context.set_source_rgba(1, 0.2, 0.2, 0.6)
        context.set_line_width(0.02)
        context.move_to(x, y)
        context.line_to(x1, y1)
        context.move_to(x2, y2)
        context.line_to(x3, y3)
        context.stroke()
    return context

demo()

Using demo() in a notebook cell will draw the SVG. Nice.

The key to making this work is Jupyter’s special methods _repr_svg_, _repr_png_, and a little _repr_html_ thrown in also.

The code is at drawing.py. I created it so that I could play around with Truchet tiles:

A multi-scale Truchet tiling

]]>
Twitter tidbits https://nedbatchelder.com/blog/202205/twitter_tidbits.html 2022-05-08T08:34:11-04:00 Ned Batchelder Lately I’ve been posting Python tidbits on Twitter. It’s been fun finding things that people might not yet know, fitting them into a tweet, and giving them some energy.

People ask how I make them. I tried a few “make a code screenshot” sites at first, but wanted more control over how the images looked. Now I create the code shots in vim with a tweaked color scheme (shades-of-purple). This lets me do things like highlight specific words I want to draw attention to. Then I use the Mac screenshot tool to grab an image:

An example of purpled code

The font is Recursive Mono SemiCasual. Recursive is a lovely and lively variable font with a number of axes to play with, especially the Casual axis, which I’ve set to halfway between Linear and Casual. You can get it now on Google Fonts. I’d like to try using Semi-Mono, but vim only understands monospaced fonts.

A sample of Recursive Mono, in Linear, SemiCasual, and Casual

Some people have asked about publishing these tidbits here on nedbatchelder.com also, but I’m not sure it’s worth the effort. Would it be helpful to have a page like this that collected them per-month or something? Or one tweet per page? Or just let them live on Twitter?

]]>
Python custom formatting https://nedbatchelder.com/blog/202204/python_custom_formatting.html 2022-04-14T19:35:54-04:00 Ned Batchelder Python f-strings use a formatting mini-language, the same as the older .format() function. After the colon comes short specifications for how to format the value:

>>> word = "Hello"

>>> f"{word:/^20}"
'///////Hello////////'
>>> amt = 12345678
>> f"{amt:20,}"
'          12,345,678'

Datetimes can use strftime syntax:

>>> f"{now:%Y-%m on day %d}"

'2022-04 on day 14'

The reason datetime uses different formatting specs than strings is because datetime defines its own __format__ method. Any object can define its own formatting mini-language. F-strings and .format() will use the __format__ method on an object, and pass it the formatting directives being used:

>>> class Confused:

...     def __format__(self, fmt):
...         return f"What is {fmt}?"
...
>>> c = Confused()
>>> f"{c:xyz12}"
'What is xyz12?'

Of course, __format__ can be used for more useful formatting than Confused is doing...

Geographic latitude and longitude are conventionally presented in a few different formats: degrees; or degrees and minutes; or degrees, minutes and seconds. Then the numbers can have varying number of decimal places, and sometimes the units are represented by symbols.

Here’s an implementation of those possibilities in __format__. The format string starts with “d”, “dm”, or “dms” to indicate the basic format. The number of decimal places can be specified with “.N”. Finally, symbols can be added, either plain or fancy, by adding a quote or minute symbol:

import dataclasses, re


@dataclasses.dataclass
class LatLong:
    lat: float
    long: float

    def __format__(self, fmt):
        dms, nfmt, opts = re.fullmatch(r"(dm?s?)([.\d]*)([′']?)", fmt).groups()
        formatted = []
        for num in [self.lat, self.long]:
            parts = []
            for ms in dms[1:]:
                parts.append(str(int(num)))
                num = abs((num - int(num)) * 60)
            parts.append(format(num, nfmt + "f"))
            syms = None
            if "'" in opts:
                syms = "°'\""
            elif "′" in opts:
                syms = "°′″"
            if opts:
                parts = [p + s for p, s in zip(parts, syms)]
            formatted.append(" ".join(parts))
        joined = ", ".join(formatted)
        return joined
>>> where = LatLong(42.359764937, -71.092068768)

>>> print(f"Location: {where:d'}")
Location: 42.359765°, -71.092069°
>>> print(f"Location: {where:d.4}")
Location: 42.3598, -71.0921
>>> print(f"Location: {where:dm'}")
Location: 42° 21.585896', -71° 5.524126'
>>> print(f"Location: {where:dms.4'}")
Location: 42° 21' 35.1538", -71° 5' 31.4476"
>>> print(f"Location: {where:dms.4}")
Location: 42 21 35.1538, -71 5 31.4476
>>> print(f"Location: {where:dms.6′}")
Location: 42° 21 35.153773, -71° 5 31.447565
>>> print("There: {:dms.6′}".format(where))
There: 42° 21 35.153773, -71° 5 31.447565
>>> print(format(where, "dms.6′"))
42° 21 35.153773, -71° 5 31.447565

This implementation doesn’t handle errors properly, but shows the basic idea. Also, lat/long are often shown with N/S E/W instead of positive and negative values. That’s left as an exercise for the reader.

]]>
Singleton is a bad idea https://nedbatchelder.com/blog/202204/singleton_is_a_bad_idea.html 2022-04-10T06:25:46-04:00 Ned Batchelder Design patterns are a great way to think about interactions among classes. But the classic Singleton pattern is bad: you shouldn’t use it and there are better options.

The classic Singleton pattern is a class which always gives you the same object when you create an instance of the class. It’s used to ensure that all users of a class are using the same object.

The first problem with Singleton is that it encourages you to mix together two different ideas into one class. The first idea is whatever your class is about. Let’s say you are writing a chess game. Your ChessBoard class should only be concerned with chess-board-ness. A separate second idea is that your program only needs one board. That’s not a fact intrinsic to chess boards, so it shouldn’t be coded into your ChessBoard class.

If your program only needs one of a certain object, you can just make one:

class ChessBoard:

    def __init__(self):
        ...

the_chess_board = ChessBoard()

If you want centralized management of the instance, use a function to manage a global:

_the_chess_board = None


def the_chess_board():
    global _the_chess_board
    if _the_chess_board is None:
        _the_chess_board = ChessBoard()
    return _the_chess_board

Or let functools do it for you:

@functools.cache        # new in 3.9

def the_chess_board():
    return ChessBoard()

If you still really want the class to manage a single instance, you can shift that function to be a classmethod:

class ChessBoard:

    def __init__(self):
        ...

    @classmethod
    @functools.cache
    def the_board(cls):
        return cls()

These ways are simpler than the usual gymnastics to implement singletons. They have the additional benefit that the users of the class clearly understand that they are accessing a global instance. This is another problem with Singleton: it gives you a class that lies. Calling the class looks like you are making new objects, but you are not.

These ways also let you still make new instances when you need to. Your ChessBoard tests will need to create many instances to keep your tests isolated from each other. The Singleton pattern makes this difficult and requires even more tricky machinations.

But I just need one!

So just make one. There’s no reason to complicate the class by adding Singleton enforcement to it.

But I need it everywhere in my code!

OK, use the_chess_board() wherever you were using ChessBoard().

But I still want a way to enforce “just one!”

The function manages the global instance. Why does it have to happen inside the class? You should separate the concept of what the class is from the idea that there should be only one.

But someone might make more instances!

Who? Document the right way to use the class. Make it clear and easy to do it the right way, and then let it be. You can’t defend against every bug, and it’s their program anyway.

But I thought globals were bad?

They are bad, but your Singleton was also a global: there was only one for the whole process, and it could be changed from anywhere. It wasn’t literally a Python global variable, but it had all the same bad qualities, just hidden behind some tricky meta-programming. If you’re going to have a global, be up-front about it.

But what about None and things like it?

True, some immutable value types can be singletons, but that’s not how people use the Singleton pattern, and how often are you writing a class like None?

But Singleton is in the Design Patterns book!

That doesn’t mean it’s a good idea. None of the examples in the book are true Singletons, they are all examples of programs that just happen to need only one instance. And the Design Patterns book was never meant to be prescriptive: it’s not telling you what to do, it’s describing what people have done.

]]>
Dinghy digests https://nedbatchelder.com/blog/202203/dinghy_digests.html 2022-03-13T07:29:40-04:00 Ned Batchelder Dinghy is a tool I wrote to summarize activity on GitHub issues and pull requests. You configure it to look at certain GitHub resources over a recent time period, and it produces a compact digest of what’s been happening:

A sample Dinghy digest

I started dinghy because I needed a way to see disparate activity in repos at work, and it’s been useful for that. So far it has the features I’ve needed. Maybe others will find it useful too.

It was also a chance to dig into the GitHub GraphQL API. GraphQL syntax is oddly strict (why can’t I define a fragment that isn’t used?), and GitHub’s implementation has quirks (sometimes requesting data with the wrong token scope is an error, other times it returns None). I can see the advantage of being able to request a graph of data with one request rather than a REST API that forces me to walk the graph myself, but paginating along multiple axes is still a pain (listing issues, and also comments on issues).

The code is still a mess in some places, and I haven’t figured out how to test most of it. But it was interesting to reverse engineer how GitHub decides on the text color for labels (they compute it in CSS!)

]]>
20 years of blogging https://nedbatchelder.com/blog/202203/20_years_of_blogging.html 2022-03-07T05:32:11-05:00 Ned Batchelder This blog is 20 years old today. My first post ever was My first job ever. 20 years later, this is my 2545th post.

It’s tempting to write a philosophical piece about the meaning of it all, and provide some grand perspective. But there is no overarching narrrative to this blog. It’s just been a place to write things and connect with people in various ways.

Here are some popular deep cuts you might have missed over the years:

Often I write about what’s going on with my autistic son Nat:

Of course, there’s lots of Python and other technical stuff:

Looking back 20 years at the early posts, many of the things I liked and linked to are gone. But for whatever reason, the puzzle-makers, geometers and type designers seem most durable. Here are some that are still around (and mostly look their age!):

I’m not sure what else to say about 20 years of blogging. I’m still doing it.

]]>
Why your mock still doesn’t work https://nedbatchelder.com/blog/202202/why_your_mock_still_doesnt_work.html 2022-02-18T07:54:47-05:00 Ned Batchelder In Why your mock doesn’t work, I explained that you have to mock objects where they are used, not where they are defined. Now we’ll talk about another reason your mock may be mysteriously failing to capture activity.

At work a test like this was failing:

# -- cool_feature.py ------------

import logging

logger = logging.getLogger(__name__)

def expensive_prep():
    ...

def do_cool_thing(text):
    expensive_prep()
    logger.info(f"Cool feature is doing {text!r}")

# -- test_cool_feature.py ------------
from unittest.mock import patch
from cool_feature import do_cool_thing

@patch("cool_feature.expensive_prep")
@patch("cool_feature.logger")
def test_cool_feature_announcement(mock_prep, mock_logger):
    do_cool_thing("hello")
    mock_logger.info.assert_called_once_with(
        "Cool feature is doing 'hello'"
    )

We couldn’t see why the log mock wasn’t getting called at all. The code was straightforward, the test was simple, how come the mock wasn’t catching the log call?

Mocks are slippery. That’s their strength and their weakness. They will happily let you call any method on them, or access any attribute, and when you do, they will give you another slippery mock that will let you do whatever you want. This can be very powerful, but also very misleading.

The problem with our test was the order of the mocks in the function signature. The first decorator corresponds to the last argument, because the first decorator is the last one applied. That’s the inside-out nature of decorators. It should be:

@patch("cool_feature.expensive_prep")

@patch("cool_feature.logger")
def test_cool_feature_announcement(mock_logger, mock_prep):

When the order was wrong, we called the logger, and then were asserting that “cool_feature.expensive_prep” had its “info” method called. That (non-existent) method hadn’t been called, so our assert failed.

We were using the mocks completely incorrectly, but mocks don’t mind: you can do whatever you want, and they let you, with no notification.

Once the order of the mocks in the signature was fixed, the test passed.

But there’s a way we could write this to be more fool-proof. Mocks can be made strict with their “spec=True” option. When set, the mock will raise an error if you use a method or attribute on it that didn’t exist on the object it was mocking.

Even better is the autospec=True option to mock.patch. This makes the mock strict with spec=True, and also recursively auto-applies spec=True to any new mock you get from the mock:

@patch("cool_feature.expensive_prep", autospec=True)

@patch("cool_feature.logger", autospec=True)
def test_cool_feature_announcement(mock_logger, mock_prep):

If we had been using autospec from the start, the incorrect test would have raised a helpful error right away:

AttributeError: 'function' object has no attribute 'info'

I wish autospec=True was the default. I can see why there are occasional needs for truly permissive mocks, but mostly we intend to use them in strict ways, and want to know when we don’t.

BTW, the usual advice to avoid mocking applies if you can, since even when you get your mocks working properly, there are still dangerous pitfalls. I’ll re-recommend these two resources about how to test differently:

]]>
Moving a git branch to a new base https://nedbatchelder.com/blog/202202/moving_a_git_branch_to_a_new_base.html 2022-02-10T08:32:33-05:00 Ned Batchelder Suppose you have some work on a git branch that you started from one branch, and you want to move that work to be based on a different branch, as if you had started from there originally. The git rebase command gives you the tools to do it, but it’s complicated, and I can never remember the details, so I finally figured it out and made an alias to do it.

In this post I’ll use an example where I’m working on branch “my-work”, which I started from “old-home” and I want to move it to “new-home”. My initial state looks like this:

% git log --oneline --decorate --graph --all
* a8ab80e (HEAD -> my-work) my work edit 2
* 5f66562 my work edit 1
| * 9682932 (old-home) old-home edit 3
|/
* 98d09e4 old-home edit 2
* a6fa334 old-home edit 1
| * e409e95 (new-home) new-home edit 2
| * 7d3ca09 new-home edit 1
|/
* 1b008d5 (main) main edit 2
* e9152aa main edit 1

The git tool to perform a move like this is “git rebase --onto”. The general form of the command needs three pieces of information, but the third is the branch to move, which defaults to the current branch, so we can just omit that. The other two items are: where you want to snip the current branch from, and where you want to graft it onto. The command looks like this:

git rebase --onto=<graft-onto> <snip-from> [<branch-to-move>]

In our example, we want to snip the branch from the point where it started on old-home. The “git merge-base” command can tell us the commit where two branches diverge, so it’s perfect for this:

% git merge-base old-home @
98d09e4b4d3ca45e3e03cf27386f7dd01f0662a8

Putting this together in one command looks like this:

% git rebase --onto new-home $(git merge-base old-home @)
Successfully rebased and updated refs/heads/my-work.

The result is just what we wanted:

% git log --oneline --decorate --graph --all
* df22106 (HEAD -> my-work) my work edit 2
* 9ac8707 my work edit 1
* e409e95 (new-home) new-home edit 2
* 7d3ca09 new-home edit 1
| * 9682932 (old-home) old-home edit 3
| * 98d09e4 old-home edit 2
| * a6fa334 old-home edit 1
|/
* 1b008d5 (main) main edit 2
* e9152aa main edit 1

But that command is complicated to type, so I made an alias in my .gitconfig:

[alias]
movebranch = "!f() { \
    : git checkout \
    git rebase --onto $2 $(git merge-base $1 @) \
}; f"

Now our branch move command would look like this:

% git movebranch old-home new-home
Successfully rebased and updated refs/heads/my-work.

And the result is the same:

% git log --oneline --decorate --graph --all
* df22106 (HEAD -> my-work) my work edit 2
* 9ac8707 my work edit 1
* e409e95 (new-home) new-home edit 2
* 7d3ca09 new-home edit 1
| * 9682932 (old-home) old-home edit 3
| * 98d09e4 old-home edit 2
| * a6fa334 old-home edit 1
|/
* 1b008d5 (main) main edit 2
* e9152aa main edit 1
]]>
Refactoring Boston https://nedbatchelder.com/blog/202201/refactoring_boston.html 2022-01-23T15:00:16-05:00 Ned Batchelder There’s a street in Boston called Broadway, but it doesn’t look like what you’d expect. It’s a quiet side street, about 200 feet long. This is the view down the length of it:

Broadway in Boston

What is going on? How did such a tiny thing get named Broadway? It turns out, the better question is, how did Broadway become such a tiny thing?

Like many cities, and probably more than most cities, Boston has undergone enormous change. When first settled in 1630, it was nearly an island, connected to the mainland by a narrow isthmus.

This 1826 map of Boston shows two streets relevant to our mystery, highlighted in yellow: Pleasant Street at the top heading up to Boston Common, and Broadway, part of South Boston at the bottom:

Boston in 1826

By 1917, much of the surrounding water has been landfilled (the 1826 map shows some planned streets), and Broadway from South Boston now runs into Pleasant Street, continuing up to Boston Common:

Boston in 1917

By 1938, Pleasant Street had been renamed, so the entire street from South Boston to Boston Common is now a major street named Broadway.

Some time in the 1970’s, after the extension of the Mass Pike to I-93, this part of Boston had a massive restructuring. Today it looks like this:

Boston today

Broadway is still a major street in South Boston at the bottom, but all that remains of the street in Boston proper is a small piece now called Park Place near Boston Common at the top, and our 200-foot street still named Broadway just below that. Refactoring a city can leave you with some strange little remnants.

Just to make it more confusing, of the three(!) street signs on the street, two say “Broadway”, and one says “Broadway St,” which doesn’t even make sense...

BTW, sorry for the changing scale and orientation in these maps. If you want to explore maps over time in a less disorienting way, mapjunction.com is fabulous.

]]>