As an example, I have a custom keyword “pypi”, so I can search PyPI for a package name by typing “pypi some name” in my browser’s address bar, and it takes me directly to https://pypi.org/search/?q=some name.
You can create your own custom keywords to have shortcuts to all kinds of searches. You provide a URL with a placeholder %s in it. When you type the keyword in the address bar, the rest of the entry is plugged into the placeholder, and you jump to the resulting URL.
These are some of the custom search keywords I use. You can use them as-is, adapt them as you need, or create your own. Instructions for creating keywords are further down the page.
Firefox defines custom keywords as bookmarks:
Chrome defines custom keywords as search engines in Settings:
A few years ago I wrote here about how to draw Cairo SVG in a Jupyter notebook. That worked, but wasn’t as convenient as I wanted. Now I have a module that manages the PyCairo contexts for me. It automatically handles the displaying of SVG and PNG directly in the notebook, or lets me write them to a file.
The module is drawing.py.
The code looks like this (with a sample drawing copied from the PyCairo docs):
from drawing import cairo_context
def demo():
with cairo_context(200, 200, format="svg") as context:
x, y, x1, y1 = 0.1, 0.5, 0.4, 0.9
x2, y2, x3, y3 = 0.6, 0.1, 0.9, 0.5
context.scale(200, 200)
context.set_line_width(0.04)
context.move_to(x, y)
context.curve_to(x1, y1, x2, y2, x3, y3)
context.stroke()
context.set_source_rgba(1, 0.2, 0.2, 0.6)
context.set_line_width(0.02)
context.move_to(x, y)
context.line_to(x1, y1)
context.move_to(x2, y2)
context.line_to(x3, y3)
context.stroke()
return context
demo()
Using demo()
in a notebook cell will draw the SVG. Nice.
The key to making this work is Jupyter’s special methods _repr_svg_, _repr_png_, and a little _repr_html_ thrown in also.
The code is at drawing.py. I created it so that I could play around with Truchet tiles:
]]>People ask how I make them. I tried a few “make a code screenshot” sites at first, but wanted more control over how the images looked. Now I create the code shots in vim with a tweaked color scheme (shades-of-purple). This lets me do things like highlight specific words I want to draw attention to. Then I use the Mac screenshot tool to grab an image:
The font is Recursive Mono SemiCasual. Recursive is a lovely and lively variable font with a number of axes to play with, especially the Casual axis, which I’ve set to halfway between Linear and Casual. You can get it now on Google Fonts. I’d like to try using Semi-Mono, but vim only understands monospaced fonts.
Some people have asked about publishing these tidbits here on nedbatchelder.com also, but I’m not sure it’s worth the effort. Would it be helpful to have a page like this that collected them per-month or something? Or one tweet per page? Or just let them live on Twitter?
]]>>>> word = "Hello"
>>> f"{word:/^20}"
'///////Hello////////'
>>> amt = 12345678
>> f"{amt:20,}"
' 12,345,678'
Datetimes can use strftime syntax:
>>> f"{now:%Y-%m on day %d}"
'2022-04 on day 14'
The reason datetime uses different formatting specs than strings is because datetime defines its own __format__ method. Any object can define its own formatting mini-language. F-strings and .format() will use the __format__ method on an object, and pass it the formatting directives being used:
>>> class Confused:
... def __format__(self, fmt):
... return f"What is {fmt}?"
...
>>> c = Confused()
>>> f"{c:xyz12}"
'What is xyz12?'
Of course, __format__ can be used for more useful formatting than Confused is doing...
Geographic latitude and longitude are conventionally presented in a few different formats: degrees; or degrees and minutes; or degrees, minutes and seconds. Then the numbers can have varying number of decimal places, and sometimes the units are represented by symbols.
Here’s an implementation of those possibilities in __format__. The format string starts with “d”, “dm”, or “dms” to indicate the basic format. The number of decimal places can be specified with “.N”. Finally, symbols can be added, either plain or fancy, by adding a quote or minute symbol:
import dataclasses, re
@dataclasses.dataclass
class LatLong:
lat: float
long: float
def __format__(self, fmt):
dms, nfmt, opts = re.fullmatch(r"(dm?s?)([.\d]*)([′']?)", fmt).groups()
formatted = []
for num in [self.lat, self.long]:
parts = []
for ms in dms[1:]:
parts.append(str(int(num)))
num = abs((num - int(num)) * 60)
parts.append(format(num, nfmt + "f"))
syms = None
if "'" in opts:
syms = "°'\""
elif "′" in opts:
syms = "°′″"
if opts:
parts = [p + s for p, s in zip(parts, syms)]
formatted.append(" ".join(parts))
joined = ", ".join(formatted)
return joined
>>> where = LatLong(42.359764937, -71.092068768)
>>> print(f"Location: {where:d'}")
Location: 42.359765°, -71.092069°
>>> print(f"Location: {where:d.4}")
Location: 42.3598, -71.0921
>>> print(f"Location: {where:dm'}")
Location: 42° 21.585896', -71° 5.524126'
>>> print(f"Location: {where:dms.4'}")
Location: 42° 21' 35.1538", -71° 5' 31.4476"
>>> print(f"Location: {where:dms.4}")
Location: 42 21 35.1538, -71 5 31.4476
>>> print(f"Location: {where:dms.6′}")
Location: 42° 21′ 35.153773″, -71° 5′ 31.447565″
>>> print("There: {:dms.6′}".format(where))
There: 42° 21′ 35.153773″, -71° 5′ 31.447565″
>>> print(format(where, "dms.6′"))
42° 21′ 35.153773″, -71° 5′ 31.447565″
This implementation doesn’t handle errors properly, but shows the basic idea. Also, lat/long are often shown with N/S E/W instead of positive and negative values. That’s left as an exercise for the reader.
]]>The classic Singleton pattern is a class which always gives you the same object when you create an instance of the class. It’s used to ensure that all users of a class are using the same object.
The first problem with Singleton is that it encourages you to mix together two different ideas into one class. The first idea is whatever your class is about. Let’s say you are writing a chess game. Your ChessBoard class should only be concerned with chess-board-ness. A separate second idea is that your program only needs one board. That’s not a fact intrinsic to chess boards, so it shouldn’t be coded into your ChessBoard class.
If your program only needs one of a certain object, you can just make one:
class ChessBoard:
def __init__(self):
...
the_chess_board = ChessBoard()
If you want centralized management of the instance, use a function to manage a global:
_the_chess_board = None
def the_chess_board():
global _the_chess_board
if _the_chess_board is None:
_the_chess_board = ChessBoard()
return _the_chess_board
Or let functools do it for you:
@functools.cache # new in 3.9
def the_chess_board():
return ChessBoard()
If you still really want the class to manage a single instance, you can shift that function to be a classmethod:
class ChessBoard:
def __init__(self):
...
@classmethod
@functools.cache
def the_board(cls):
return cls()
These ways are simpler than the usual gymnastics to implement singletons. They have the additional benefit that the users of the class clearly understand that they are accessing a global instance. This is another problem with Singleton: it gives you a class that lies. Calling the class looks like you are making new objects, but you are not.
These ways also let you still make new instances when you need to. Your ChessBoard tests will need to create many instances to keep your tests isolated from each other. The Singleton pattern makes this difficult and requires even more tricky machinations.
So just make one. There’s no reason to complicate the class by adding Singleton enforcement to it.
OK, use
the_chess_board()
wherever you were using
ChessBoard()
.
The function manages the global instance. Why does it have to happen inside the class? You should separate the concept of what the class is from the idea that there should be only one.
Who? Document the right way to use the class. Make it clear and easy to do it the right way, and then let it be. You can’t defend against every bug, and it’s their program anyway.
They are bad, but your Singleton was also a global: there was only one for the whole process, and it could be changed from anywhere. It wasn’t literally a Python global variable, but it had all the same bad qualities, just hidden behind some tricky meta-programming. If you’re going to have a global, be up-front about it.
True, some immutable value types can be singletons, but that’s not how people use the Singleton pattern, and how often are you writing a class like None?
That doesn’t mean it’s a good idea. None of the examples in the book are true Singletons, they are all examples of programs that just happen to need only one instance. And the Design Patterns book was never meant to be prescriptive: it’s not telling you what to do, it’s describing what people have done.
]]>I started dinghy because I needed a way to see disparate activity in repos at work, and it’s been useful for that. So far it has the features I’ve needed. Maybe others will find it useful too.
It was also a chance to dig into the GitHub GraphQL API. GraphQL syntax is oddly strict (why can’t I define a fragment that isn’t used?), and GitHub’s implementation has quirks (sometimes requesting data with the wrong token scope is an error, other times it returns None). I can see the advantage of being able to request a graph of data with one request rather than a REST API that forces me to walk the graph myself, but paginating along multiple axes is still a pain (listing issues, and also comments on issues).
The code is still a mess in some places, and I haven’t figured out how to test most of it. But it was interesting to reverse engineer how GitHub decides on the text color for labels (they compute it in CSS!)
]]>It’s tempting to write a philosophical piece about the meaning of it all, and provide some grand perspective. But there is no overarching narrrative to this blog. It’s just been a place to write things and connect with people in various ways.
Here are some popular deep cuts you might have missed over the years:
Often I write about what’s going on with my autistic son Nat:
Of course, there’s lots of Python and other technical stuff:
Looking back 20 years at the early posts, many of the things I liked and linked to are gone. But for whatever reason, the puzzle-makers, geometers and type designers seem most durable. Here are some that are still around (and mostly look their age!):
I’m not sure what else to say about 20 years of blogging. I’m still doing it.
]]>At work a test like this was failing:
# -- cool_feature.py ------------
import logging
logger = logging.getLogger(__name__)
def expensive_prep():
...
def do_cool_thing(text):
expensive_prep()
logger.info(f"Cool feature is doing {text!r}")
# -- test_cool_feature.py ------------
from unittest.mock import patch
from cool_feature import do_cool_thing
@patch("cool_feature.expensive_prep")
@patch("cool_feature.logger")
def test_cool_feature_announcement(mock_prep, mock_logger):
do_cool_thing("hello")
mock_logger.info.assert_called_once_with(
"Cool feature is doing 'hello'"
)
We couldn’t see why the log mock wasn’t getting called at all. The code was straightforward, the test was simple, how come the mock wasn’t catching the log call?
Mocks are slippery. That’s their strength and their weakness. They will happily let you call any method on them, or access any attribute, and when you do, they will give you another slippery mock that will let you do whatever you want. This can be very powerful, but also very misleading.
The problem with our test was the order of the mocks in the function signature. The first decorator corresponds to the last argument, because the first decorator is the last one applied. That’s the inside-out nature of decorators. It should be:
@patch("cool_feature.expensive_prep")
@patch("cool_feature.logger")
def test_cool_feature_announcement(mock_logger, mock_prep):
When the order was wrong, we called the logger, and then were asserting that “cool_feature.expensive_prep” had its “info” method called. That (non-existent) method hadn’t been called, so our assert failed.
We were using the mocks completely incorrectly, but mocks don’t mind: you can do whatever you want, and they let you, with no notification.
Once the order of the mocks in the signature was fixed, the test passed.
But there’s a way we could write this to be more fool-proof. Mocks can be made strict with their “spec=True” option. When set, the mock will raise an error if you use a method or attribute on it that didn’t exist on the object it was mocking.
Even better is the autospec=True option to mock.patch. This makes the mock strict with spec=True, and also recursively auto-applies spec=True to any new mock you get from the mock:
@patch("cool_feature.expensive_prep", autospec=True)
@patch("cool_feature.logger", autospec=True)
def test_cool_feature_announcement(mock_logger, mock_prep):
If we had been using autospec from the start, the incorrect test would have raised a helpful error right away:
AttributeError: 'function' object has no attribute 'info'
I wish autospec=True was the default. I can see why there are occasional needs for truly permissive mocks, but mostly we intend to use them in strict ways, and want to know when we don’t.
BTW, the usual advice to avoid mocking applies if you can, since even when you get your mocks working properly, there are still dangerous pitfalls. I’ll re-recommend these two resources about how to test differently:
In this post I’ll use an example where I’m working on branch “my-work”, which I started from “old-home” and I want to move it to “new-home”. My initial state looks like this:
% git log --oneline --decorate --graph --all
* a8ab80e (HEAD -> my-work) my work edit 2
* 5f66562 my work edit 1
| * 9682932 (old-home) old-home edit 3
|/
* 98d09e4 old-home edit 2
* a6fa334 old-home edit 1
| * e409e95 (new-home) new-home edit 2
| * 7d3ca09 new-home edit 1
|/
* 1b008d5 (main) main edit 2
* e9152aa main edit 1
The git tool to perform a move like this is “git rebase --onto”. The general form of the command needs three pieces of information, but the third is the branch to move, which defaults to the current branch, so we can just omit that. The other two items are: where you want to snip the current branch from, and where you want to graft it onto. The command looks like this:
git rebase --onto=<graft-onto> <snip-from> [<branch-to-move>]
In our example, we want to snip the branch from the point where it started on old-home. The “git merge-base” command can tell us the commit where two branches diverge, so it’s perfect for this:
% git merge-base old-home @
98d09e4b4d3ca45e3e03cf27386f7dd01f0662a8
Putting this together in one command looks like this:
% git rebase --onto new-home $(git merge-base old-home @)
Successfully rebased and updated refs/heads/my-work.
The result is just what we wanted:
% git log --oneline --decorate --graph --all
* df22106 (HEAD -> my-work) my work edit 2
* 9ac8707 my work edit 1
* e409e95 (new-home) new-home edit 2
* 7d3ca09 new-home edit 1
| * 9682932 (old-home) old-home edit 3
| * 98d09e4 old-home edit 2
| * a6fa334 old-home edit 1
|/
* 1b008d5 (main) main edit 2
* e9152aa main edit 1
But that command is complicated to type, so I made an alias in my .gitconfig:
[alias]
movebranch = "!f() { \
: git checkout \
git rebase --onto $2 $(git merge-base $1 @) \
}; f"
Now our branch move command would look like this:
% git movebranch old-home new-home
Successfully rebased and updated refs/heads/my-work.
And the result is the same:
]]>% git log --oneline --decorate --graph --all
* df22106 (HEAD -> my-work) my work edit 2
* 9ac8707 my work edit 1
* e409e95 (new-home) new-home edit 2
* 7d3ca09 new-home edit 1
| * 9682932 (old-home) old-home edit 3
| * 98d09e4 old-home edit 2
| * a6fa334 old-home edit 1
|/
* 1b008d5 (main) main edit 2
* e9152aa main edit 1
What is going on? How did such a tiny thing get named Broadway? It turns out, the better question is, how did Broadway become such a tiny thing?
Like many cities, and probably more than most cities, Boston has undergone enormous change. When first settled in 1630, it was nearly an island, connected to the mainland by a narrow isthmus.
This 1826 map of Boston shows two streets relevant to our mystery, highlighted in yellow: Pleasant Street at the top heading up to Boston Common, and Broadway, part of South Boston at the bottom:
By 1917, much of the surrounding water has been landfilled (the 1826 map shows some planned streets), and Broadway from South Boston now runs into Pleasant Street, continuing up to Boston Common:
By 1938, Pleasant Street had been renamed, so the entire street from South Boston to Boston Common is now a major street named Broadway.
Some time in the 1970’s, after the extension of the Mass Pike to I-93, this part of Boston had a massive restructuring. Today it looks like this:
Broadway is still a major street in South Boston at the bottom, but all that remains of the street in Boston proper is a small piece now called Park Place near Boston Common at the top, and our 200-foot street still named Broadway just below that. Refactoring a city can leave you with some strange little remnants.
Just to make it more confusing, of the three(!) street signs on the street, two say “Broadway”, and one says “Broadway St,” which doesn’t even make sense...
BTW, sorry for the changing scale and orientation in these maps. If you want to explore maps over time in a less disorienting way, mapjunction.com is fabulous.
]]>