Notes on subclassing Python’s dict

Update 2011-05-10: This post was written after implementing securedict in Securetypes. If this post doesn’t make sense, see the code.

The notes:

If for some reason you must subclass Python’s, dict, keep these in mind:

  1. Both dict.__init__ and dict.update use the update algorithm, which doesn’t necessarily iterate over the object you pass in:

    >>> help({}.update)

D.update(E, **F) -> None. Update D from dict/iterable E and F.
If E has a .keys() method, does: for k in E: D[k] = E[k]
If E lacks .keys() method, does: for (k, v) in E: D[k] = v
In either case, this is followed by: for k in F: D[k] = F[k]

    Above, for k in E: D[k] should actually read for k in E.keys(): D[k].

    It also omits a CPython implementation quirk: the update algorithm has a fast path for dicts (and subclasses of it), which ignores the keys method. CPython’s dictobject.c actually does this:

    D.update(E, **F) -> None. Update D from dict/iterable E and F.
If isinstance(E, dict), does: for k in E: D[k] = E[k], bypassing E.__iter__
Else if E has a .keys() method, does: for k in E.keys(): D[k] = E[k]
Else if E lacks .keys() method, does: for (k, v) in E: D[k] = v
In any case, this is followed by: for k in F: D[k] = F[k]

  2. If you override __eq__ and __ne__, remember to override __cmp__ as well, or else cmp(yourCustomDict, ...) will be broken.

  3. If your custom dict behaves a lot like the real Python dict, consider copying many of the unit tests from CPython’s Lib/test/test_dict.py:DictTest. These tests have some omissions, though: they don’t test .iteritems(), the new .view*() methods, or dict instantiation with **kwargs. They’re also missing comprehensive equality tests.

  4. A custom __repr__ is tricky to implement, if you’re trying to avoid infinite recursion when the dict contains itself. Built-in types solve the problem by repr’ing to something like [[...]] or {"a": {...}}. In your dict subclass with a custom __repr__, use an instance variable to track whether you’ve already been __repr__ ‘ed, and remember to reset that variable in a finally: block.

  5. Do you want .copy() to return an instance of your own custom dict? If so, better implement it.

  6. In Python 2.7+, dicts have three new methods: viewkeys, viewitems, and viewvalues. Changing their behavior from pure Python code doesn’t look easy, especially since the view objects have a special type.