I've moved my blog to jmcneil.net. This is no longer being updated!

Thursday, May 14, 2009

Odd Python Errors

Over the past year or so I've started keeping a text file around that details some of the not-so-obvious errors that Python might throw out there. Every now and then I run into one of them and it makes my brain hurt just a little bit.

Here are a couple of my favourites.

Metaclass Conflicts

There are a few reasons these might pop up, but one error message in particular seems to stand out. Each time I see it I've got to read through it a few times and mentally break it down into components.

Traceback (most recent call last):
File "t.py", line 44, in
class C3(C1, C2): pass
TypeError: Error when calling the metaclass bases
metaclass conflict: the metaclass of a derived \
class must be a (non-strict) subclass of the metaclasses \
of all its bases

Pfft? Wha? Ouch. The offending code is as follows.

class M1(type): pass
class M2(type): pass
class C1(object):
__metaclass__ = M1

class C2(object):
__metaclass__ = M2

class C3(C1, C2): pass #<--- Boom!

In other words, C1 and C2 each define a __metaclass__ attribute, but issubclass(M1, M2) and issubclass(M2, M1) both return false. Python raises a TypeError as the metaclasses are not in the same inheritance chain.

The following, however, does work:

class M1(type): pass
class M2(M1): pass
class C1(object):
__metaclass__ = M1

class C2(object):
__metaclass__ = M2

class C3(C2, C1): pass #<--- No Boom!

In this situation, issubclass(M2, M1) is true so Python is able to build C3 using M2. What happens if we add another base class to C3?

class M1(type): pass
class M2(M1): pass
class M3(M1): pass

class C1(object):
__metaclass__ = M1

class C2(object):
__metaclass__ = M2

class G(object):
__metaclass__ = M3

class C3(G, C2, C1): pass #<--- Boom!

Inserting 'G' into the mix causes the same error to occur.

Traceback (most recent call last):
File "t7.py", line 16, in
class C3(G, C2, C1): pass #<--- Boom!
TypeError: Error when calling the metaclass bases
metaclass conflict: the metaclass of a derived \
class must be a (non-strict) subclass of the metaclasses \
of all its bases

The same error, but why? M3 is a child of M1, so according to the above example, this should work, correct? Well, not really. Consider the wording of the above error message. The metaclass of C3 is not a subclass of all of it's bases. The fix?

class M1(type): pass
class M2(M1): pass
class M3(M2): pass

class C1(object):
__metaclass__ = M1

class C2(object):
__metaclass__ = M2

class G(object):
__metaclass__ = M3

class C3(G, C2, C1): pass #<--- No Boom!

Now the metaclass of C3 (M3) is a subclass of all of the metaclasses of all of C3's bases.

Layout Conflicts

This is another one of my favourites. I spent quite a while tracking this one down the first time it showed up.

Traceback (most recent call last):
File "t88.py", line 7, in
class C3(C2, C1): pass
TypeError: Error when calling the metaclass bases
multiple bases have instance lay-out conflict

The offending code is as follows:

class C1(object):
__slots__ = ('attr1', 'attr2')

class C2(object):
__slots__ = ('attr3', 'attr4')

class C3(C2, C1): pass #<--- Boom!

This one isn't quite so bad. The '__slots__' attribute is used when allocating heap memory for the new type. In general terms, pointers to the attributes defined in '__slots__' are inserted into the new type's structure. Since C1 and C2 both define '__slots__' and they're not from the same inheritance chain, the type structures differ. Thus, we have an instance lay-out conflict.

The fix?

class C1(object):
__slots__ = ('attr1', 'attr2')

class C2(C1):
__slots__ = ('attr3', 'attr4')

class C3(C2, C1): pass #<-- No Boom!

Note that it is entirely possible to add another class to C3's bases so long as it doesn't define any additional '__slots__' attributes.
Are there any other odd errors out there that trip people up? If you've got any, I'd love to add them to my little text file.

3 comments:

rgz said...

Excellent post!

Michele Simionato said...

There is an error that happens
when you accidentally use a module
instead of a class as base:

In [1]: import os

In [2]: class C(os): pass
...:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)

/home/msimionato/trunk/ROnline/ipython console in module()

TypeError: Error when calling the metaclass bases
module.__init__() takes at most 2 arguments (3 given)

Jeff McNeil said...

I don't think I've ever bumped into that one. Module object's init function takes a doc and a name, so that makes sense... good one.