A Tale of Two Factorings
December 16, 2018
In two different Python projects I’ve recently had some odd issues with “MixIn” style inheritance – that is, beyond the issue that I’ve come to completely loathe pretty much anything that has “MixIn” in the class-name.
Ned Batchelder has an appropriate post from a few years ago (Multiple Inheritance is Hard) conveniently using testing as a scapegoat/example.
“Convenient” because one of my issues is with some unit-tests using approximately 9000 different
MixIn classes to do setup and teardown things, littering
self. with all kinds of state. In this post I talk about why the MixIn pattern is bad specifically in the
What is “MixIn” For?
Usually, you’re supposed to use this pattern to “mix in” some fun new functionality. That is, so you can re-use some code “merely” by inheriting from the so-called “mix in” class. Apparently, it’s fairly tempting to use it to add more setup code to your test classes (like in Ned’s example) since I’ve certainly seen that several times too. So a (very) simplified version of this might look like:
from unittest import TestCase class ComplicatedTests(TestCase): def setUp(self): # set up some things we need for the test self.one = "something" self.two = "a thing" def test_example(self): # use .one, .two to do some test stuff
Now, obviously this is just a contrived and super simplistic example, but let’s say that
.two are useful things to setup for other situations (e.g tests not in this test-case or module) and have some non-trivial amount of code to set up – to share this code, the
MixIn pattern will tell you to do something like this:
class OneMixIn(object): def setUp(self): super(OneMixIn, self).setUp() self.one = "something" class TwoMixIn(object): def setUp(self): super(TwoMixIn, self).setUp() self.two = "a thing" class ComplicatedTests(OneMixIn, TwoMixIn, TestCase): def test_example(self): # use .one, .two to do some test stuff
Now even presuming that you got the
super() calls right (and everything in your class tree got it right, too! 1) this is a terrible idea. For any real-world code, if you keep following this pattern for “common setup code” you’ll end up with a horrific mess of a class tree with the
setUp code for any given test becoming hard to find, and tons of
self.* attributes sprinkled all over the place.
This will make it hard to read, fragile, and hard to update – not to mention error-prone. Now, for fun, if you’ve got event-based code you’re probably returning
Deferred instances from at least some
setUp methods, so besides getting
super() correct you also need to get return values in
So There’s a Better Way, Right?
First, let’s step back and remember what we’re trying to accomplish here: we want to re-use some common set-up code. That is, to compose these setup helpers in interesting new ways. (A side note that helpers for tests like this are usually called “fixtures”).
MixIn pattern, you force everything that wants to use your helpers into inheriting from a class. This is inconvenient and hard to do – what if you have a thing that’s not a
unittest.TestCase that wants to use one of the helpers? For example, an integration test or example code.
So, turn them into some simple helper-functions:
def setup_one(): return "something" def setup_two(): return "a thing" class ComplicatedTests(TestCase): def setUp(self): self.one = setup_one() self.two = setup_two() def test_example(self): # use .one, .two to do some test stuff
That’s starting to look a little bit better! However, we still have the state littering our test-case (in
self.*), and if we have two tests we don’t know if they both use all the state (granted, if you’ve factored the test-cases correctly they will, but that doesn’t always happen). The above doesn’t deal with any tear-down that may need to happen, either.
So we could try to turn them into context-managers and just use them in the tests that need them. Maybe like this:
from unittest import TestCase from contextlib import contextmanager @contextmanager def setup_one(): # setup code yield "something" # any teardown code @contextmanager def setup_two(): yield "a thing" class ComplicatedTests(TestCase): def test_example(self): with setup_one() as one, setup_two() as two: pass
Okay, now things are starting to look a bit more explicit; we can see that
test_example is definitely using two different things for its test and can easily search for those methods. We can see what state (if any) the fixtures use, and they can have “private” state easily (by not returning it). We can also tell if the tests use the state by whether they keep the return values from the context-managers. If you add another test that needs just one of the fixtures, you can do that too and still be explicit.
So, the above style is almost like you can achieve with py.test which is the testing framework I personally greatly prefer over
unittest. Although of course you’re free to write pretty similar code in either
unittest, I find
unittest tends to produce more mix-in style code. In any case, here is the final re-factor to use
import pytest @pytest.fixture def one(request): # any teardown stuff: request.addfinalizer(lambda: None) return "something" @pytest.fixture def two(): return "a thing" def test_example(one, two): pass
Very nice and clean, and we can tell which fixtures
test_example uses by the names of its arguments. (As I hinted at event-based testing earlier, pytest’s @fixtures play nicely with
Deferreds via an additional small library).
|1||so it turns out that