refactoring: txaio

Longer Story

This is actually a moderately interesting exploration of refactoring something that affects 3 different repositories, all open-source.

Our story begins as with all good development tasks ;) at a GitHub pull-request, in this case AutobahnPython #358 which had gone from a simple “I should improve error-handling” to quite a big change, including some attempt at resolving an API difference between Twisted and asyncio.

So, sanity prevailed, and this got split. The one we’re interested in became AutobahnPython #363 which focuses on fixing the errback APIs Autobahn was using between asyncio and Twisted.

As @oberstet mentioned in #358, there was an existing “semi-private” API in AutobahnPython as a sort of incubator-API that did what txaio now does. This was a little weird, as it depended on particluar methods existing in concrete subclasses of ApplicationSession where those methods were actually called from the superclass – but they only existed via a mix-in style class.

These methods were provided by a class called FutureMixin available in both a Twisted flavour as well as a corresponding asyncio one

Although perhaps a little strange, inheritance-wise, these worked fairly well unless you wanted to do async things outside of ApplicationSession subclasses, or for error-handling. asyncio lacks any abstraction like Twisted’s Failure class – if you use asyncio with the callback-style API (add_done_callback) you have to call Future.result() to get the value out, and if there was an error you get the exception then. Contrast this with Twisted’s API, where either your callback or errback is called. (asyncio’s API also has no way to mutate the values as they go to subsequent callbacks, nor to cancel errors).

Since txaio (and FutureMixin) followed a more Twisted style, this meant I would have to write a Failure -like wrapper for the asyncio case. This is in fact the only wrapper in txaio; the rest of the library is basically just call-throughs.

Refactoring

After ripping things out into the txaio repository that @oberstet had provided, playing with the API a little and some discussion in #twisted I shortened a few calls and kept the API fairly similar to what was in FutureMixin already. This is a nice, simply wrapper library that’s easy to understand and gives you “native” objects back (e.g. you get a real Future for asyncio and a real Deferred for Twisted; no wrappers).

Note that if you want a full asyncio/Twisted compatibility thing and don’t mind having a Twisted dependency for all users, you should look at txtulip.

Ultimately the changes to AutobahnPython are these while Crossbar gets a few changes also.

In a great fit of irony, the one changed line that I didn’t cover with a unit-test was the one where I (probably) fat-fingered a C-t one too many times (I use ctrl-t as my tmux prefix-key) making Emacs transpose taxio creating a regression in a timeout-case. Darn. So I wrote a unit-test for that one too.

Conclusions

Moving this stuff to a separate library was, I think, absolutely the right decision. I’m not sure if it will be useful to very many people, but it makes the API more self-contained, easier to test (txaio has 100% test-coverage) and a little easier to document.

It also changes ApplicationSession from being an async compatibility layer itself to simply using that functionality – which provides better separation and is easier to understand. (Really not strictly true since txaio just became a global in the end, but conceptually …)

Although some of the timeout tests were a bit tricky to get working, writing unit-tests and then changing things always, always works out better – and my unfortunate typo regression certailny confirms this.

There’s one moderately neat thing I do with the coverage analysis, summing coverage over all the tox environments to prove I’m running all the asyncio plus Twisted code. See tox.ini