October 22, 2023
Giving more context to Magic Wormhole’s Dilation feature, and my fowl project.
Setting the Stage
Magic Wormhole answers a still-common need: for two computers to securely connect across the internet, mediated by human action.
The “Dilation” feature of this protocol is extremely flexible and durable and has great potential for many, many use-cases. This allows account-less, identity-less streaming connections across unstable and changing network conditions (even in the face of disconnects, multiple NAT devices, etc).
Of course, there’s at least one downside: there is only a Python implementation (of Dilation). Additionally, although there are plenty of FOSS services that can in principal be set up and hosted locally, there’s sometimes more typing and configuration than one might hope.
Continuing the “magic” theme of Magic Wormhole, my code-name for this line of thinking is “Wizard Gardens”. (There’s also an unimplemented feature in Magic Wormhole called “Seeds”).
The Wizard Garden is a place to put “glue” code that knows how to set up local servers (and how to run local clients to connect to them). Magic Wormhole lets this “glue” code act like those local servers are on the very same machine as the client.
So, for many network services, this then means taking advantage of Magic Wormhole’s networking features without any code changes.
In this analogy / whimsical concept, the Wizards represent computing devices. When their humans give them the correct magic spell (i.e. a code like “2-foo-bar”) and two Wizards speak it at the same time, a Magic Wormhole is born: a secure, streaming connection between the two Wizards (computers).
(The unimplemented Seeds feature lets one “save” such a Wormhole for the future, so that the spell is no longer required: just the True Name of the conncetion … yes, I love “A Wizard of Earthsea”).
I recently released a Python program called fowl which is one step towards this vision.
This allows you to use Magic Wormhole (with Dilation) to set up streaming connections. Conceptually pretty similar to the
ssh -L and
-R options, although with the benefits of Magic Wormhole: no identity (i.e. only ephemeral keys, no need to pre-share anything), durability (re-connect across network changes) and NAT/etc traversal.
fowl lacks important features, but allows implementors to play with Dilation from any language that can start a subprocess and communicate on stdin + stdout.
This is easier to talk about with a concrete example: tty-share.
tty-share is a nice program for terminal sharing. One person runs a server and one or more other people run a client.
- However, currently to use it you must choose:
- use the author-run public relay service;
- run your own such service on a public IP;
- arrange your own network conditions so the “server” / sharing user appears on the network
In this conception, Wizard Garden offers a new general answer: use Magic Wormhole for the networking.
The insight here is that it’s often fairly straightforward to set up a listener on your own computer, and then use a client on the local network (or same machine) to connect to it.
Magic Wormhole brings this simplicity to the public Internet
So the Wizard Garden houses some (ideally fairly simple) “glue” code that encapsulates knowledge about how to run a service (both client or service side). That is: which ports to open and what programs to run with which options.
tty-share example, this means that the host runs
tty-share with no options (which will open a server on
localhost:8000 by default). One can immediately then use
fowl to set up a Wormhole and tell it to open a “far side” listener on
localhost:8000 that will connect back “here”.
Finally, the “far side” user runs
tty-share http://localhost:8000/s/local/. The
fowl code that is listening there then opens a Magic Wormhole Dilation subchannel. The near-side
fowl code sees this subchannel open and makes a TCP connection to
localhost:8000 for it, before simply forwarding bytes (forth and back).
The net result is a
tty-share connection between two computers anywhere on the Internet – but without requiring a public IP address or a third-party service.
(Note that Magic Wormhole itself does run a “third party” mailbox service, but see that project for security arguments – the insight here is that we only need that one, not one for every kind of network application).
Generalization of the Concept
We can generalize this concept for anything that acts like
tty-share in the above example. That is, any program (or collection of programs) that require streaming connections and can listen on (or connect to) localhost TCP (or unix sockets).
- The advantages here are:
- no need to set up additional public Internet services;
- a single cohesive place to handle security considerations;
- all communications are definitely end-to-end encrypted (and usually peer-to-peer);
- no code-changes for “end user” applications
fowl is one part of this. I imagine (and am working on) an additional application (that uses
fowl, or something like it) which hosts and runs the “glue” code and handles user interactions.
Ideally, there might be several such applications (e.g. some people like GUIs, some people prefer TUIs or command-lines, etc).
Ideas? Excited? Get in touch or file some PRs! You may also follow up on Mastodon.