The TunTap Router

The TunTap router is an ad-hoc solution for giving network connectivity to Linux processes running in a dedicated network namespace. Well obviously it has other uses, but very probably this is the most interesting one, and the one for which it was developed. Currently it builds only on Linux platforms.

The TunTap router is not compiled in by default.

For having it in one shot:

  1. UWSGI_EMBED_PLUGINS=tuntap make

(yes the plugin is named only ‘tuntap’ as effectively it exposes various tuntap devices features)

The best way to use it is binding it to a unix socket, allowing processes in new namespaces to reach it (generally unix sockets are the best communication channel for linux namespaces).

The first config

We want our vassals to live in the 192.168.0.0/24 network, with 192.168.0.1 as default gateway.

The default gateway (read: the tuntap router) is managed by the Emperor itself

  1. [uwsgi]
  2. ; create the tun device 'emperor0' and bind it to a unix socket
  3. tuntap-router = emperor0 /tmp/tuntap.socket
  4. ; give it an ip address
  5. exec-as-root = ifconfig emperor0 192.168.0.1 netmask 255.255.255.0 up
  6. ; setup nat
  7. exec-as-root = iptables -t nat -F
  8. exec-as-root = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
  9. ; enable linux ip forwarding
  10. exec-as-root = echo 1 >/proc/sys/net/ipv4/ip_forward
  11. ; force vassals to be created in a new network namespace
  12. emperor-use-clone = net
  13. emperor = /etc/vassals

The vassals spawned by this Emperor will born without network connectivity.

To give them access to the public network we create a new tun device (it will exist only in the vassal network namespace)instructing it to route traffic to the Emperor tuntap unix socket:

  1. [uwsgi]
  2. ; we need it as the vassal have no way to know it is jailed
  3. ; without it post_jail plugin hook would be never executed
  4. jailed = true
  5. ; create uwsgi0 tun interface and force it to connect to the Emperor exposed unix socket
  6. tuntap-device = uwsgi0 /tmp/tuntap.socket
  7. ; bring up loopback
  8. exec-as-root = ifconfig lo up
  9. ; bring up interface uwsgi0
  10. exec-as-root = ifconfig uwsgi0 192.168.0.2 netmask 255.255.255.0 up
  11. ; and set the default gateway
  12. exec-as-root = route add default gw 192.168.0.1
  13. ; classic options
  14. uid = customer001
  15. gid = customer001
  16. socket = /var/www/foobar.socket
  17. psgi-file = foobar.pl
  18. ...

The embedded firewall

The TunTap router includes a very simple firewall for governing vassal’s traffic

Firewalling is based on 2 chains (in and out), and each rule is formed by 3 parameters: <action> <src> <dst>

The firewall is applied to traffic from the clients to the tuntap device (out) and the opposite (in)

The first matching rule stops the chain, if no rule applies, the policy is “allow”

the following rules allows access from vassals to the internet, but block vassals intercommunication

  1. [uwsgi]
  2. tuntap-router = emperor0 /tmp/tuntap.socket
  3.  
  4. tuntap-router-firewall-out = allow 192.168.0.0/24 192.168.0.1
  5. tuntap-router-firewall-out = deny 192.168.0.0/24 192.168.0.0/24
  6. tuntap-router-firewall-out = allow 192.168.0.0/24 0.0.0.0
  7. tuntap-router-firewall-out = deny
  8. tuntap-router-firewall-in = allow 192.168.0.1 192.168.0.0/24
  9. tuntap-router-firewall-in = deny 192.168.0.0/24 192.168.0.0/24
  10. tuntap-router-firewall-in = allow 0.0.0.0 192.168.0.0/24
  11. tuntap-router-firewall-in = deny
  12.  
  13. exec-as-root = ifconfig emperor0 192.168.0.1 netmask 255.255.255.0 up
  14. ; setup nat
  15. exec-as-root = iptables -t nat -F
  16. exec-as-root = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
  17. ; enable linux ip forwarding
  18. exec-as-root = echo 1 >/proc/sys/net/ipv4/ip_forward
  19. ; force vassals to be created in a new network namespace
  20. emperor-use-clone = net
  21. emperor = /etc/vassals

Security

The “switching” part of the TunTap router (read: mapping ip addresses to vassals) is pretty simple: the first packet received from a vassal by the TunTap routerregister the vassal for that ip address. A good approach (from a security point of view) is sending a ping packet soon after network setup in the vassal:

  1. [uwsgi]
  2. ; create uwsgi0 tun interface and force it to connect to the Emperor exposed unix socket
  3. tuntap-device = uwsgi0 /tmp/tuntap.socket
  4. ; bring up loopback
  5. exec-as-root = ifconfig lo up
  6. ; bring up interface uwsgi0
  7. exec-as-root = ifconfig uwsgi0 192.168.0.2 netmask 255.255.255.0 up
  8. ; and set the default gateway
  9. exec-as-root = route add default gw 192.168.0.1
  10.  
  11. ; ping something to register
  12. exec-as-root = ping -c 1 192.168.0.1
  13.  
  14. ; classic options
  15. ...

after a vassal/ip pair is registered, only that combo will be valid (so other vassals will not be able to use that address until the one holding it dies)

The Future

This is becoming a very important part of the unbit.it networking stack. We are currently working on:

  • dynamic firewall rules (luajit resulted a great tool for writing fast networking rules)
  • federation/proxy of tuntap router (the tuntaprouter can multiplex vassals networking over a tcp connection to an external tuntap router [that is why you can bind a tuntap router to a tcp address])
  • authentication of vassals (maybe the old UNIX ancillary credentials could be enough)
  • a stats server for network statistics (rx/tx/errors)
  • a bandwidth shaper based on the blastbeat project