How to Manage your Development System behind firewall using Tutum

Tutum  can now manage hosts behind firewalls as well. This is especially useful for development environments on Virtual Machines (VMs) running on laptops or systems without having public IPs.  It’s always better to know what is happening behind the scenes, both from gaining knowledge and applying the techniques in other similar situations.

In this article, let’s see some behind the scenes magic on how Tutum is able to manage a docker host that is not reachable publicly.

First, let us go through the steps to add a node to Tutum. In this example, we’ll use a VM provisioned via Vagrant (libvirt plugin), that uses NAT for outside connectivity.

This is the example VM with private IP. This VM can reach internet using NAT that is set up on the host.

vagrant@tutum-node:~$ ifconfig 
eth0   Link encap:Ethernet  HWaddr 52:54:00:64:47:10  
          inet addr:192.168.122.43  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe64:4710/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2076 errors:0 dropped:6 overruns:0 frame:0
          TX packets:1280 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3777257 (3.7 MB)  TX bytes:118646 (118.6 KB)
[snip]

Steps to add a node to Tutum

Option-1: Using Tutum CLI
Install the Tutum CLI on the node (VM )

vagrant@tutum-node:~$ sudo pip install tutum

Get the command line to install the agent on the local node.

You will need Tutum login details

vagrant@tutum-node:~$ tutum node byo
Not Authorized, Please login:
Username: 
Password: 
Login succeeded!
Tutum lets you use your own servers as nodes to run containers. For this, you have to install our agent.
Run the following command on your server:
	curl -Ls https://files.tutum.co/scripts/install-agent.sh | sudo -H sh -s 604d69b2ce234d81b39e49e489a66024

Install Tutum agent on the node by running the command printed above.

vagrant@tutum-node:~$ curl -Ls https://files.tutum.co/scripts/install-agent.sh | sudo -H sh -s 604d69b2ce234d81b39e49e489a66024
->Adding Tutum's GPG key...
[snip]
->Installing tutum-agent...
->Configuring tutum-agent...
->Done!

*******************************************************************************
Tutum Agent installed successfully
*******************************************************************************

That is all. You can now deploy containers to this node using Tutum

Option-2: Using Tutum web dashboard
Click ‘Bring your own node’ option under the ‘Nodes’ tab, which will provide a command that needs to be run on the node.

byon-tutum

byon-cmd-tutum

Install the agent on the node by running the command printed above

vagrant@tutum-node:~$ curl -Ls https://files.tutum.co/scripts/install-agent.sh | sudo -H sh -s 100176fed0bb48afa561fafca4d97697
-> Adding Tutum's GPG key...
[snip]
-> Installing tutum-agent...
-> Configuring tutum-agent...
-> Done!

*******************************************************************************
Tutum Agent installed successfully
*******************************************************************************

Checking the docker process on this node, you will see something like the following:

vagrant@tutum-node:~$ps aux | grep docker
[snip]
 /usr/lib/tutum/docker -d -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --tlscert /etc/tutum/agent/cert.pem --tlskey /etc/tutum/agent/key.pem --tlscacert /etc/tutum/agent/ca.pem --tlsverify
[snip]

The node will be available in the dashboard and you can run containers and perform operations on this node from the dashboard.

How was this made possible for a private IP ? What’s the magic :-)

Checking the agent.log in /var/log/tutum will give you some hints. Search for ‘ngrok’ in the log and you’ll see something like the following in your log :

[snip]
2015/03/29 16:37:33 Set ngrok server address to tunnel.tutum.co:4443
2015/03/29 16:37:33 Creating ngrok config file in /etc/tutum/agent/ngrok.conf ...
2015/03/29 16:37:33 About to tunnel to private ngrok service
2015/03/29 16:37:33 Starting montoring tunnel: [/usr/lib/tutum/ngrok -config /etc/tutum/agent/ngrok.conf -log stdout -proto tcp 2375]
2015/03/29 16:37:33 Starting NAT tunnel: [/usr/lib/tutum/ngrok -config /etc/tutum/agent/ngrok.conf -log stdout -proto tcp 2375]
2015/03/29 16:37:35 Found new tunnel:tcp://tunnel01.tutum.co:53656
2015/03/29 16:37:35 Patching tunnel address to Tutum
2015/03/29 16:37:35 Successfully Patched tunnel address to Tutum
2015/03/29 16:42:24 Node registration successful with https://dashboard.tutum.co/
[snip]

The magic is in the usage of ‘ngrok‘ program. ngrok is a program to expose local service to the internet via secure tunnels. Basically, ngrok allocates a port on a publicly accessible system running the ngrok server (usually its ngrok.com, but can be your own server as well) and forwards all traffic on that port to the local service.

You can see the following ‘ngrok’ process running on your node.

/usr/lib/tutum/ngrok -config /etc/tutum/agent/ngrok.conf -log stdout -proto tcp 2375

Tutum makes use of ‘ngrok’ to setup tunnel and expose the docker port externally. Tutum uses its own ngrok server to setup the tunnel as is evident in the ngrok configuration file:

vagrant@tutum-node:~$ cat /etc/tutum/agent/ngrok.conf
server_addr: tunnel.tutum.co:4443
trust_host_root_certs: false

So, to put it simply, there is a tunnel setup between the local node (port 2375) and ngrok server (tunnel.tutum.co).

If you want to know more about ngrok tunneling protocol, you can refer to the following development guide.

Pradipta Kumar Banerjee

I'm a Cloud and Linux/ OpenSource enthusiast, with 16 years of industry experience at IBM. You can find more details about me here - Linkedin

You may also like...