Introducing Linux Network Namespaces
http://blog.scottlowe.org/2013/09/04/introducing-linux-network-namespaces/
In this post, I’m going to introduce you to the concept of Linux network namespaces. While it might seem a bit esoteric right now, trust me that there is a reason why I’m introducing you to network namespaces—if you, like me, are on a journey to better understand OpenStack, you’ll almost certainly run into network namespaces again.
So what are network namespaces? Generally speaking, an installation of Linux shares a single set of network interfaces and routing table entries. You can modify the routing table entries using policy routing (here’s an introduction I wrote and here’s a write-up on a potential use case for policy routing), but that doesn’t fundamentally change the fact that the set of network interfaces and routing tables/entries are shared across the entire OS. Network namespaces change that fundamental assumption. With network namespaces, you can have different and separate instances of network interfaces and routing tables that operate independent of each other.
This concept is probably best illustrated through some examples. Along the way, I’ll introduce a few new ideas as well. First, though, I need to provide some assumptions.
Assumptions
Throughout these examples, I’m using Ubuntu Server 12.04.3 LTS. Please note that support for network namespaces varies between Linux distributions; Ubuntu supports them but Red Hat doesn’t. (I’m not sure about Fedora. If you know, speak up in the comments.) If you’re thinking about using network namespaces, be sure your Linux distribution includes support.
Further, I’ll assume that you’re either running as root, or that you will prependsudo to the commands listed here as necessary.
Creating and Listing Network Namespaces
Creating a network namespace is actually quite easy. Just use this command:
ip netns add
For example, let’s say you wanted to create a namespace called “blue”. You’d use this command:
ip netns add blue
To verify that the network namespace has been created, use this command:
ip netns list
You should see your network namespace listed there, ready for you to use.
Assigning Interfaces to Network Namespaces
Creating the network namespace is only the beginning; the next part is to assign interfaces to the namespaces, and then configure those interfaces for network connectivity. One thing that threw me off early in my exploration of network namespaces was that you couldn’t assign physical interfaces to a namespace. How in the world were you supposed to use them, then?
It turns out you can only assign virtual Ethernet (veth) interfaces to a network namespace. Virtual Ethernet interfaces are an interesting construct; they always come in pairs, and they are connected like a tube—whatever comes in one veth interface will come out the other peer veth interface. As a result, you can use veth interfaces to connect a network namespace to the outside world via the “default” or “global” namespace where physical interfaces exist.
Let’s see how that’s done. First, you’d create the veth pair:
ip link add veth0 type veth peer name veth1
I found a few sites that repeated this command to create veth1 and link it toveth0, but my tests showed that both interfaces were created and linked automatically using this command listed above. Naturally, you could substitute other names for veth0 and veth1, if you wanted.
You can verify that the veth pair was created using this command:
ip link list
You should see a pair of veth interfaces (using the names you assigned in the command above) listed there. Right now, they both belong to the “default” or “global” namespace, along with the physical interfaces.
Let’s say that you want to connect the global namespace to the blue namespace. To do that, you’ll need to move one of the veth interfaces to the blue namespace using this command:
ip link set veth1 netns blue
If you then run the ip link list command again, you’ll see that the veth1 interface has disappeared from the list. It’s now in the blue namespace, so to see it you’d need to run this command:
ip netns exec blue ip link list
Whoa! That’s a bit of a complicated command. Let’s break it down:
· The first part, ip netns exec, is how you execute commands in a different network namespace.
· Next is the specific namespace in which the command should be run (in this case, the blue namespace).
· Finally, you have the actual command to be executed in the remote namespace. In this case, you want to see the interfaces in the blue namespace, so you run ip link list.
When you run that command, you should see a loopback interface and the veth1 interface you moved over earlier.
Configuring Interfaces in Network Namespaces
Now that veth1 has been moved to the blue namespace, we need to actually configure that interface. Once again, we’ll use the ip netns exec command, this time to configure the veth1 interface in the blue namespace:
ip netns exec blue ifconfig veth1 10.1.1.1/24 up
As before, the format this command follows is:
ip netns exec
In this case, you’re using ifconfig to assign an IP address to the veth1 interface and bring that interface up. (Note: you could use the ip addr, ip route, and ip link commands to accomplish the same thing.)
Once the veth1 interface is up, you can verify that the network configuration of the blue namespace is completely separate by just using a few different commands. For example, let’s assume that your “global” namespace has physical interfaces in the 172.16.1.0/24 range, and your veth1 interface is in a separate namespace and assigned something from the 10.1.1.0/24 range. You could verify how network namespaces keep the network configuration separate using these commands:
· ip addr list in the global namespace will not show any 10.1.1.0/24-related interfaces or addresses.
· ip netns exec blue ip addr list will show only the 10.1.1.0/24-related interfaces and addresses, and will not show any interfaces or addresses from the global namespace.
· Similarly, ip route list in each namespace will show different routing table entries, including different default gateways.
Connecting Network Namespaces to the Physical Network
This part of it threw me for a while. I can’t really explain why, but it did. Once I’d figured it out, it was obvious. To connect a network namespace to the physical network, just use a bridge. In my case, I used an Open vSwitch (OVS) bridge, but a standard Linux bridge would work as well. Place one or more physical interfaces as well as one of the veth interfaces in the bridge, and—bam!—there you go. Naturally, if you had different namespaces, you’d probably want/need to connect them to different physical networks or different VLANs on the physical network.
So there you go—an introduction to Linux network namespaces. It’s quite likely I’ll build on this content later, so while it seems a bit obscure right now just hang on to this knowledge. In the meantime, if you have questions, clarifications, or other information worth sharing with other readers, please feel free to speak up in the comments.
Example:
2232 sudo ip link add veth0 type veth peer name veth1
2233 ip link list
2234 sudo ip link add veth1 type veth peer name veth0
2235 ip link set veth1 netns blue
2236 sudo ip link set veth1 netns blue
2237 sudo ip netns exec blue ifconfig -a
2238 sudo ip netns exec blue link list
2239 sudo ip netns exec blue ip link list
2240 sudo ip netns exec blue ifconfig veth1 192.168.10.21/24/ up
2241 sudo ip netns exec blue ifconfig veth1 192.168.10.21/24 up
2242 sudo ip netns exec blue ip link list
2243 sudo ip netns exec blue ifconfig -a
2244 sudo ip netns exec blue ping 127.0.0.1
2245 sudo ip netns exec blue ping 192.168.10.21
2246 ip link list
2247 sudo ip netns exec blue ip link set lo up
2248 sudo ip netns exec blue ip link set veth1 up
2249 sudo ip netns exec blue ping 192.168.10.21
2250 ip addr add 192.168.10.20 dev veth1
2251 sudo ip addr add 192.168.10.20 dev veth1
2252 sudo ip addr add 192.168.10.20 dev veth0
2253 sudo ip netns exec blue ping 192.168.10.20
2254 ip link set veth0 up
2255 sudo ip link set veth0 up
2256 sudo ip netns exec blue ping 192.168.10.20
2257 ifconfig -a
2258 sudo ip netns exec blue ping 192.168.10.20
2259 sudo ip netns exec blue ping 192.168.10.21
2260 which brctl
2261 ifconfig -a
2262 sudo ifconfig eth1 192.168.10.29 netmask 255.255.255.0 up
2263 ifconfig -a
2264 ping 192.168.10.20
2265 ping 192.168.10.29
2266 sudo ip netns exec blue ping 192.168.10.20
2267 sudo ip netns exec blue ping 192.168.10.29
2268 sudo ip netns exec blue ping 192.168.10.20
weng@weng-VirtualBox:~$ sudo brctl addbr br0
weng@weng-VirtualBox:~$ ifconfig -a
br0 Link encap:Ethernet HWaddr 92:c8:e4:57:50:e2
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 08:00:27:b4:78:a5
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:feb4:78a5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9689500 errors:0 dropped:0 overruns:0 frame:0
TX packets:3594826 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:40367033 (40.3 MB) TX bytes:316821133 (316.8 MB)
eth1 Link encap:Ethernet HWaddr 08:00:27:6c:e0:ac
inet6 addr: fe80::a00:27ff:fe6c:e0ac/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:331912 errors:0 dropped:0 overruns:0 frame:0
TX packets:392537 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:45464309 (45.4 MB) TX bytes:573045550 (573.0 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:105663 errors:0 dropped:0 overruns:0 frame:0
TX packets:105663 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8750846 (8.7 MB) TX bytes:8750846 (8.7 MB)
veth0 Link encap:Ethernet HWaddr be:db:ce:49:3c:8c
inet addr:192.168.10.20 Bcast:0.0.0.0 Mask:255.255.255.255
inet6 addr: fe80::bcdb:ceff:fe49:3c8c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:24 errors:0 dropped:0 overruns:0 frame:0
TX packets:58 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1224 (1.2 KB) TX bytes:9977 (9.9 KB)
weng@weng-VirtualBox:~$ sudo brctl addif br0 eth1
weng@weng-VirtualBox:~$ sudo brctl addif br0 veth0
weng@weng-VirtualBox:~$ ifconfig -a
br0 Link encap:Ethernet HWaddr 08:00:27:6c:e0:ac
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 08:00:27:b4:78:a5
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:feb4:78a5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9689676 errors:0 dropped:0 overruns:0 frame:0
TX packets:3594992 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:40391737 (40.3 MB) TX bytes:316840728 (316.8 MB)
eth1 Link encap:Ethernet HWaddr 08:00:27:6c:e0:ac
inet6 addr: fe80::a00:27ff:fe6c:e0ac/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:331966 errors:0 dropped:0 overruns:0 frame:0
TX packets:392598 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:45472046 (45.4 MB) TX bytes:573059466 (573.0 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:105825 errors:0 dropped:0 overruns:0 frame:0
TX packets:105825 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8768711 (8.7 MB) TX bytes:8768711 (8.7 MB)
veth0 Link encap:Ethernet HWaddr be:db:ce:49:3c:8c
inet addr:192.168.10.20 Bcast:0.0.0.0 Mask:255.255.255.255
inet6 addr: fe80::bcdb:ceff:fe49:3c8c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:24 errors:0 dropped:0 overruns:0 frame:0
TX packets:58 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1224 (1.2 KB) TX bytes:9977 (9.9 KB)
weng@weng-VirtualBox:~$ sudo ip netns exec blue ping 192.168.10.20
PING 192.168.10.20 (192.168.10.20) 56(84) bytes of data.
From 192.168.10.21 icmp_seq=1 Destination Host Unreachable
From 192.168.10.21 icmp_seq=2 Destination Host Unreachable
From 192.168.10.21 icmp_seq=3 Destination Host Unreachable
^C
--- 192.168.10.20 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 5066ms
pipe 3
weng@weng-VirtualBox:~$ sudo ip netns exec blue ping 192.168.10.29
PING 192.168.10.29 (192.168.10.29) 56(84) bytes of data.
From 192.168.10.21 icmp_seq=1 Destination Host Unreachable
From 192.168.10.21 icmp_seq=2 Destination Host Unreachable
From 192.168.10.21 icmp_seq=3 Destination Host Unreachable
^C
--- 192.168.10.29 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 5010ms
pipe 3
weng@weng-VirtualBox:~$
weng@weng-VirtualBox:~$ sudo ip netns exec blue ping 192.168.10.29
[sudo] password for weng:
PING 192.168.10.29 (192.168.10.29) 56(84) bytes of data.
From 192.168.10.21 icmp_seq=1 Destination Host Unreachable
From 192.168.10.21 icmp_seq=2 Destination Host Unreachable
From 192.168.10.21 icmp_seq=3 Destination Host Unreachable
From 192.168.10.21 icmp_seq=4 Destination Host Unreachable
From 192.168.10.21 icmp_seq=5 Destination Host Unreachable
From 192.168.10.21 icmp_seq=6 Destination Host Unreachable
^C
--- 192.168.10.29 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6014ms
pipe 3
weng@weng-VirtualBox:~$ sudo ifconfig br0 192.168.10.1 netmask 255.255.255.0 up
weng@weng-VirtualBox:~$ ifconfig -a
br0 Link encap:Ethernet HWaddr 08:00:27:6c:e0:ac
inet addr:192.168.10.1 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe6c:e0ac/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:830 (830.0 B) TX bytes:7666 (7.6 KB)
eth0 Link encap:Ethernet HWaddr 08:00:27:b4:78:a5
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:feb4:78a5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9698506 errors:0 dropped:0 overruns:0 frame:0
TX packets:3603495 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:41771143 (41.7 MB) TX bytes:317842105 (317.8 MB)
eth1 Link encap:Ethernet HWaddr 08:00:27:6c:e0:ac
inet6 addr: fe80::a00:27ff:fe6c:e0ac/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:351589 errors:0 dropped:0 overruns:0 frame:0
TX packets:395546 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:48957881 (48.9 MB) TX bytes:573716498 (573.7 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:113711 errors:0 dropped:0 overruns:0 frame:0
TX packets:113711 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:9661222 (9.6 MB) TX bytes:9661222 (9.6 MB)
veth0 Link encap:Ethernet HWaddr be:db:ce:49:3c:8c
inet addr:192.168.10.20 Bcast:0.0.0.0 Mask:255.255.255.255
inet6 addr: fe80::bcdb:ceff:fe49:3c8c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:45 errors:0 dropped:0 overruns:0 frame:0
TX packets:109 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2106 (2.1 KB) TX bytes:19299 (19.2 KB)
weng@weng-VirtualBox:~$ sudo ip netns exec blue ping 192.168.10.20
PING 192.168.10.20 (192.168.10.20) 56(84) bytes of data.
64 bytes from 192.168.10.20: icmp_req=1 ttl=64 time=0.253 ms
64 bytes from 192.168.10.20: icmp_req=2 ttl=64 time=0.088 ms
64 bytes from 192.168.10.20: icmp_req=3 ttl=64 time=0.033 ms
64 bytes from 192.168.10.20: icmp_req=4 ttl=64 time=0.101 ms
64 bytes from 192.168.10.20: icmp_req=5 ttl=64 time=0.114 ms
64 bytes from 192.168.10.20: icmp_req=6 ttl=64 time=0.071 ms
^C
--- 192.168.10.20 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 4997ms
rtt min/avg/max/mdev = 0.033/0.110/0.253/0.068 ms
weng@weng-VirtualBox:~$ sudo ip netns exec blue ifconfig -a
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:50 errors:0 dropped:0 overruns:0 frame:0
TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5320 (5.3 KB) TX bytes:5320 (5.3 KB)
veth1 Link encap:Ethernet HWaddr 26:22:a1:15:d7:e3
inet addr:192.168.10.21 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::2422:a1ff:fe15:d7e3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:210 errors:0 dropped:0 overruns:0 frame:0
TX packets:55 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:30411 (30.4 KB) TX bytes:2954 (2.9 KB)
weng@weng-VirtualBox:~$
