The OVN Gateway Router

Posted on Tuesday, Sep 27, 2016


Building upon my previous post I will now add an OVN gateway router into the lab setup. This gateway router will provide access to the lab network from within our overlay network.

The Lab

In order to demonstrate the gateway router we will need to add another physical network to our Ubuntu hosts. For my purposes I will add the network via eth1 of each of my hosts. The final lab diagram is illustrated below.

Introducing the OVN L3 Gateway

An OVN Gateway serves as an onramp/offramp between the overlay network and the physical network. They come in two flavors: layer-2 which bridge an OVN logical switch into a VLAN, and layer-3 which provide a routed connection between an OVN router and the physical network. For the purposes of this lab we will focus on creating a layer-3 gateway which will serve as the demarcation point between our physical and logical networks.

Unlike a distributed logical router (DLR), an OVN gateway router is centralized on a single host (chassis) so that it may provide services which cannot yet be distributed (NAT, load balancing, etc…). As of this publication there is a restriction that gateway routers may only connect to other routers via a logical switch, whereas DLRs may connect to one other directly via a peer link. Work is in progress to remove this restriction.

It should be noted that it is possible to have multiple gateway routers tied into an environment, which means that it is possible to perform ingress ECMP routing into logical space. However, it is worth mentioning that OVN currently does not support egress ECMP between gateway routers. Again, this is being looked at as a future enhancement.

Make ubuntu1 an OVN Host

Rather than using a host which houses VMs, lets use ubuntu1 for our OVN gateway router host. Begin by registering ubuntu1 with OVN Central (itself):

  ovs-vsctl set open . external-ids:ovn-remote=tcp:
  ovs-vsctl set open . external-ids:ovn-encap-type=geneve
  ovs-vsctl set open . external-ids:ovn-encap-ip=

And verify connectivity:

  [email protected]:~# netstat -antp | grep
  tcp        0      0         ESTABLISHED 4999/ovsdb-server
  tcp        0      0          ESTABLISHED 15212/ovn-controller

Also, be sure to create the integration bridge if wasn’t created automatically by OVN:

  ovs-vsctl list-br

  # create if above command had no output
  ovs-vsctl add-br br-int -- set Bridge br-int fail-mode=secure

OVN Logical Design

Lets review the planned design before we start configuring things. The OVN logical network we are creating is illustrated below.

As you can see we are adding the following new components:

  • OVN gateway router (edge1)
  • logical switch (transit) used to connect the edge1 and tenant1 routers
  • logical switch (outside) used to connect edge1 to the lab network

Adding the L3 Gateway

As mentioned earlier the gatway router will be bound to a specific chassis (ubuntu1 in our case). In order to accomplish this binding we will need to locate the chassis id for ubuntu1. Using the ovn-sbctl show command from ubuntu1 you should see output similar to this:

  ovn-sbctl show
  Chassis "833ae1bd-ced3-494a-a95b-f2dc54172b71"
      hostname: "ubuntu1"
      Encap geneve
          ip: ""
          options: {csum="true"}
  Chassis "239f2c28-90ff-468f-a701-655585c630bf"
      hostname: "ubuntu3"
      Encap geneve
          ip: ""
          options: {csum="true"}
      Port_Binding "dmz-vm2"
      Port_Binding "inside-vm4"
  Chassis "517d558e-158a-4cb2-8870-283e9d39685e"
      hostname: "ubuntu2"
      Encap geneve
          ip: ""
          options: {csum="true"}
      Port_Binding "inside-vm3"
      Port_Binding "dmz-vm1"

Copy the Chassis UUID of the ubuntu1 host for use below.

Create the new logical router. Be sure to substitute {chassis_id} with a valid UUID. From ubuntu1:

  # create router edge1
  ovn-nbctl create Logical_Router name=edge1 options:chassis={chassis_uuid}

  # create a new logical switch for connecting the edge1 and tenant1 routers
  ovn-nbctl ls-add transit

  # edge1 to the transit switch
  ovn-nbctl lrp-add edge1 edge1-transit 02:ac:10:ff:00:01
  ovn-nbctl lsp-add transit transit-edge1
  ovn-nbctl lsp-set-type transit-edge1 router
  ovn-nbctl lsp-set-addresses transit-edge1 02:ac:10:ff:00:01
  ovn-nbctl lsp-set-options transit-edge1 router-port=edge1-transit

  # tenant1 to the transit switch
  ovn-nbctl lrp-add tenant1 tenant1-transit 02:ac:10:ff:00:02
  ovn-nbctl lsp-add transit transit-tenant1
  ovn-nbctl lsp-set-type transit-tenant1 router
  ovn-nbctl lsp-set-addresses transit-tenant1 02:ac:10:ff:00:02
  ovn-nbctl lsp-set-options transit-tenant1 router-port=tenant1-transit

  # add static routes
  ovn-nbctl lr-route-add edge1 ""
  ovn-nbctl lr-route-add tenant1 ""

  ovn-sbctl show

Notice the port bindings on the ubuntu1 host. You can now test connectivity to edge1 from vm1 on ubuntu2:

  [email protected]:~# ip netns exec vm1 ping
  PING ( 56(84) bytes of data.
  64 bytes from icmp_seq=1 ttl=253 time=1.07 ms
  64 bytes from icmp_seq=2 ttl=253 time=1.13 ms
  64 bytes from icmp_seq=3 ttl=253 time=1.00 ms

Connecting to the “data” Network

We’re going to use the eth1 interface of ubuntu1 as our connection point between the edge1 router and the “data” network. In order to accomplish this we’ll need to set up OVN to used the eth1 interface directly through a dedicated OVS bridge. This type of connection is known as a “localnet” in OVN.

From ubuntu1:

  # create new port on router 'edge1'
  ovn-nbctl lrp-add edge1 edge1-outside 02:0a:7f:00:01:29

  # create new logical switch and connect it to edge1
  ovn-nbctl ls-add outside
  ovn-nbctl lsp-add outside outside-edge1
  ovn-nbctl lsp-set-type outside-edge1 router
  ovn-nbctl lsp-set-addresses outside-edge1 02:0a:7f:00:01:29
  ovn-nbctl lsp-set-options outside-edge1 router-port=edge1-outside

  # create a bridge for eth1
  ovs-vsctl add-br br-eth1

  # create bridge mapping for eth1. map network name "dataNet" to br-eth1
  ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=dataNet:br-eth1

  # create localnet port on 'outside'. set the network name to "dataNet"
  ovn-nbctl lsp-add outside outside-localnet
  ovn-nbctl lsp-set-addresses outside-localnet unknown
  ovn-nbctl lsp-set-type outside-localnet localnet
  ovn-nbctl lsp-set-options outside-localnet network_name=dataNet

  # connect eth1 to br-eth1
  ovs-vsctl add-port br-eth1 eth1

Test connectivity to edge1-outside from vm1

  [email protected]:~# ip netns exec vm1 ping
  PING ( 56(84) bytes of data.
  64 bytes from icmp_seq=1 ttl=253 time=1.74 ms
  64 bytes from icmp_seq=2 ttl=253 time=0.781 ms
  64 bytes from icmp_seq=3 ttl=253 time=0.582 ms

Giving the Ubuntu Hosts Access to the “data” Network

Lets give the Ubuntu hosts a presence on the data network. For ubuntu2/ubuntu3 its simply a matter of setting an IP on their physical nics (eth1 in my setup). For ubuntu1 we’ll set an IP on the br-eth1 interface.

On ubuntu1:

  ip addr add dev br-eth1
  ip link set br-eth1 up

On ubuntu2:

  ip addr add dev eth1
  ip link set eth1 up

On ubuntu3:

  ip addr add dev eth1
  ip link set eth1 up

Testing from ubuntu1 to edge1

  [email protected]:~# ping  
  PING ( 56(84) bytes of data.
  64 bytes from icmp_seq=1 ttl=254 time=0.563 ms
  64 bytes from icmp_seq=2 ttl=254 time=0.290 ms
  64 bytes from icmp_seq=3 ttl=254 time=0.333 ms

Configuring NAT

Lets see what happens when we attempt to ping ubuntu1 from vm1:

  [email protected]:~# ip netns exec vm1 ping
  PING ( 56(84) bytes of data.
  --- ping statistics ---
  3 packets transmitted, 0 received, 100% packet loss, time 2016ms

Unsurprisingly, this does not work. Why not? Lets look at the output of a tcpdump from ubuntu1:

  [email protected]:~# tcpdump -i br-eth1
  tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  listening on br-eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
  14:41:53.057993 IP > ICMP echo request, id 19359, seq 1, length 64
  14:41:54.065696 IP > ICMP echo request, id 19359, seq 2, length 64
  14:41:55.073667 IP > ICMP echo request, id 19359, seq 3, length 64

We can see the requests coming in, however our responses are returning through a different interface (not seen in the tcpdump output). This is due to the fact that ubuntu1 has no route to and is responding via its own default gateway. In order to get things working we will need to take 1 of 2 possible approaches:

  1. add static routes on the Ubuntu hosts
  2. set up NAT on the OVN gateway router

We’ll opt for option 2 because it is much less of a hassle than trying to manage static routes.

With OVN there are 3 types of NAT which may be configured:

  • DNAT – used to translate requests to an externally visible IP to an internal IP
  • SNAT – used to translate requests from one or more internal IPs to an externally visible IP
  • SNAT-DNAT – used to create a “static NAT” where an external IP is mapped to an internal IP, and vice versa

Since we don’t need (or want) the public network to be able to directly access our internal VMs, lets focus on allowing outbound SNAT from our VMs. In order to create NAT rules we’ll need to manipulate the OVN northbound database directly. The syntax is a bit strange but I’ll explain later. From ubuntu1:

  # create snat rule which will nat to the edge1-outside interface
  ovn-nbctl -- [email protected] create nat type="snat" logical_ip= \
  external_ip= -- add logical_router edge1 nat @nat

In brief, this command is creating an entry in the “nat” table of the northbound database, storing the resulting UUID within the ovsdb variable “@nat”, and then adding the UUID stored in @nat to the “nat” field of the “edge1” entry in the “logical_router” table of the northbound database. If you want to know the details of the northbound database then be sure to check out the man page for ovn-nb. The man page for ovn-nbctl also explains the command syntax used above.

Testing connectivity from vm1:

  [email protected]:~# ip netns exec vm1 ping
  PING ( 56(84) bytes of data.
  64 bytes from icmp_seq=40 ttl=62 time=2.39 ms
  64 bytes from icmp_seq=41 ttl=62 time=1.61 ms
  64 bytes from icmp_seq=42 ttl=62 time=1.28 ms

As seen above, we can now ping the outside world from our internal VMs.

Final Words

Overlay networks are almost entirely useless unless you can connect them to the outside world. OVN gateways provide a means for making such a connection.

In the next post I will explore another important feature of OVN: the OVN load balancer.