Saturday, July 30, 2016

CoreOS as a NAT Gateway

Untitled Document.md

Overview

Recently I’ve started working a bit with CoreOS and decided to test it as a NAT gateway for my lab environment. This guide provides a walk through of how I configured CoreOS for this purpose. You will need a workstation with access to (preferably) a bash shell in order to follow along.

This post is the first in a series on installing a CoreOS based lab environment. The other relevant posts are:

Description of the Environment

My lab environment consists of a handful of physical servers running VMware esx and the following four infrastructure networks:

  • public – public facing external network x.x.x.x/y
  • mgmt – private management network 10.127.0.0/24
  • data – private data transfer network 10.127.1.0/24
  • dmz – private dmz network 10.127.2.0/24

All physical servers have access to all 4 of the infrastructure networks. The CoreOS VM (referred to as “natcore”) is connected to all 4 networks, is configured as the default gateway for the private nets, and is set up for IP masquerade on its public interface.

My personal preference is to name interfaces in such a way as to reflect their function. In order to accomplish this I will make use of systemd.link files to name interfaces based on their mac-address. Additionally, I will use systemd.network files to set up static IP addresses for the internal interfaces. Finally, I will provide iptables rules files (v4 and v6) to both filter requests to natcore and provide the actual nat functionality.

Preparing the configdrive

The installation will make use of the config drive feature of CoreOS to configure the system upon first boot. As a first step in this process we will define a few variables which will be used to populate the cloud-config file. These are as follows:

  • COS_CONFIG_DRIVE_PATH: the base path of the config drive
  • COS_HOSTNAME: the hostname of the machine
  • COS_LAB_NET: the summary address for the lab ip space
  • COS_PUB_IF: the name of the public interface
  • COS_PUB_IF_MAC: the mac address of the public net interface
  • COS_MGMT_IF: the name of the mgmt interface
  • COS_MGMT_IF_MAC: the mac address of the mgmt net interface
  • COS_MGMT_IF_IP: the ip/mask of the mgmt net interface
  • COS_DATA_IF: the name of the data interface
  • COS_DATA_IF_MAC: the mac address of the data net interface
  • COS_DATA_IF_IP: the ip/mask of the data net interface
  • COS_DMZ_IF: the name of the dmz interface
  • COS_DMZ_IF_MAC: the mac address of the dmz net interface
  • COS_DMZ_IF_IP: the ip/mask of the dmz net interface
  • COS_USERNAME: the primary user to configure
  • COS_AUTH_KEY: rsa public key to be used for key-based ssh

From the terminal of your workstation define the variables. You’ll need to adjust the username, auth key, and mac-addresses to fit your environment.

COS_CONFIG_DRIVE_PATH=/tmp/configdrive
COS_HOSTNAME=natcore
COS_LAB_NET=10.127.0.0/22
COS_PUB_IF=pub0
COS_PUB_IF_MAC=00:0c:29:83:dd:8e
COS_MGMT_IF=mgmt0
COS_MGMT_IF_MAC=00:50:56:00:00:01
COS_MGMT_IF_IP=10.127.0.1/24
COS_DATA_IF=data0
COS_DATA_IF_MAC=00:50:56:01:00:01
COS_DATA_IF_IP=10.127.1.1/24
COS_DMZ_IF=dmz0
COS_DMZ_IF_MAC=00:50:56:02:00:01
COS_DMZ_IF_IP=10.127.2.1/24
COS_USERNAME=user1
COS_AUTH_KEY="ssh-rsa AAAAB..."

Create the file structure for the config drive.

mkdir -p ${COS_CONFIG_DRIVE_PATH}/openstack/latest

Create the cloud-config file.

cat > ${COS_CONFIG_DRIVE_PATH}/openstack/latest/user_data << EOF
#cloud-config
hostname: "${COS_HOSTNAME}"
manage_etc_hosts: "localhost"
users:
- name: "${COS_USERNAME}"
    groups:
    - "sudo"
    - "rkt"
    - "wheel"
    ssh-authorized-keys:
    - "${COS_AUTH_KEY}"
        
write_files:
- path: "/etc/systemd/network/10-${COS_PUB_IF}.link"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    MACAddress=${COS_PUB_IF_MAC}      
    [Link]
    Name=${COS_PUB_IF}

- path: "/etc/systemd/network/10-${COS_MGMT_IF}.link"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    MACAddress=${COS_MGMT_IF_MAC}
    [Link]
    Name=${COS_MGMT_IF}
    
- path: "/etc/systemd/network/10-${COS_DATA_IF}.link"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    MACAddress=${COS_DATA_IF_MAC}
    [Link]
    Name=${COS_DATA_IF}
    
- path: "/etc/systemd/network/10-${COS_DMZ_IF}.link"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    MACAddress=${COS_DMZ_IF_MAC}
    [Link]
    Name=${COS_DMZ_IF}
    
- path: "/etc/systemd/network/20-${COS_MGMT_IF}.network"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    Name=${COS_MGMT_IF}
    [Network]
    Address=${COS_MGMT_IF_IP}
    
- path: "/etc/systemd/network/20-${COS_DATA_IF}.network"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    Name=${COS_DATA_IF}
    [Network]
    Address=${COS_DATA_IF_IP}
    
- path: "/etc/systemd/network/20-${COS_DMZ_IF}.network"
    permissions: "0644"
    owner: "root:root"
    content: |
    [Match]
    Name=${COS_DMZ_IF}
    [Network]
    Address=${COS_DMZ_IF_IP}
    
- path: "/var/lib/iptables/rules-save"
    permissions: "0644"
    owner: "root:root"
    content: |
    *filter
    :INPUT DROP [0:0]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]

    # allow to loopback
    -A INPUT -i lo -j ACCEPT
    
    # allow established
    -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

    # allow icmp
    -A INPUT -p icmp --icmp-type any -j ACCEPT

    # allow ssh
    -A INPUT -p tcp --dport 22 -j ACCEPT

    # allow dhcp from internal interfaces
    -A INPUT -i ${COS_MGMT_IF} -p udp --dport 67:68 -j ACCEPT
    -A INPUT -i ${COS_DATA_IF} -p udp --dport 67:68 -j ACCEPT
    -A INPUT -i ${COS_DMZ_IF} -p udp --dport 67:68 -j ACCEPT

    # allow dns from internal networks
    -A INPUT -s ${COS_LAB_NET} -p udp --dport 53 -j ACCEPT
    
    # allow ntp from internal networks
    -A INPUT -s ${COS_LAB_NET} -p udp --dport 123 -j ACCEPT

    COMMIT

    *nat
    -A POSTROUTING -o ${COS_PUB_IF} -s ${COS_LAB_NET} -j MASQUERADE

    COMMIT
    
- path: "/var/lib/ip6tables/rules-save"
    permissions: "0644"
    owner: "root:root"
    content: |
    *filter
    :INPUT DROP [0:0]
    :FORWARD DROP [0:0]
    :OUTPUT ACCEPT [0:0]
    COMMIT
    
coreos:
update:
    # I dont want my nat gateway rebooting automatically
    reboot-strategy: off
units:
    - name: iptables-restore.service
    enable: true
    - name: ip6tables-restore.service
    enable: true
    - name: ntpd.service
    enable: true
    command: start
    - name: systemd-timesyncd.service
    enable: true
    
EOF

Create an ISO image from our config drive directory structure.

mkisofs -R -V config-2 -o configdrive.iso ${COS_CONFIG_DRIVE_PATH}
rm -r ${COS_CONFIG_DRIVE_PATH}

Copy the file “configdrive.iso” to the datastore of the esx machine hosting

the CoreOS machine (eg. /vmfs/volumes/datastore1).

Boot the machine

For the purposes of my lab I generally install from ISO. To do so, download the latest CoreOS image and transfer it to the datastore of the esx machine hosting the CoreOS machine (eg. /vmfs/volumes/datastore1).

Next, create a VM with (roughly) the following specs:

  • 2G memory
  • 1 disk (size based on your planned usage)
  • 4 nics (e1000) attached to networks public,mgmt,data,dmz
  • cdrom drive (with CoreOS image attached)
  • cdrom drive (with configdrive.iso attached)

Boot the VM from the CoreOS ISO image.

Once the machine boots you may ssh to it using the auth key provided as part of the cloud-config. Upon login you will notice that your config files specified as part of the cloud-config are present, however, they are not applied. This is because CoreOS creates these files after systemd has alread registered network interfaces and started iptables. This is not a big deal since the very next step is to install to disk and reboot.

sudo coreos-install -d /dev/sda -c /media/configdrive/openstack/latest/user_data

The above command will download/install CoreOS using the cloud-config created as part of our config drive. Once it completes then you should reboot the system. Upon reboot CoreOS will configure itself using the cloud-config file. As before, the config files are created but not applied. Rebooting a second time will apply them.

After a second reboot you will notice that your network interfaces are named according to the systemd.link rules that we defined and that iptables rules are applied. CoreOS seems to have ip forwarding enabled by default, but it is good to verify that the following command returns a 1.

cat /proc/sys/net/ipv4/ip_forward

Some additional rules

I tend to run an rdp machine within my lab and prefer to pipe rdp through directly to that host. In order to enable this we’ll need to set up a static nat rule which, in my case, will forward rdp to 10.127.0.2. Edit the file /var/lib/iptables/rules-save and make the nat section look as follows:

*nat
# rdp
-A PREROUTING -i pub0 -p tcp --dport 3389 -j DNAT --to 10.127.0.2

-A POSTROUTING -o pub0 -s 10.127.0.0/22 -j MASQUERADE

You may now apply this change by running

iptables-restore /var/lib/iptables/rules-save

You should now have a fully functional nat gateway. In a later post I’ll cover adding dhcp and tftp services to this machine.

No comments:

Post a Comment