How to setup NAT on your PVE node background

How to setup NAT on your PVE node

This tutorial is mainly made for PVE users. It should work on every linux that meets the criteria provided bellow, but the configuration might differ.

The tutorial assumes that you have a debian or debian-like distribution with iptables and ifupdown2 installed and using networking service for network management. It also assumes that your external address is 192.168.1.100/24 and the network in NAT is 10.10.10.0/24. These may vary, so pay attention what values you configure.

The initial networking configuration should look something like the following file. We have the loopback interface, at least one - in my case four - physical interfaces (enp*) and at least two virtual bridges (vmbr*).

In my case I am only using the enp3s0f0 NIC. I have the vmbr0 bridge in the external network and vmbr1010 will be used in the internal (NAT-ed) network.

auto lo
iface lo inet loopback

iface enp3s0f0 inet manual

iface enp3s0f1 inet manual

iface enp4s0f0 inet manual

iface enp4s0f1 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.1.100/24
gateway 192.168.1.1
bridge-ports enp3s0f0
bridge-stp off
bridge-fd 0

auto vmbr1010
iface vmbr1010 inet static
address 10.10.10.1/24
bridge-ports enp3s0f1
bridge-stp off
bridge-fd 0

We can not apply the required changes via the web UI, so we need to edit the file directly. As noted in PVE network doc, it is recommended to copy the /etc/network/interfaces file to /etc/network/interfaces.new and edit the latter one.

Add the following lines to the iface vmbr1010 section. This will make sure that every time the vmbr1010 bridge is enabled the rules are applied. The first rule will make sure that ip forwarding is enabled on this system, the second one will forward all the traffic to fwbr chain which will make sure that the VM-level firewall rules will still be applied, the third one will actually be "translating" the packets and the fourth one will disable packet forwarding when the vmbr1010 interface will be disabled.

auto vmbr1010
iface vmbr1010 inet static
address 10.10.10.1/24
bridge-ports enp3s0f1
bridge-stp off
bridge-fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
post-up iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -o vmbr0 -j SNAT --to-source 192.168.1.100
post-down iptables -t nat -D POSTROUTING -s 10.10.10.0/24 -o vmbr0 -j SNAT --to-source 192.168.1.100

Now we need to apply the newly created changes. There ware two ways of applying them;
The "reboot" way: restart the node and the settings will be automatically applied on startup.
The "reload" way: install the ifupdown2 package on the PVE node by executing apt-get install ifupdown2 -y in the node terminal, navigate to node-level network configuration and click on the "Apply Configuration" button in the toolbar.

Now create a few virtual machines with their virtual NICs connected to the vmbr1010 bridge and configure their networking in static IPv4 configuration. Each VM gets its address following the pattern 10.10.10.{VMID} where is {VMID} replaced with the id of the virtual machine. Their default gateway should be set to 10.10.10.1 which is the internal address of the PVE node itself.

You can now try to ping some external server and if everything is setup correctly you will get a reply.

Congrats, you have now successfully setup a NAT-ed network on your PVE node which also respects the vm-level firewall rules.