FiOS “Three-Router” with VyOS and ESXi, Part 9: Secondary Router Configuration

5 minute read

The secondary router requires much less configuration to get working, though there is a slight twist. Let’s briefly rediscuss what the role of the secondary router is.

In essence, the secondary router’s job is to sit between the Verizon hardware router and the primary router, allowing the VZ hardware to believe it is connected directly to the FiOS ISP network. It does this by assigning (via DHCP) the VZ router the same public IP that the FiOS ISP network has assigned the primary router for actual Internet connectivity, and then performing Network Address Translation to convert packets originating from the VZ router to correct IP segment for the home network. They can then be handled by the primary router just like any other internal host’s traffic.

Recall that for our secondary router, the first vNIC is placed on the Home Network port group on the ESXi virtual switch, and the second vNIC is placed on the Verizon Router Network. Let’s log into our secondary router’s console and run some commands to confirm that we see two interfaces.

show interfaces

The interface called eth0 corresponds to the adapter on the Home Network port group, and the eth1 interface corresponds to the adapter on the Verizon Router Network. Let’s configure the eth0 interface first.

configure
set interfaces ethernet eth0 address 10.0.0.2/24
set interfaces ethernet eth0 address 10.0.0.3/24
set interfaces ethernet eth0 duplex auto
set interfaces ethernet eth0 speed auto

The eth0 interface will have two static addresses assigned to it on the internal network. 10.0.0.2 will be the address for the secondary router itself, and we will configure 10.0.0.3 to be the address that the VZ router will use via 1:1 NAT.

Next we need to configure eth1, which will be the secondary router’s interface to the Verizon Router Network and the Verizon router itself. In order to imitate the external network for the VZ hardware router, we need to review the settings for our primary router and pull them over.

On the primary router:

vyos@primary-router:~$ show interfaces ethernet    
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface        IP Address                        S/L  Description
---------        ----------                        ---  -----------
eth0             108.0.0.123/24                    u/u  FiOS Public Internet 
eth1             10.0.0.1/24                       u/u  Home Network 
eth2             10.10.10.1/24                     u/u  Lab Network 

We can see that the DHCP address that the FiOS ISP network has assigned to our primary router is 108.0.0.123. Additionally, this address is on a /24 network, which has 256 addresses. This is an example only; your address (and possibly network size) will be different. We need to recreate the network presented to primary-eth0 for the VZ router, using secondary-eth1 to do so.

On the secondary router:

configure
set interfaces ethernet eth1 address 108.0.0.1/24
set interfaces ethernet eth1 duplex auto
set interfaces ethernet eth1 speed auto

Basically, we are setting the eth1 address to be in the same IP space as what we pulled from primary-eth0. Assuming a /24 network, you can just take the first three octets of that address and add a .1 to the end.

Next we need to set up DHCP to statically assign the correct address to to hardware router. To do this we will need the WAN MAC address of the VZ hardware router once more. You can read it from the label on the router again, or run the following command on the primary router to pull it up:

show interfaces ethernet eth0 | grep -i link/ether | gawk -F " " '{print $2}'

Let’s start our DHCP configuration with some basic configs:

set service dhcp disabled false
set service dhcp shared-network-name VZ-RouterNet
set service dhcp shared-network-name VZ-RouterNet authoritative enable

Now we need to configure the correct subnet. This will be the net that we are impersonating from primary-eth0; continuing from our example earlier it is 108.0.0.0/24.

set service dhcp shared-network-name VZ-RouterNet subnet 108.0.0.0/24

Let’s set a short 60 second lease time, to allow us to quickly make changes if the upstream IP at primary-eth0 ever changes.

set service dhcp shared-network-name VZ-RouterNet subnet 108.0.0.0/24 lease 60

The default router for this net should be the secondary router, which listens on this net at 108.0.0.1.

set service dhcp shared-network-name VZ-RouterNet subnet 108.0.0.0/24 default-router 108.0.0.1

Since this VZ router is used to run the Verizon services (such as set-top boxes) it makes sense to use the Verizon DNS servers.

set service dhcp shared-network-name VZ-RouterNet subnet 108.0.0.0/24 dns-server 68.237.161.14

The network in our example is a /24, which has 256 addresses. We’re going to make our DHCP pool start after our router (.1) and stop before the broadcast address (.255). This shouldn’t really matter in any case, since we are going to statically-assign the appropriate address in the next step.

set service dhcp shared-network-name VZ-RouterNet subnet 108.0.0.0/24 start 108.0.0.2 stop 108.0.0.254

We always want the hardware router to get the same address that primary-eth0 has, so we’re going to statically assign it based on the WAN MAC of the hardware router. Make sure to substitute the correct MAC from your hardware here, as well as the correct IP from primary-eth0.

set service dhcp shared-network-name VZ-RouterNet subnet 108.0.0.0/24 static-mapping vz-router mac-address 00:11:22:33:44:55
set service dhcp shared-network-anme VZ-RouterNet subnet 108.0.0.0/24 static-mapping vz-router ip-address 108.0.0.123

Lastly, we need to set up 1:1 NAT so that traffic can pass through the secondary router appropriately. Run these commands to set up the NAT rules, being sure to substitute the correct public IP where appropriate.

set nat destination rule 10 destination address 10.0.0.3
set nat destination rule 10 inbound-interface eth0
set nat destination rule 10 protocol all
set nat destination rule 10 translation address 108.0.0.123

set nat source rule 10 outbound-interface eth0
set nat source rule 10 protocol all
set nat source rule 10 source address 108.0.0.123
set nat source rule 10 translation address 10.0.0.3

That pretty much does it for secondary router configs. Review, commit, save.

compare
commit
save

You can now plug your Verizon router into the appropriate port on your switch (with VLAN 102) and power it up. If all is well the hardware router will come online and start functioning. You can test this by going to http://10.0.0.3/ with a web browser from any host on the 10.0.0.0/24 internal network. If you are greeted with the Verizon router login page, things are working.