Interfaces/Uplinks for ESXi/Edge – Part 2

by

In these three parts, I will speak about the configuration of the interfaces and uplinks respectively for physical ESXi and Edge VM.

Since NSX-T 2.3 version, Edges can have up to 4 interfaces eth0, fp-eth0, fp-eth1 and fp-eth2. These fp interfaces are assigned to the DPDK fastpath (data plane development kit).

  • Eth0 is used for the Management.
  • fp-eth0 is commonly used in both T1 and T0 Edges for the Overlay network.
  • fp-eth1 and fp-eth2 are commonly used in T0 Edges to provide the North-South Uplink (physical world).

Note: It’s possible to enbale only the fp-eth1 (explain in this topic) however it’s a best practice to configure both fp-eth1 and fp-eth2 for redundancy of the Edge T0 (explain in this topic) -> two different VLANs are required though.
If some fp-eth interfaces are not used it’s recommended to disconnect these interfaces.

Edge T1 (east/west)

T1 Edges are easier to understand so we will start with them.
These Edges have no North/South connection and are confined within our NSX network.
As they are only used by Overlay networks, we will define that two ESXi uplink interfaces are going to be used for active-active overlay.

The important point and difference between NSX Edge and ESXi configuration is that Edges only allow one interface per NVDS!
For the T1 Edges, we only have an Overlay network for that reason we only have one NVDS -> we will have to match our two ESXi uplink interfaces in one NVDS interface via the portgroup (vCenter).

The VLAN Tagging for Edges happens at the portgroup level (vCenter) -> We can set the Transport Vlan to 0 in the NSX-T Profile configuration.

The teaming policy doesn’t allow the Edge to do Active-Passive -> we can’t have an fp-ethX in “Active Uplinks” and another fp-ethY in “Standby Uplinks”.

In the vCenter we find the portgroup with vlan 825 (TEP vlan).
As we mention earlier we can see that both ESXi uplink interfaces are used in Active-Active -> It’is possible because both are used in the Overlay network.

Edge T0 (north/south)

The Edges T0 have to provide the Overlay network communication, as well as the Edges T1. Additionally, they are used to establish the North-South communication (link with the physical world).

If we have only one fp-eth1 assuming the North-South link, we will have two different NVDS (one for the Overlay and the second one for this uplink).
Note: If we had a redundant link the first fp-eth1 will be used in one NVDS with a specific VLAN and the second fp-eth2 interface in another NVDS with a different VLAN.

In this topic we don’t have redundant Edge T0 uplink. In that case, we have two different NVDS.
Here we will play with the two different portgroups (vCenter) in order to always have the two physical links active:

  • (Overlay) pg-nsx-tz-825-t0 will have the Uplink1 active and the Uplink2 passive.
  • (North-South Connection) pg-nsx-edge-epenet-211 will have the Uplink1 active and the Uplink2 passive.

Overlay connectivity

In the vCenter we find the portgroup with the transport vlan 825 (TEP).
As we said earlier, it’s the “Uplink 1” which is in Active mode for the Overlay Network.
Note: The inversion of the ESXi Uplinks with the North-South network (see below).

North-South connectivity

In the vCenter we find the portgroup with the vlan 211 (Noth-South).
As we said earlier, it’s the “Uplink 2” which is in Active mode for the North-South connectivity.
Note: The inversion of the ESXi Uplinks with the Overlay network.

Helpful reference: https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.5/installation/GUID-370D06E1-1BB6-4144-A654-7AF2542C3136.html

Leave a Comment