Juniper QFX 5100 & VMware ESXI Host NIC Teaming -Design Consideration

The objective of this article is to highlight design consideration for NIC Teaming between  Juniper QFX 5100 (Virtual Chassis -VC) and VMWare ESXI host.

Reference topology is as under:-

We have 2 x Juniper QFX 5100 48S switches which are deployed as VC in order to provide connectivity to  compute machines. All compute machines are running VMWare ESXI Hyper-visor. Link Aggregation Group (LAG or Active/ Active NIC Teaming) is  required between compute machines and QFX 5100 VC.

• Data Traffic from server to switch – xe-0/0/0  interface on both switches connected to NIC 3 & 4 on a single Compute Machine.
• ESXI Host Management  and V-Motion Traffic from server to Switch-  xe-0/0/45 interface from both switches connected to NIC 1 & 2 ports on compute machine.
• VLANs-ID
  • Data VLANs – 116, 126
  • V-Motion- 12
  • ESXI Management-11

Hence,the requirement is to configure  LAG (Active/ Active NIC Teaming) between compute machines and network switch for optimal link utilization in addition to fault tolerance if in case one physical link goes down between network switch and compute machine.

In order to achieve the required results one’s needs to understand default load balancing mechanism over LAG member interfaces in Juniper devices and same load balancing mechanism must be  configured on VMware ESXI  for NIC Teaming.

• Juniper default load balancing over LAG member interfaces is based on “layer 2 payload” and it takes into consideration “Src IP”, “Dst IP”, Src Port” and Dst Port
• In order to support similar behavior on VMWare ESXI hosts Active/ Active NIC teaming must be enabled with “Route Based on IP Hash”

• Data Traffic
  •  LAG  will be configured on Juniper switch with interface-mode trunk and all required VLANs will be allowed
  • Active-Active NIC Teaming must be enables with “Route Based on IP Hash”  (LACP only supported in VCenter vDS where as without V-Center we can configure simple LAG with out LACP)
•  VM-Motion and ESXI Management Traffic
  • VMWare does not recommend  Active-Active NIC Teaming over a link needed for VM-Kernal (VM-Motion). So Active/ Passive NIC teaming will be configured for such link with “Route Based on Originating Port ID” .
  •  Both links on Juniper Switch will be configured as trunk  by allowing both VM-Motion and ESXI MGMT VLANs and in addition ESXI MGMT VLAN must be allowed as native VLAN.