I was building nested VMware vSphere lab for POCs. While configuration I was facing strange issue after installing nested ESXi server as virtual machine on Physical ESXi Server, I created Few VMs and VMKernel ports on nested ESXi VM. But when I was trying to connect VMs or new VMKernel port it was failing. I tried to ping Secondary configured VMK port IPs or VM under nested ESXi server it was also failing. The main part was primary VMK0 was reachable and working fine, I was able to open the ESXi portal or join them to the vCenter Server.
To resolve this issue you need to edit and configure security settings of Physical Esxi server's Virtual Switch (vSwitch) or Virtual Portgroup where ESXi VM or VMkernel port is added/configured. In the vSwitch Security configuration, there are 3 settings Promiscuous mode, MAC address changes and Forged Transmits. They are set to Reject by default which need to set as Accept manually.
Promiscuous Mode: This option is used to receive all incoming traffic on the configured port group on that physical host. This settings is used to monitor/sniff network traffic. When this settings is enabled VM network adapter can read and receive all the network traffic.
MAC Address changes: One of the security settings for a virtual machine is MAC address changes. This setting determines how the virtual switch handles the traffic that enters a virtual machine with a different MAC address than the one assigned by the hypervisor. If MAC address changes are set to Accept, the virtual switch forwards the traffic to the virtual machine, regardless of the MAC address mismatch. This allows the virtual machine to spoof or change its MAC address and receive traffic that was meant for another VM on the same network. This could be useful for some network applications or testing scenarios, but it could also pose a security risk if an attacker gains access to a virtual machine and tries to intercept or modify the network traffic of other VMs.
Forged Transmits: The Forged transmits option is a security policy of a virtual switch that controls how ESXi handles traffic from a virtual machine with a different MAC address than the one configured in the VMX file. This option can prevent MAC impersonation attacks, where an attacker tries to spoof the MAC address of an authorized device. If the option is set to Reject, ESXi will drop any packets that have a mismatch between the source MAC address and the effective MAC address of the virtual machine adapter. If the option is set to Accept, ESXi will allow such packets to pass through.
To configure security Edit Settings of Virtual Switch (vSwitch), go to Security and set Promiscuous mode, MAC address changes and Forged transmits to Accept.
Useful Articles
PART 1 : BUILDING AND BUYING GUIDE IDEAS FOR VMWARE LAB
PART 2 : BUILDING AND HARDWARE BUYING GUIDE IDEAS FOR VMWARE LAB
PART 3 : MY VSPHERE LAB CONFIGURATION ON VMWARE WORKSTATION
PART 4 : CONFIGURING VMWARE WORKSTATION NETWORKING IN HOME LAB
PART 5 : CONFIGURING STORAGE IN VMWare WORKSTATION FOR OPTIMAL SPEED
PART 6 : CONFIGURE VMWARE WORKSTATION TO SAVE SSD SPACE AND TIME
PART 7 : CREATING NESTED VMWARE ESXI SERVER VM IN HOMELAB ON VMWARE WORKSTATION
PART 8 : CPU COOLING SOLUTION FOR MY HOME LAB ON VMWARE WORKSTATION
Part 9: Guide to building a Home Lab