#VCP #vSphere #Concept ## ✅Requirements In general for compatibility check [VMware's compatibility guides](https://www.vmware.com/resources/compatibility/search.php) - CPU - 2 CPU cores - 64-bit x86 CPU released after 2006 - NX/XD bit enabled in BIOS - For 64-Bit VMS VT-x or AMD RVI enabled in BIOS - RAM - 8GB RAM / 12GB for production - Network - 1GB or faster supported NIC - Storage - SCSI disk or local LUN with unpartitioned space - SATA over SAS or SATA Controller Disks are considered remote - 8GB free space for USB sticks or SD devices - 32GB free space for other devices - boot device must not be shared between ESXi hosts ## Disk Layout vSphere 7.0 and higher has a completely new disk layout. ![[_media/vSphere - ESXi-2024-05-14.png]] ### Difference between 6.x and 7 ![[_media/vSphere - ESXi-2024-05-14-1.png]] ## CPU virtualization Generally physical CPUs or pCPUs on an ESXi host can be shared by multiple vCPUs of a VM. ESXi is scheduling time slots for overlapping vCPUs on a pCPU. The ratio between physical and virtual CPUs is called vCPU/pCPU ratio. Generally a ratio of 3-5 (meaning 3-5 vCPUs per pCPU) is common and with normal workloads is not impacting performance much. Under the surface ESXi is load balancing the vCPUs to the different pCPUs to improve performance, CPU affinity is disabling this. ### Scheduling Strict scheduling (depreceated): ![[_media/vSphere - ESXi2024-07-06.png|vSphere - ESXi2024-07-06.png]] relaxed Co-Scheduling: ![[_media/vSphere - ESXi2024-07-06-1.png|vSphere - ESXi2024-07-06-1.png]] With relaxed Co-Scheduling **CPU skew** can occur, which means that not all vCPUs can be scheduled on pCPUs. Therefore the unscheduled vCPUs are behind. If this occurs more often, problems can occur. **CPU skew** is Measured with **costop**. Understanding the way ESXi handles scheduling vCPUs. We can understand, that **rightsizing** VMs is very important for the performance of a VM and can even cause the VM to perform better with less vCPUs due to the decreased **Ready time** and **Costop**. ### Worlds thread of execution runninf for each virtual machines or processes inside an ESXi host. ### SMP Support for multiple CPUs for each VM ### CPU Affinity bonds a VM to one physical CPU. Disables vMotion and HA. Only used for licensing. ### ESXtop monitoring tool for analyzing ESXi utilization ### Hyperthreading Instructions/ load balancing can be more efficient on hyperthreading enabled CPUs. Instructions of VMs are sent to normal threads and hyperthreads at the same time. Hyperthreading does not make the CPU much more faster, but it decreases the idle time for instructions of the VMs ### Performance indicators #### CPU ready VM is ready to be scheduled but cannot be scheduled because pCPUs are busy. #### Co Stop VM vCPUs are split up because not all vCPUs were able to be scheduled at the same time. This happens when e.g. the Host has 2/4 busy pCPUs and a VM has 4vCPUs and is only able to be scheduled for 2 vCPUs. ## Memory virtualization Memory can be oversubscribed. ESXi is shareing memory across virtual machines. Memory is only thin provisioned, only used on the hypervisor if an application is actually requiring it. VM is never informing the ESXi host that it is no longer needing memory, reclaiming is only done on ESXi level. ### Reservation can be set on a VM level. guarantees that level of memory and will block it on the ESXi host. generally not desirable to have reservations. ### Balooning The amount of RAM that is being reclaimed to be used by other VMs because of memory sparsity. ### Swap-in ESXi host is out of memory and using disk space for Swap. ## Required Ports 🌐 ### vCenter - ESXi | Source | Destination | Direction | Protocol | Port | Purpose | | ------ | ----------- | ------------- | -------- | ---- | -------------------- | | ESXi | vCenter | incoming | UDP | 902 | vpxd | | ESXi | vCenter | bidirectional | TCP | 2014 | RPC | | ESXi | vCenter | bidirectional | UDP | 6500 | ESXi Dump | | ESXi | vCenter | bidirectional | TCP | 6501 | Auto deploy | | ESXi | vCenter | bidirectional | TCP | 6502 | Auto deploy | | ESXi | vCenter | bidirectional | TCP | 6502 | Auto deploy | | ESXi | vCenter | bidirectional | TCP | 7475 | Authentication Proxy | | ESXi | vCenter | bidirectional | TCP | 7476 | Authentication Proxy | | ESXi | vCenter | bidirectional | TCP | 9084 | lifecycle Management | | | | | | | | | | | | | | | ## Auto deploy ### Requirements - DHCP Server - TFTP Server - vCenter configured for auto deploy ### Workflow - DHCP Server is giving ESXi host network configuration - TFTP Server is giving the ESXi Server a PXE file for the initial boot - ESXi Host is requesting image from Auto deploy server on vCenter - vCenter is determining which image to pick based on MAC of Vendor etc. - Host profile on vCenter is applied on the ESXi after deployment - ESXi boots up ## Time Following Time synchronisations are supported: - manual - PTP - NTP NTP and PTP are represented by their services NTP Service and PTP daemon. PTP can be forwarded to a VM to allow for very precise time synchronisation. ## Troubleshooting ### Restarting Managament agents in the DCUI, go to Troubleshooting options - Restart Management Agents to restart all management agents alternetively you can do this via ssh: ```bash /etc/init.d/hostd restart /etc/init.d/vpxa restart ``` or on a specific vmk: ```bash esxcli network ip interface set -e false -i vmk0; esxcli network ip interface set -e true -i vmk0 ``` or all management agents: ```bash services.sh restart ``` ### Releasing DHCP lease of Managament IP in the DUI go to Management network - restart management network. This will release the current DHCP lease and renew it. ## 🔗Resources - HCL https://www.vmware.com/resources/compatibility/search.php - [ESXi System Storage Overview](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.esxi.upgrade.doc/GUID-474D003B-C6FB-465D-BC1B-5FD30F8E2209.html) ### Ports - https://ports.esp.vmware.com/home/vSphere ### CPU Performance - https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmware-vsphere-cpu-sched-performance-white-paper.pdf (old but gold) - https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-resource-management/GUID-98BD5A8A-260A-494F-BAAE-74781F5C4B87.html ### Esxtop Cheat Sheet - https://www.running-system.com/wp-content/uploads/2015/04/ESXTOP_vSphere6.pdf