201
2

Making room for NSX… Deleting ESXi VIB’s

VIB’s are kernel modules which make up the various capabilities of the VMware vSphere ESXi hypervisor together with hardware device drivers. When building vSphere ESXi Servers, hardware vendors can assist by providing ISO images which include hardware monitoring VIB’s together with common storage and network driver VIB’s for their servers.

There is a limit to the size of the vSphere ESXi boot image location (bootbank) of around 250MB and unfortunately, some hardware vendor ISO images are filled with so many device drivers, there is no room for additional drivers. This can cause NSX host preparations to fail…

Auto Deploy

In larger organizations with mixed vendor or mixed generation servers, server ESXi image management can be simplified using vSphere’s Auto Deploy solution which provides for ESXi image and configuration management. See https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2005131 for more information on Auto Deploy.

Vendor ISO’s

For many years now, various compute hardware vendors will distribute an ISO specific to their hardware. This ISO image includes the base ESXi image together with additional VIB’s for specific vendor hardware monitoring together with common storage and network hardware VIB’s packaged by the vendor with their servers.

In an attempt to minimize the work involved to deploy vSphere ESXi, smaller organizations may use the hardware vendor’s ISO image without customization. Worst case, this image will include so many VIB’s there is no room to install additional ESXi software features including NSX.

Lenovo seems to be an offender here as I have seen numerous issues with the Lenovo ESXi ISO filled to capacity which prohibits additional kernel additions and updates.

Removing VIB’s

Removing VIB’s is a manual process and there are two approaches –

  1. Create an ESXi ISO Boot Image that only contains the needed VIB’s. Start with a base ESXi image and add vendor hardware monitoring VIB’s, and VIB’s to match server storage and network devices. Use this image for new Hypervisor deployments.
  2. Delete unused VIB’s from an existing deployed and running Hypervisor.

For the second option, there is no easy method to identify what VIB’s are being used, but it’s easy to make some general assumptions on what to delete.

In the attached example on a Dell R730, the following VIB’s are being used:


[root@inf-v-150:~] esxcli software vib get | egrep Summary:
Summary: Ethernet driver for Broadcom NetXtreme-C/E Ethernet Controller
Summary: This VIB contains Dell Support Information
Summary: beima: IMA plugin for VMware ESX
Summary: be2iscsi: scsi driver for VMware ESX
Summary: i40e: net driver for VMware ESX
Summary: igb: net driver for VMware ESX
Summary: ixgbe: net driver for VMware ESX
Summary: QLogic IMA plugin for VMware ESX, ESXi
Summary: cnic_register: misc driver for VMware ESX
Summary: bnx2: net driver for VMware ESX
Summary: bnx2x: net driver for VMware ESX
Summary: cnic: net driver for VMware ESX
Summary: qlcnic: net driver for VMware ESX
Summary: Qlogic Native FC driver
Summary: bnx2fc: scsi driver for VMware ESX
Summary: bnx2i: scsi driver for VMware ESX
Summary: qla4xxx: scsi driver for VMware ESX
Summary: libata_92: ata driver for VMware ESX
Summary: pata_amd: ata driver for VMware ESX
Summary: pata_atiixp: ata driver for VMware ESX
Summary: pata_cmd64x: ata driver for VMware ESX
Summary: pata_hpt3x2n: ata driver for VMware ESX
Summary: pata_pdc2027x: ata driver for VMware ESX
Summary: pata_serverworks: ata driver for VMware ESX
Summary: pata_sil680: ata driver for VMware ESX
Summary: pata_via: ata driver for VMware ESX
Summary: cciss: block driver for VMware ESX
Summary: random: char driver for VMware ESX
Summary: Updates the ESX 6.5.0 ehci-ehci-hcd
Summary: net driver for VMware ESX
Summary: hid: hid driver for VMware ESX
Summary: Network driver for Intel(R) Gigabit Server Adapters
Summary: ipmi_devintf: ipmi driver for VMware ESX
Summary: ipmi_msghandler: ipmi driver for VMware ESX
Summary: ipmi_si_drv: ipmi driver for VMware ESX
Summary: Updates the ESX 6.5.0 ixgben
Summary: Emulex FC Driver
Summary: Avago Native MegaRAID SAS
Summary: Avago (LSI) Native 6Gbps SAS MPT Driver
Summary: Avago (LSI) Native 12Gbps SAS MPT Driver
Summary: Updates the ESX 6.5.0 misc-drivers
Summary: P32x/P42x PCIe SSD
Summary: Updates the ESX 6.5.0 ne1000
Summary: Cisco VIC Native driver for VMware ESX
Summary: cdc_ether: net driver for VMware ESX
Summary: e1000: net driver for VMware ESX
Summary: e1000e: net driver for VMware ESX
Summary: enic: net driver for VMware ESX
Summary: fcoe: net driver for VMware ESX
Summary: forcedeth: net driver for VMware ESX
Summary: libfcoe_92: net driver for VMware ESX
Summary: mlx4_core: net driver for VMware ESX
Summary: mlx4_en: net driver for VMware ESX
Summary: nx_nic: net driver for VMware ESX
Summary: tg3: net driver for VMware ESX
Summary: usbnet: net driver for VMware ESX
Summary: vmxnet3: net driver for VMware ESX
Summary: HPSA native driver
Summary: Mellanox Technologies ConnectX Native Core Driver
Summary: Mellanox Technologies ConnectX Native Ethernet Driver
Summary: Mellanox Technologies ConnectX Native RDMA Driver
Summary: Mellanox Technologies ConnectX4 Native Core Driver
Summary: Broadcom NetXtreme BCM5718 Family NIC Driver for ESXi
Summary: Non-Volatile memory controller driver
Summary: Network driver for VMware vmxnet3 Virtual Ethernet Controller
Summary: usb-ohci: ohci driver for VMware ESX
Summary: PVSCSI Driver
Summary: Network driver for QLogic FastLinQ QL45604 Ethernet Controller
Summary: Network driver for QLogic E3 Controller
Summary: Network driver for QLogic NetXtreme II PCI/PCIe Gigabit Ethernet Adapters
Summary: ahci: sata driver for VMware ESX
Summary: ata_piix: sata driver for VMware ESX
Summary: sata_nv: sata driver for VMware ES
Summary: sata_promise: sata driver for VMware ESX
Summary: sata_sil24: sata driver for VMware ESX
Summary: sata_sil: sata driver for VMware ESX
Summary: sata_svw: sata driver for VMware ESX
Summary: aacraid: scsi driver for VMware ESX
Summary: adp94xx: scsi driver for VMware ESX
Summary: aic79xx: scsi driver for VMware ESX
Summary: fnic: scsi driver for VMware ESX
Summary: hpsa: scsi driver for VMware ESX
Summary: ips: scsi driver for VMware ESX
Summary: iscsi_linux_92: scsi driver for VMware ESX
Summary: libfc_92: scsi driver for VMware ESX
Summary: megaraid_mbox: scsi driver for VMware ESX
Summary: megaraid_sas: scsi driver for VMware ESX
Summary: megaraid2: scsi driver for VMware ESX
Summary: mpt2sas: scsi driver for VMware ESX
Summary: mptsas: scsi driver for VMware ESX
Summary: mptspi: scsi driver for VMware ESX
Summary: iscsi_linux_9_2_1_0: shim driver for VMware ESX
Summary: iscsi_linux_9_2_2_0: shim driver for VMware ESX
Summary: libata_9_2_1_0: shim driver for VMware ESX
Summary: libata_9_2_2_0: shim driver for VMware ESX
Summary: libfc_9_2_1_0: shim driver for VMware ESX
Summary: libfc_9_2_2_0: shim driver for VMware ESX
Summary: libfcoe_9_2_1_0: shim driver for VMware ESX
Summary: libfcoe_9_2_2_0: shim driver for VMware ESX
Summary: vmklinux_9_2_1_0: shim driver for VMware ESX
Summary: vmklinux_9_2_2_0: shim driver for VMware ESX
Summary: vmklinux_9_2_3_0: shim driver for VMware ESX
Summary: usb-uhci: uhci driver for VMware ESX
Summary: usb-storage: usb-storage driver for VMware ESX
Summary: usb: usbcore driver for VMware ESX
Summary: ATA Driver
Summary: vmkplexer: vmkplexer driver for VMware ESX
Summary: Updates the ESX 6.5.0 vmkusb
Summary: Updates the ESX 6.5.0 vmw-ahci
Summary: xhci: xhci driver for VMware ESX
Summary: CPU microcode updates
Summary: esxcli plugin for Emulex elxnet driver
Summary: Updates the ESX 6.5.0 esx-base
Summary: dvfilter-generic-fastpath module
Summary: ESXi Trusted Boot
Summary: VMware Host Client
Summary: ESXi X.Org Xserver
Summary: HPSA LSU Management Plugin
Summary: LSI MR3 LSU Management Plugin
Summary: LSI MSGPT3 LSU Management Plugin
Summary: LSI MegaRAID SAS LSU Management Plugin
Summary: LSI MPT2SAS LSU Management Plugin
Summary: Miscellaneous drivers for ESX
Summary: rste: SCU SAS/SATA for VMware ESX
Summary: esxcli plugin for VMware nvme driver
Summary: VMware Fault Domain Manager
Summary: ESXi firewall rules for VR2C traffic
Summary: Updates the ESX 6.5.0 vsan
Summary: ESXi VSAN Health Service
Summary: ESXi tools

As a vSphere administrator, you will have a pretty good idea of what hardware has been installed in the server and as such, you can target driver vendors that are not being used. Based on the above list, knowing the server is using Intel NIC’s, I would focus on deleting QLogic and Melanox drivers.

Deleting VIB’s is covered in numerous articles and the steps involve are:

  • Place the hypervisor into maintenance mode.
  • Enable ssh on the hypervsor
  • Perform a  vib List or Summary
  • esxcli software vib list
  • Delete unwanted VIB’s
  • esxcli software vib remove --vibname=OpenManage
  • Reboot the hypervisor

In my opinion, Option 1 of building an ISO image using the base ESXi image as a foundation is the better approach for deploy ESXi servers. Minimizing the Hypervisor footprint by removing unused kernel modules only can improve overall Hypervisor security. Vendor ISO’s are convenient when they help simplify ESXi deployments, but when they are crammed to capacity with unused VIB’s, they become a liability.

Related Posts
UCS and Mixed vPC / Non vPC VLANs
NSX Multi Hypervisor and NTP
NSX-vSphere 6.2 VIB’s and VXLAN Troubleshooting