Enable the VMware balloon driver vmmemctl to prevent swapping in an imported Linux VM

After running Alan Renouf’s excellent vCheck5 script against a vCenter Server, I noticed alot of VM’s where ESXi was actively swapping memory without any memory being ballooned

This, ofcourse, required further investigation.

I checked one of the ESXi hosts that was serving the VM’s using resxtop on our vMA and found that the balloon driver was not active:

I found the possible reasons for a non-working balloon driver on page 34 of the Resource Management guide (ESXI 4.1 U2 is in use):

ESX/ESXi hosts use swapping to forcibly reclaim memory from a virtual machine when the vmmemctl driver is not available or is not responsive.

– It was never installed.
– It is explicitly disabled.
– It is not running (for example, while the guest operating system is booting).
– It is temporarily unable to reclaim memory quickly enough to satisfy current system demands.
– It is functioning properly, but maximum balloon size is reached.

The balloon driver cannot be disabled on host level and the VM’s didn’t have their sched.mem.maxmemctl parameter set to 0. The VMware Tools were installed. This eliminated options 1 & 2. It was now time to look into the VM.

After cloning a test VM and disconnecting it’s virtual cable, it was time to boot. After logging in I ran the

vmware-toolbox-cmd –version

command to show the version. It was running the latest version 8.3.12 build 559003, so that seemed OK. Next was running the config script


and I was promptly faced with the root cause of the problem because I received the following warning message

The module vmmemctl has already been installed on this system by another installer or package and will not be modified by this installer. Use the flag –clobber-kernel-modules=vmmemctl to override.

It seemed that almost every VM with this problem was an imported Oracle VM virtual machine. During it’s migration to the vSphere platform, the VMware Tools were installed, although now it seemed the installation was not correct/complete.

There were no obstacles to enable the correct VMware Tools vmmemctl driver, so running the suggested command

vmware-config-tools –clobber-kernel-modules=vmmemctl

fixed the problem. After a reboot I checked if the balloon driver was working

All OK. Onto the next challenge. Did I mention I love my job? =]

EDIT: On some VM’s it seems the pvscsi, vmxnet3, vsock and vmci kernel modules were having the same problem. I choose to fix them too. The complete command would be

vmware-config-tools –clobber-kernel-modules=vmmemctl,vsock,vmxnet3,vmci,pvscsi


Creating an HP IRF (Intelligent Resilient Framework) Networking stack between 2 switches

A short checklist for creating a IRF stack on 2 HP switches. I executed this on 2 HP A3600 EI switches:

  1. Login onto the switch using the console port
  2. sys (Enter system view)
  3. show version (Ensure that both switches are running the same software version)
  4. reset saved-configuration (Reset the config)
  5. irf member 1 renumber 1 (Assign an IRF member number to the first switch)
  6. irf member 1 renumber 2 (Assign an IRF member number to the second switch)
  7. quit (Quit to user view)
  8. save (Save the config)
  9. reboot (Reboot the switches)
  10. irf mac-address persistent always (Enable MAC address persistance)
  11. irf member 1 priority 32 (Set the highest prio on the first member/switch)
  12. irf member 2 priority 30 (Set the second highest prio on the second member/switch)
  13. int GigabitEthernet 1/0/51
  14. shut
  15. int GigabitEthernet 1/0/52
  16. shut
  17. int GigabitEhternet 2/0/51
  18. shut
  19. int GigabitEthernet 2/0/52
  20. shut (shutdown all interfaces you want to use for IRF on both switches)
  21. irf port 1/1 (Create IRF port 1/1 on the first member)
  22. port group interface GigabitEthernet 1/0/51 (add the switch port to the IRF port)
  23. quit
  24. irf port 1/2 (Create IRF port 1/2 on the first member)
  25. port group interface GigabitEthernet 1/0/52 (add the switch port to the IRF port)
  26. quit
  27. irf port 2/1 (Create IRF port 2/1 on the second member)
  28. port group interface GigabitEthernet 2/0/51 (add the switch port to the IRF port)
  29. quit
  30. irf port 2/2 (Create IRF port 2/2 on the second member)
  31. port group interface GigabitEthernet 2/0/52 (add the switch port to the IRF port)
  32. quit
  33. save (Save config)
  34. interface GigabitEthernet 1/0/51
  35. undo shut
  36. interface GigabitEthernet 1/0/52
  37. undo shut
  38. interface GigabitEthernet 2/0/51
  39. undo shut
  40. interface GigabitEthernet 2/0/52
  41. undo shut (enable all interfaces you want to use for IRF on both switches)
  42. irf-port-configuration active (Activate the IRF config on BOTH switches)
  43. Now connect your fiber CROSSWISE, so 1/0/51 to 2/0/52 and 1/0/52 to 2/0/51
  44. ATTENTION: The second IRF member will reboot! Wait for it to get back up. You will see the switches negotiate for about 30 seconds before the IRF becomes active.
  45. If all works well;
  46. quit
  47. save
  48. reboot
  49. disp irf (Display the IRF setup)
  50. disp irf topology (Display the IRF Topology)
  51. Both irf port should be up. If one is DOWN or DIS(abled), something went wrong.
  52. Check the IRF prio of the second member. It should be 30.

Download the IRF Configuration Guide for more information on configuring IRF including optional parameters

I got my info mainly from here and adjusted it so it applies to a stack with just 2 switches.

%d bloggers like this: