Remember to balance your memory DIMMs: BIOS reports that NUMA node 1 has no memory

This week I was presented with the following PSOD boot error while trying to install ESXi 4.1 on a new host. It was a new HP Blade node with an added CPU but with the default amount of RAM. So it ended up with 3 banks filled with 2GB DIMMs on 1 NUMA node.

Luckily, we still had some DIMMs laying around, so it ended up not being a real problem. But next time I’ll check new hardware before trying to install =)


Install the HP/ServerEngines/Emulex Corporation NC553i 10Gb 2-port FlexFabric Converged Network Adapter be2net driver for your BL460c G7 blade based ESXi host using the vMA

While trying to apply a host profile to a new HP BL460c G7 blade based ESXi 4.1 host, I noticed I was missing 2 vmnics 🙂

It seemed the NC553i CNA had no driver installed. No problem, I downloaded the driver CD here. However, this 4.0.306.0 driver is newer then the one installed on the other ESXi hosts. It seems there are 3 other versions, , 2.102.554.0 and 2.102.518.0 available for download and at the moment our ESXi hosts run the oldest version.

We decided to install the newest driver and use this as a test case. We don’t expect any problems because this driver is 6 months old now and it has gone through rigorous testing before recieving the ‘VMware certified’ stamp.

Normally I would copy the to a local datastore and install it. This server, however doesn’t have persistent storage because it boots from a flash disk. So we’re gonna use the vMA to do it. It will be a usefull step towards using the vMA for everything =]

First, extract the downloaded .iso file and copy the to your vMA using SCP.

Login to the vMA and use the command

vihostupdate –server <server name> –username root –password <root password> –install –bundle

to install the driver.

When the installation is finished, reboot the host.

I used the vSphere client for convenience.

After the host is up again, you can see if your hardware is working by checking the network adapters page

Success! By logging into the ESXi host and running the command

esxcfg-nics -l

you can see all your network adapters. Running the command

ethtool -i vmnic<number>

you can see what driver is being used. You can clearly see the difference in driver version and even firmware version in the screenshots below.

Old driver and firmware:

New driver and firmware:

We’ll continue to test these newer versions until we’re statisfied we can update all hosts. Before we do this, however, both versions and the combination of the two, should be available in the VMware HCL. ATM they are not.

Please note this and this if you’re using VMware ESXi 5.0!

A quick note on WSUS error 0x80072EFD (Send failed with hr = 80072efd)

After installing a new WSUS server, my test machine failed to make contact. I had created a copy of the current WSUS GPO, edited it to the new server and applied it to only my test Windows 7 machine. I received the following error in the C:\Windows\WindowsUpdate.log

2012-03-05 14:34:12:187 1024 2748 Agent *************
2012-03-05 14:34:12:187 1024 2748 Agent ** START ** Agent: Finding updates [CallerId = AutomaticUpdates]
2012-03-05 14:34:12:187 1024 2748 Agent *********
2012-03-05 14:34:12:187 1024 2748 Agent * Online = Yes; Ignore download priority = No
2012-03-05 14:34:12:187 1024 2748 Agent * Criteria = “IsInstalled=0 and DeploymentAction=’Installation’ or IsPresent=1 and DeploymentAction=’Uninstallation’ or IsInstalled=1 and DeploymentAction=’Installation’ and RebootRequired=1 or IsInstalled=0 and DeploymentAction=’Uninstallation’ and RebootRequired=1”
2012-03-05 14:34:12:187 1024 2748 Agent * ServiceID = {XXXXXXX-E39D-4DA6-8A4B-XXXXXXXXXX} Managed
2012-03-05 14:34:12:187 1024 2748 Agent * Search Scope = {Machine}
2012-03-05 14:34:12:236 1024 2748 Setup Checking for agent SelfUpdate
2012-03-05 14:34:12:269 1024 2748 Setup Client version: Core: 7.5.7601.17514 Aux: 7.5.7601.17514
2012-03-05 14:34:12:279 1024 2748 Misc Validating signature for C:\Windows\SoftwareDistribution\SelfUpdate\
2012-03-05 14:34:12:353 1024 2748 Misc Microsoft signed: Yes
2012-03-05 14:34:15:667 1024 2748 Misc WARNING: Send failed with hr = 80072efd.


2012-03-05 14:34:25:587 1024 1a78 AU # WARNING: Search callback failed, result = 0x80072EFD
2012-03-05 14:34:25:587 1024 1a78 AU # WARNING: Failed to find updates with error code 80072EFD
2012-03-05 14:34:25:587 1024 1a78 AU #########
2012-03-05 14:34:25:587 1024 1a78 AU ## END ## AU: Search for updates [CallId = {XXXXXXX-9BE7-4DBC-8EC8-XXXXXXXXXX}]
2012-03-05 14:34:25:587 1024 1a78 AU #############

After some Googling it was clear it had something to do with the network. But after all usual network troubleshooting I wasn’t any closer to a solution. I decided to check the IIS server and finally got a bright moment. I installed the WSUS server on a clean Windows Server 2008 R2 SP1 installation. But I the old server has many services installed. Thus the new WSUS website was installed in IIS on port 80 while the old server was using port 8530. And because I copied the GPO and only adjusted the FQDN, it was still pointing to port 8530!

Instead of adjusting the GPO, I decided to bind the WUS website to port 8530 too. This way, when migrating to the new WSUS server, only the FQDN had to be changed.

I now got this documented, so I hope I will never waste another minute on this.

%d bloggers like this: