VMware Auto Deploy error SSLError: [Errno 336265225] _ssl.c:337: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib

After upgrading to vCenter 5.1 Update 1a I decided to take a look at the Auto Deploy component. Since upgrading to vCenter 5.0 it wasn’t working. Because we didn’t plan on using it, I just disabled the service at that time. After the improvements to Auto Deploy in 5.1 especially the Stateless Caching mode, this technique  might be something we want to use in the future.

Enabling the service resulted in the same error as before. The service starts, but the vSphere Client plugin cannot connect and displays the error ‘The request failed because of a connection failure. (Unable to connect to the remote server)’


I verified that there were no listeners on port 6501 and 6502 through a netstat -a on the Autodeploy server.


I browsed to VMware KB 2000988 to start my quest for a solution.

I opened the vmconfig-autodeploy.xml from %ProgramData%\VMware\VMware vSphere Auto Deploy\ and verified all paths mentioned were correct. I also checked the settings in HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\VMware, Inc.\VMware vSphere Auto Deploy with regedit. The only thing I changed in the registry and the XML file was the username. I changed it from the Down-Level Logon Name [NETBIOS DOMAIN]\[USER] to the User Principal Name [USER]@[DNS DOMAIN].

A reboot of the service didn’t change the situation.

Because I didn’t do any configuration, I tried to re-register the service and got an error thrown at me, yay!


I opened the log file location %ProgramData%\VMware\VMware vSphere Auto Deploy\logs and immediately saw that a lot of logging was being created. A good thing in it self and a clear indication something is wrong 🙂


Opening one of the log files revealed that same error

SSLError: [Errno 336265225] _ssl.c:337: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib

and another one

Error: [(‘PEM routines’, ‘PEM_read_bio’, ‘no start line’), (‘SSL routines’, ‘SSL_CTX_use_certificate_file’, ‘PEM lib’)]


So, it seems to be a certificate issue. A Google search revealed nothing to further help me.

I opened the location of the SSL certificates %ProgramData%\VMware\VMware vSphere Auto Deploy\ssl and there was the root of the problem.


Because we still use the default installer certificates, these were quickly replaced with copies from e.g. the %ProgramData\VMware\VMware ESXi Dump Collector\ssl folder.

I tried to re-register the service again, but was thrown another error:

error: attempt to write a readonly database


Again, because there was no configuration done, I simple renamed the db file in the Data directory and ran the command again. A new db file was created automatically.


The command did throw another error or warning,

‘openssl’ is not recognized as an internal or external command, operable program or batch file.

but it did finish successfully.


I restarted the Auto Deploy service and was able to connect my vSphere Client. All good!



Installing patches or updates on your Exchange 2010 DAG cluster

To install patches or update on your Exchange 2010 DAG cluster, follow the steps below.

  • OPTIONAL: Log on to one of the servers of your DAG cluster
  • Open an Exchange Mangement Console instance
  • cd to the directory with the maintenance scripts:
  • cd $exscripts
  • Execute the ‘start maintenance’ script:
  • .\StartDagServerMaintenance.ps1 -serverName [NETBIOS name of server] (DONT USE THE FQDN or some parts of the script wont work)
  • The PowerShell script will activate the mailbox databases on other servers of the DAG cluster, it will pause the server in Failover Clustering and will set some Exchange parameters to prevent failover to this server. Outlook clients will receive a disconnect message immediately followed by a connect message.
  • You can now install patches or updates and reboot at will. When the server reboots, Outlook clients (connected to this server) will receive a disconnect message immediately followed by a connect message.
  • When you’re done, execute the ‘stop maintenance’ script:
  • .\StopDagServerMaintenance.ps1 -serverName [NETBIOS name of server] (DONT USE THE FQDN )
  • This script executes all maintenance action in reverse order.
  • Check the Failover Cluster Manager for errors and verify the cluster is up and running. In the screenshots below there is an error because the File Share Witness had a scheduled reboot at 06:00AM. No errors occurred because of the maintenance that was performed.
  • exchange-dag-maintenance-01
  • exchange-dag-maintenance-02
  • exchange-dag-maintenance-03
  • You can now repeat above steps on all other members of the DAG cluster.
  • Because I’ve only got 2 cluster members, at the end of the maintenance actions, all mailbox databases will be active on one server. To show this, execute the command:
  • Get-MailboxDatabase | ft name, server, activationpreference -AutoSize
  • exchange-dag-maintenance-04
  • To re-balance the mailbox databases over all members of the cluster according to your activation preference, execute the command:
  • .\RedistributeActiveDatabases.ps1 -DagName DAG01 -BalanceDbsByActivationPreference (Somehow, this command didn’t work on my admin PC, only on the Exchange servers themselves)
  • You will be asked if you really want to set the active database on one of the other servers. The script will give a summary of all actions taken when it’s finished. Outlook clients will receive a disconnect message immediately followed by a connect message.
  • exchange-dag-maintenance-05
  • Check if the DAG cluster is working as it should by executing:
  • Get-MailboxServer
  • exchange-dag-maintenance-06
  • All values should normally be as seen in the above screenshot unless configured otherwise.
%d bloggers like this: