Vmware Vcenter Server Service Stuck On Starting

[Danny Hosford]

Wecould see something was wrong due to the vCenter Server showing down in the VMware Horizon Admin Console. When checking on the vCenter service itself, it looked like the VMware VirtualCenter Server service was not started. When trying to manually start the service, it was getting stuck on starting the service for several minutes and failed. You may also see event errors with ID: 1000.

It turns out, a Windows Update enabled the built in Windows BranchCache feature. By going to the Non-Plug and Play Drivers in Device Manager and going into HTTP as seen above, we could see what was trying to use the HTTP protocol. (I was a little surprised as our environment uses HTTPS but it looks like vCenter still relies on port 80 when starting). Checking in here showed that BranchCache was butting in and taking over port 80 which was causing vCenter to fail to start. (Thank you VMware support for showing me!)

The first thing I checked was that the time was set correctly. If the time is incorrect, it can cause several issues. Since we were able to log into vCenter Server Appliance Management Interface (VAMI), I was able to check that the NTP source was set up and that there were no issues. This can also be done in vCenter BASH Shell via SSH by using the commands ntp.get to check the configured NTP, and the command date.I also checked if the STS certificate had expired

I then checked if there was enough disk space with df -h.After confirming that the services were still getting stuck on 85% after a reboot of the vCenter, I checked the certificate manager log, which you can find here: /var/log/vmware/vmcad/certificate-manager.log.In the log, I could see that there were inconsistencies in how the Fully Qualified Domain Name (FQDN) of the vCenter was written, and there were errors pointing to problems with the Subject Alternate Name (SAN).

After all of this, I was confident that I had found the error, but it was still stuck on 85% after resetting the certificates. I then tried resetting the Security Token Service (STS) certificate and replacing all the certificates through certificate-manager, but no luck.

In conclusion, there was a lot to learn from this issue. Firstly, what might seem like red herrings, may very well be underlying problems that also needs to be solved. In the end, the final issue was fixed by VMware Support, but without all the troubleshooting steps performed before they were brought on board, the fix might not have been so straight forward.

When deploying VMWare VCSA in a lab environment the installer often gets stuck at stage 2 starting services. In my case, starting authentication network 2%. This seems to be an issue with VCSA wanting to verify the SSO domain through DNS resolution.

The solution for me was to create an entry in the hosts file of the VCSA appliance through SSH to the CLI. Before doing so, it is best to start from scratch. Delete the partially deployed VCSA appliance. Deploy VCSA through Stage 1 but do not enter Stage 2.

Once Stage 1 is complete and the VM is booted, SSH can be enabled by accessing the console of the VCSA machine, hitting F2 and enabling SSH. Alternatively, one can use the Alt+F1 method to access the CLI directly through the console.

In this article I will cover steps and actions we took in order to ensure a seamless migration of our current SQL Database server to a new one. This will include a lot of recommendations and what issues we faced both in testing and real environment.

Recommendation: have an DNS alias name created of the current DB Server and start this procedure. When the new SQL DB is ready, just modify the alias name in DNS to point to the new SQL Server. This will give some leverage, in case something goes wrong with the new one, to revert back in short amount of time. (just a matter or changing the alias IP back to old server and restarting services)

-Master password for the SSO (admin@System-Domain). This is the initial password and even if you change it in the mean time you need the primary password to continue. If you do not have it or forgot it, than the only option is to re-install the vCenter Server completely.

VMware Fusion and Workstation continues to be another popular way for customers to run a VMware Homelab while leveraging a users existing desktop. In the early days of vSphere 6.5, the method to deploy the vCenter Server Appliance (VCSA) to Fusion/Workstation was less than ideal with a lot of manual steps. In 2017, the Fusion/Workstation team introduced native OVF support and that made deploying the VCSA much simpler, especially with the VCSA two stage installer.

Even though this is not an officially supported method from VMware for deploying the VCSA, the process has not changed for the last several releases and it just works which was great for our users. With vSphere 8, it looks like there has been a change to the VCSA installer that causes a failure during the Stage 2 configuration.

Using one of my previous blog post for guidance, he discovered a quick workaround to the problem by simply ensuring this variable is configured with a default value. After running into the problem myself and verifying the solution, I figure this might be useful for anyone looking to run vSphere 8 using VMware Fusion or Workstation, so here are the instructions to work around this issue.

Step 1 - Extract the contents of the VCSA 8.0 ISO to your local desktop. If you are on a macOS system, you may also need to remove security quarantine flag from the extracted directory and ensure the directory has write access (e.g. chmod 755).

Note: If you do not have OVFTool installed, there is a local version bundled in the ovftool directory and you run that by specifying the local path to the OVFTool utility.

In my setup, I am using VMware Fusion and once the import of the VCSA OVF has completed, you will only need to fill out the networking and the password section for both SSO and the OS as shown in the screenshot below. If are going to use DHCP, then you only need to fill out two properties along with the passwords and then click continue.

The VCSA will now begin its bootstrap process for Stage 1 and this can take up to several minutes depending on your available resources including transient failed messages including VM reboot. Be patient and wait until the VCSA displays the DCUI gray/blue screen with IP Address that you had specified earlier.

Hello to ALL:

We installed VCSA 6.7 as a VM on Vmware Workstation using the method VMware recommends. It installed without any issues and worked perfectly.

We then installed VCSA 7.0 the same way without issues. But when we tried to install VCSA 7 U3 it would fail during stage 2 of the install process. So we installed VCSA 7 U3 on a virtual ESXi 7 Server (nested). The install finished without any issues. Then we opened Vmware Workstation and conected to the ESXi 7 Hypervisor. We then saw the virtual machines that were installed using the ESXi 7 Hypervisor and we selected the VCSA 7 U3 Virtual Machine and there is a download option so we selected the download option and downloaded all the Virtual Machine files to a directory on our Windows 11 PC. Then we opened the .VMX file with Vmware Workstation and started this VCSA 7 U3 virtual machine and all worked perfectly.

Yes this is somewhat time consuming but since we could not install VCSA 7 U3 on Vmware Workstation without error using the Vmware's suggested method, we did this workaround and now we have VCSA 7 U3 installed as a virtual machine on our Vmware Workstation.

I follow the guide deploy vcsa8 in fusion12, but the vm network was always DHCP, and root password was not set. the ovf configurations dose not passed into vm. Are there some additional configuration

The install went properly for me in VMware workstation. The issue I have is when I tried to launch the vsphere 8 client, it states that it cannot reach page. I've checked everything: DNS records, disabled firewall and antivirus, cleared browser cache, etc.... but nothing works. What is weird is that the vsphere 7 client runs fine.

Thank you for the write-up! I am experiencing the same error as you described above, but while trying to install on ESXi instead of Workstation/Fusion. Could you advise me on how to deploy the modified OVA through the VCSA installer on ESXi? I attempted to do so after making the modifications mentioned above, but I received an error message stating that the installer was unable to validate the OVA file. I attempted this yesterday, but forgot to note down the exact error message. I am currently trying to reproduce the error.

If the above method does not work, is it possible to log in to the appliance via SSH after the first stage has completed but before the second stage has started? This way, I could make modifications then instead of modifying the OVA. I have been troubleshooting this issue for about a week now and I am unsure how to move forward. Does anyone know why this error is thrown in the first place?

If you have ESXi, WHY are you using this method of deployment? The VCSA provides native UI/CLI installer that can deploy to ESXi or vCenter Server endpoint which will give you best experience. The workaround mentioned here is ONLY applicable if you're not using ESXi/vCenter and want deploy to Fusion/Workstation which doesn't have support for VCSA Installer. Please use the VCSA Installer

Thanks for the quick response. To clarify, I am receiving the same error you mentioned above while attempting to install via the VCSA installer on my ESXi host during the second stage. That is why I was attempting to modify the OVA as per your suggestion.

I was able to replicate the issue again and here are a couple of screenshots of what I am referring to: It looks like both Stephen Yu and myself have experienced the same issue while trying to deploy the VCSA to an ESXi host via the GUI installer.

3a8082e126