Category Archives: vSphere 5

Adding a Host to vCenter Fails Because of Time Issues

I recently experienced an issue adding a vSphere 5.1 host to vCenter while using the Add-VMHost cmdlet in PowerCLI. I’m pretty sure the same problem would have occured if I was using the GUI, but this work was for part of some automated deployment work.

On a freshly baked ESXi 5.1 install one of the first tasks is to get it into vCenter. However, this was failing with what initially appeared to be a license issue, despite there being plenty of available licenses.

“The Evaluation Mode license assigned to Host xxxx.xxxx.xxxx has expired. Recommend updating the license.”

Add-VMHost01

 

Add-VMHost02

 

Turns out the issue was related to the date and time being incorrect on the host (it had been powered off for some time) and consequently the eval license had ‘expired’ even though it has just been installed.

 http://kb.vmware.com/kb/2011655

Despite the fact that my automated deployment configures NTP settings and starts the NTP service before adding the host to vCenter, the host had not yet corrected to the current date and time – possibly because it was so far out, in this case more than 1 year.

So to ensure the date and time are correct before adding the host to vCenter, I created the Set-VMHostToCurrentDateandTime function below. This uses the UpdateDateTime method to set the time on the ESXi host to the current time. I recommend you configure NTP settings and start the NTP service, then carry out the time set. You can use it in the following manner:

Get-VMHost ESXi01| Set-VMHostToCurrentDateandTime

 

function Set-VMHostToCurrentDateandTime {
<#
 .SYNOPSIS
 Function to set the Date and Time of a VMHost to current.

 .DESCRIPTION
 Function to set the Date and Time of a VMHost to current.

 .PARAMETER VMHost
 VMHost to configure Date and Time settings for.

.INPUTS
 String.
 System.Management.Automation.PSObject.

.OUTPUTS
 None.

.EXAMPLE
 PS> Set-VMHostToCurrentDateandTime -VMHost ESXi01

 .EXAMPLE
 PS> Get-VMHost ESXi01,ESXi02 | Set-VMHostToCurrentDateandTime

#>
[CmdletBinding()]

Param
 (

[parameter(Mandatory=$true,ValueFromPipeline=$true)]
 [ValidateNotNullOrEmpty()]
 [PSObject[]]$VMHost
 )

begin {

}

 process {

 foreach ($ESXiHost in $VMHost){

try {

if ($ESXiHost.GetType().Name -eq "string"){

 try {
 $ESXiHost = Get-VMHost $ESXiHost -ErrorAction Stop
 }
 catch [Exception]{
 Write-Warning "VMHost $ESXiHost does not exist"
 }
 }

 elseif ($ESXiHost -isnot [VMware.VimAutomation.ViCore.Impl.V1.Inventory.VMHostImpl]){
 Write-Warning "You did not pass a string or a VMHost object"
 Return
 }

# --- Set the Date and Time to the current Date and Time
 Write-Verbose "Setting the Date and Time to the current Date and Time for $ESXiHost"
 $Time = Get-Date
 $DateTimeSystem = $ESXiHost | ForEach-Object { Get-View $_.ExtensionData.ConfigManager.DateTimeSystem }
 $DateTimeSystem.UpdateDateTime((Get-Date($Time.ToUniversalTime()) -Format u))
 Write-Verbose "Successfully set the Date and Time to the current Date and Time for $ESXiHost"
 }
 catch [Exception]{

 throw "Unable to set current Date and Time"
 }
 }
 }
 end {

 }
}

Thanks to this post for refreshing my memory on how to do this

Migrating Email Alarm Actions between vCenter 5.0 and 5.1

I needed to migrate some Email Alarm Actions between two vCenters; the target at version 5.1 being a replacement for an existing 5.0 vCenter. The first task was to identify which Alarm Definitions had been configured with an email alert. To do that I used the following PowerCLI command to export them to a CSV file:

Get-AlarmDefinition | Select Name,@{N="EmailAction";E={$_ | Get-AlarmAction | Where {$_.ActionType -eq "SendEmail"}}} | Export-Csv AlarmActions.csv -NoTypeInformation

I could then easily identify those which needed to be migrated across.

Alarm2

Before creating new Email Alarm Actions in the 5.1 vCenter I wanted to check that the alarms which had been configured with an Email Actions still existed in 5.1, since it’s reasonably likely to assume that there will be some changes in Alarm Definitions that exist between vCenter versions. After modifying my CSV to cut out those I didn’t need, I imported the data into PowerShell and ran a query to establish if any no longer matched:

$data = Import-CSV AlarmActions.csv
Get-AlarmDefinition ($data | Select -ExpandProperty Name)

One didn’t exist anymore; one appeared similar, but significantly renamed; a couple of others appeared to have a minor name change.

Alarm1

The two minor name changes to me look unnecessary, inconsistent and frankly a bit sloppy. To the naked eye you might not even notice them, and you might even think what’s the big deal, but adding a full stop character at the end of an Alarm Definition when it was not there in the previous version and the majority of other definitions don’t have them is pretty poor.

vCenter 5.0 (no full stop)

Alarm3

vCenter 5.1 (full stop)

Alarm4

Anyway, I updated my data source to match the new names and created the new Email Actions on the Alarm Definitions that required them:

$data = Import-CSV AlarmActions.csv
Get-AlarmDefinition ($data | Select -ExpandProperty Name) | New-AlarmAction -Email -To "[email protected]"

Removing ESXi 5.0 host from vCenter – General system error: Invalid Fault

While I was removing some ESXi 5.0 hosts from a 5.1 vCenter I encountered the following issue with a couple of them:

General system error: Invalid Fault

Lockdown

In the vCenter vpxd log file this message was accompanied by

msg = “vim.fault.AdminNotDisabled”

It turned out to be a known fault where Lockdown Mode was out of sync between vCenter and the ESXi host. In vCenter it showed as enabled:

Lockdown2

However, in the ESXi host it was showing as disabled. You can identify this by using the following command in the console:

vim-cmd -U dcui vimsvc/auth/lockdown_is_enabled

The KB article states to enable lockdown via the console

vim-cmd -U dcui vimsvc/auth/lockdown_mode_enter

Lockdown3

 

and once successful disable via the GUI.

Once complete, you’ll then be able to remove the host from vCenter.

Windows Server 2012 on vSphere 5.0 – The Case of the Missing D:\ Drive

I sincerely hope this saves somebody else some time because I had a fair amount of head scratching with this today. Installed a Windows Server 2012 VM on vSphere 5.0 U2, pretty standard install with a C:\ drive for OS, Software etc and D:\ for data. This particular server needed to run SQL Server 2012, however the install kept repeatedly failing with the error “Could not find the Database Engine startup handle”. Initially I suspected that it might be a problem with a corrupt ISO as there are numerous postings around that and some, such as the below, suggesting that it could be an issue with mounting an ISO (I was pretty dubious about this though).

http://mbmccormick.com/2012/12/sql-server-2012-could-not-find-the-database-engine-startup-handle/

Having messed around with extracting the ISO, copying the files locally and other methods, eventually I tracked down in the SQL Server install log that it seemed to be having an issue reading and writing files to and from the D:\ drive during setup. During the SQL Server install wizard I’d specified that the path for databases, logs etc would be D:\. So I re-ran the installation and this time left the default of C:\ for everything and voilà, it worked!

No big deal I thought, I’ll simply change these locations post install. However, when I went to change the database server properties (or attempt to create a database) I was greeted with a dialogue box which only displayed the C:\ drive as accessible, no sign of D:\.

HotPlug4

Although clearly Windows was happy that a D:\ drive was present.

HotPlug2

This started ringing other bells because during the time I had been spending copying extracted ISO files around the $Admin share of D:\ had not been accessible (which I was going to look into later – should have done it then!)

Googling Researching this issue landed me at this forum post on SqlServerCentral.com – Sql Server 2012 can’t see 2nd drive. In there they link to this VMware KB article – Disabling the HotAdd/HotPlug capability in ESXi 5.x and ESXi/ESX 4.x virtual machines . Essentially what is happening is that Windows is marking the drive as Removable (i.e. like a USB drive) and consequently some operations that we would normally expect to be available are not. If you look in the taskbar the drives are potentially removable – although in practice if you try to they are not because they are in use.

HotPlug1

You can get round this either by applying the suggested workaround from VMware which is to add devices.hotplug = false into the .vmx file for the VM or in Windows  disable a security policy in gpedit.msc under Computer Configuration\Windows Settings\Security Settings\Advanced Audit Policy Configuration\System Audit Policies\Object Access .

I went for the former which you achieve by:

Connect to the ESXi/ESX host or vCenter Server using the vSphere Client.
Power off the virtual machine.
Right-click the virtual machine and click Edit Settings.
Click the Options tab.
Click General > Configuration Parameters > Add Row.
Insert a new row with the name devices.hotplug and a value of false.
Power on the virtual machine.

HotPlug3

Once powered on Removable Devices are no longer present:

HotPlug5

The D:\ drive is available over the network via the $Admin share and SQL is now able to see the D:\ drive:

HotPlug6

Incidentally, I blogged about this similar issue back in 2010 and at the time put the devices.hotplug setting into standard templates for Windows Servers; looks like I’ll need to do that for Windows Server 2012 too! Never seem to hear much about other people having issues with this problem though, would like to hear in the comments if it has caused you issues too…..

VCP5 Study Resources

Having passed the VCP5 exam last week, I thought I would write the obligatory study resources blog post. So below are the resources I found most useful in preparing for the exam:

1) TrainSignal VMware vSphere 5 Training

I’ve previously used the TrainSignal VMware vSphere 4 VCAP Training Package to begin preparations for the VCAP-DCA exam so was well accustomed with the style of the TrainSignal videos. Since I was in the position of needing to upgrade my VCP from 4 to 5 before February 29th 2012 to avoid needing to fulfill a class requirement again, I found this to be a great resource to quickly get up to speed on some of the new features in vSphere 5 and also provide a refresher for some of the topics that have been in the exam since the VCP 3 days. The topics covered are listed below; the great thing about the modular style nature of these videos is that you can easily focus in on particular topics of interest and skip over others if you want to. I was tempted to skip a few of the chapters I thought I knew everything about, however I stuck with them and not only had a good refresher of those topics, but also picked up a few things I didn’t know.

Lesson 1 – Getting Started with VMware vSphere 5 Training Course
Lesson 2 – Lab Setup
Lesson 3 – Course Scenario
Lesson 4 – Overview of VMware vSphere 5
Lesson 5 – Installing VMware ESXi 5
Lesson 6 – Installing vCenter 5
Lesson 7 – Installing vCenter 5 as a Linux Appliance (vCSA)
Lesson 8 – Using the vSphere 5 Web Client
Lesson 9 – What’s New in vSphere 5
Lesson 10 – Navigating vSphere Using the vSphere Client
Lesson 11 – vCenter 5 – Configuring Your New Virtual Infrastructure
Lesson 12 – Creating and Modifying Virtual Guest Machines
Lesson 13 – Installing and Configuring VMware Tools
Lesson 14 – Understanding and Using Tasks, Events, and Alarms
Lesson 15 – Virtual Storage 101 and Storage Terminology
Lesson 16 – vSphere Storage Appliance (VSA)
Lesson 17 – Creating a Free iSCSI SAN with OpenFiler
Lesson 18 – Administering VMware ESXi Server Security
Lesson 19 – vSphere Virtual Networking
Lesson 20 – Using the vSphere Distributed Virtual Switch (dvswitch)
Lesson 21 – Moving Virtual Machines with vMotion
Lesson 22 – Moving Virtual Storage with svMotion
Lesson 23 – Performance Optimization with Distributed Resource Scheduler (DRS)
Lesson 24 – Implementing High Availability with VMware HA (VMHA)
Lesson 25 – Super High Availability with VMware Fault Tolerance (FT)
Lesson 26 – Upgrading from VMware vSphere 4 to vSphere 5
Lesson 27 – vSphere Command Line Interface (CLI) Options
Lesson 28 – vSphere Auto Deploy
Lesson 29 – Storage DRS
Lesson 30 – Policy-driven Storage
Lesson 31 – Understanding the New vSphere 5 vRAM Pooled Pricing
Lesson 32 – Network I/O Control (NIOC)
Lesson 33 – Storage I/O Control (SIOC)
Lesson 34 – ESXi Firewall
Lesson 35 – VMware Data Recovery (VDR) 2
Lesson 36 – Administering vSphere Using an iPad

Both David Davis and Elias Khnaser have excellent presenting styles, in their clear explanations and enthusiasm for the topic – which I am particularly grateful for having watched most of them on the early commute into work. While not directly aimed at the VCP exam, the videos provide a good breadth of coverage of most of the exam topics. As with the other series they are available online streamed from the website as soon as you make the purchase, but also shipped to you on a DVD in formats suitable for PC, iPhone iPad etc.

2) VCP5 Exam Blueprint

Reading the VCP5 Exam Blueprint document is essential to ensure that you have covered all of the bases. I worked through the document and ticked off each objective as I went. Most of my work is with the larger Enterprise deployments of vSphere, so this is a good way to ensure that you have not missed anything that might be more commonly used in say SMB deployments.

 

3) VCP5 Exam Blueprint Study Guide PDF

While working through the official blueprint document I discovered this Study Guide PDF which Jason Langer and Josh Coen have put together. They have put a lot of work into this document which typically contains a paragraph or two on each objective with highlight points and links to pages in vSphere documentation pdfs for further research. Given the limited time I had to prepare for the exam, this was great for quickly covering all of the objectives and I really appreciated the effort they had made in putting it together.

4) VCP 5 Brownbags

The Professional Vmware site is well known for running 1 hour brownbag sessions for VMware topics, including the VCAP exams. Recently they have started them for VCP5 too and although currently there are only a few, they are well worth checking out.

5)  vSphere 5.o Clustering DeepDive

I bought this book by Frank Denneman and Duncan Epping when it came out last year because I wanted know about new vSphere 5 features, however it is also worth reading as preparation for this exam. While the book goes into far more technical depth than required for the VCP, it was still useful for an early appreciation of these topics.

6) Practice Questions

It’s good to at least get a feel of the kind of questions you will be asked in the exam. Both the VMware VCP site with a mock exam and Simon Long’s blog are a great resource for this.

7) Hands-on experience with real world vSphere experience

Most important of all was hands-on experience of vSphere. There were many questions I knew the answer to that were not through the result of self-study, but either I had configured them at work or experienced the issue and had resolved it. You can emulate a fair amount of this if you have access to say a home-lab, but you can’t beat experience and unfortunately there is only one way to get that.

I was pleased to see that the exam was heading more in this direction. When I took the VCP 3.5 in 2008 the Minimum and Maximums document was a core part of my study process, in fact I took the exam while my family were away for a few days, so my house resembled something out of Prison Break with different pages of the Min / Max guide on the walls! I didn’t do that this time and while a good knowledge of the fundamental minimum and maximums is still required on the blueprint, the days of having to memorise every single one in the guide just to pass the exam appear to be over which is a good thing.

sdfsd

vSphere 5 vRAM Licensing: Working out what you will need

The release of vSphere 5 brings a new licensing model where existing CPU restrictions were removed and replaced with the concept of vRAM. This was a significant change and caused quite a kerfuffle during and after the launch event, to the extent that a lot of technology advancements were overshadowed. Full details of the new scheme are here.

OK, so there are already a ton of blog posts out there on this topic which cover the pros and cons of this change. A couple I found useful are below:

Initially I thought this licensing change might be a bit of a problem for us, but thought I would wait for the dust to settle and spend some time figuring out what the impact could be.

The day after the announcement there were already two PowerCLI solutions available to help calculate the impact:

I didn’t have much time to research these solutions too far before attending the London VMUG 2 days after the vSphere 5 launch. However, they were both showing that we would potentially need to increase our licensing significantly to be able to upgrade. Whilst at the VMUG I struggled to find anyone else in a similar scenario; most others seemed to be in one of the following two licensing positions:

  • I’ve got enough, with some headroom to spare
  • I haven’t quite got enough, so will either need to buy some more or upgrade some to Enterprise Plus

So this week I have spent some time properly researching the subject, during which time Luc has updated his function to split out the different license types and the below was also published:

A combination of the updated function from Luc, Alan’s script which produces a comprehensive HTML report and more time spent researching it has now confirmed that we are in the ‘I’ve got enough, with some headroom to spare’ camp, so a lot better position than I thought.

I am really pleased about this since it now means I can go back to focusing on the technology advancements rather than being concerned about whether it might be affordable or not.

PS When running Alan’s script watch out for a limit in the 32-bit version of PowerCLI which might trip you up.