Tag Archives: powercli

Automate vRealize Automation with PowerShell: Introducing PowervRA

While putting together the PowerCLI book 2nd Edition we initially included in the proposed Table of Contents a chapter on vRealize Automation. However, it was fairly apparent that at that time (early 2015) there wasn’t a lot which could be done to fill out the chapter with good content. Firstly, most of the relevant content would be included in the vRO chapter, i.e. use vRA to call a vRO workflow to run PowerShell scripts. Secondly, automating elements within vRA 6.x could be done in part via the REST API, but a) there was a roughly 50 / 50 split between what was in the REST API vs Windows IaaS and b) I didn’t really have the time to make both a PowerShell toolkit for vRA and write a book about PowerCLI.

So we shelved that chapter and I put the thought to the back of my mind that I would revisit the idea when vRA 7 came out and the likelihood of greater coverage in the vRA REST API. At the start  of 2016 this topic came up in a conversation with Xtravirt colleague Craig Gumbley who it turned out had the same idea for making a PowerShell vRA toolkit. So we decided to combine our efforts to produce a PowerShell toolkit for vRA for both our own use as consultants and also to share with the community; consequently the project PowervRA was born.

Initial Release

For the initial release we have 60 functions available covering a sizeable chunk of the vRA 7 REST API. Compatibility is currently as follows:

vRA: version 7.0 – some of the functions may work with version 6.2.x, but we haven’t tested them (yet). Also, they have not been tested with 7.0.1.

PowerShell: version 4 is required.  We haven’t tested yet with version 5, although we wouldn’t expect significant issues.

You can get it from Github  or the PowerShell Gallery

We have provided an install script on Github if you are using PowerShell v4. If you have v5 you can get it from the PowerShell Gallery with:

Getting Started

Get yourself a copy of the module via one of the above methods or simply downloading the zip, unblocking the file and unzipping,  then copying it to somewhere in your $env:PSModulePath.

PowervRA01

PowervRA02

Import the module first:

You can observe the functions contained in the module with the following command:

Before running any of the functions to do anything within vRA, you will first of all need to make a connection to the vRA appliance. If you are using self signed certificates, ensure that you use the IgnoreCertRequirements parameter. :

You’ll receive a response, which most importantly contains an authentication token. This response is stored automatically in a Global variable: $vRAConnection. Values in this variable will be reused when using functions in the module, which basically means you don’t need to get a new authentication token each time, nor have to specify it with a function – it’s done for you.

Each of the functions has help built-in, alternatively you can visit this site http://powervra.readthedocs.org

Example Use Case: Create a vRA Tenant

Having made a connection to the appliance, it’s now time to start using some of the functions. To create a Tenant in vRA we need to have made a connection to vRA with an account that has permissions to do so in the default tenant (typically [email protected]) and then it is as simple as the following:

PowervRA03

PowervRA04

If you look through the rest of the functions, you may notice that a lot of them contain a JSON parameter. So if you know the JSON required for the REST API request or are working with a system that produces it as an output, you can do something like the following:

The Future

VMware may put out something official at some point (I have no inside info on that, it could be weeks, months, years away or not even planned right now). Until that happens Craig and I have various things planned including greater coverage of the API, dealing with any feedback from this release and looking at automating some of our own testing so that we can more easily figure out which vRA versions are supported.

In the meantime, fill your boots and if you want to help us, feel free to get involved via the Develop branch on GitHub.

 

PowerCLI Book 2nd Edition is Now Available!

…..well in Kindle format anyway 🙂 The paperback version will be available on 11th January in the US and 12th January in the UK and according to the publisher, apparently in book shops from the 19th January. Seriously, if you actually see one in a book shop then please send me a photo, since I’ve not seen that happen outside of a VMworld.

 

9781118925119.pdf

With my fellow authors, Luc Dekens, Glenn Sizemore, Brian Graf, Andrew Sullivan and Matt Boren, we spent the best part of 2015 putting this book together. The 1st edition was written in 2010 and published in 2011; since then the VMware virtualisation landscape has changed significantly from pretty much VMware vSphere infrastructure and P2V projects and maybe some desktop work with VMware View to a wide variety of Management, Desktop, Application, Infrastructure and Cloud products.

Given this expansion of products and the fact the we all buy tech books ourselves and want value for money from them, we didn’t want to just ship an updated version to cover what was in the 1st Edition and make sure it worked with the latest versions of vSphere, PowerCLI and PowerShell. So as well as updating every chapter to make sure it worked with vSphere 6, PowerCLI 6 and PowerShell 4, and adding or replacing content and code to make sure it was still relevant (for instance the Distributed Switch is now covered out of the box, previously we contributed our own functions), we had the foolish excellent idea to add a number of additional chapters covering things such as:

  • vCloud Director
  • vCloud Air
  • vRealize Orchestrator
  • Site Recovery Manager
  • PowerActions
  • and an introduction to DevOps topics

Additionally the original Storage and Network chapter was split into two and content on new technologies VSAN and NSX added appropriately. To be honest each of VSAN and NSX could probably have their own chapters…maybe next time 😉

This took the page count to 984, around almost 200 pages more than the 1st edition, which is almost the size on its own of some tech books. Unfortunately, this partly led to the book taking longer than it should have to complete and there were still areas that we would have liked to include, but had to make some tough decisions not to or it would never have seen the light of day. As a wise man often says….

JeffreySnover2

While there is some introductory content in some of the chapters, this is not a book that typically runs you through how to use each of the PowerCLI cmdlets in their relevant areas, rather this is a book that goes beyond what ships out of the box and into areas where you will need to do some of the hard work yourself.  We had some negative feedback that the 1st edition was not better in introductory areas and may well do so again this time; there are other great PowerCLI books out there now which will do a better job for beginners. So imagine the front cover has Deepdive stamped across it and that will give you a better feel for what to expect.

Personally I feel I made a way better contribution to this book than the previous one for various reasons:

  • I was involved from the beginning, rather than parachuting in halfway through to try and get the book out of the door.
  • Consequently I was able to think about and plan for the content of the chapters I was responsible for, rather than just trying to make the deadline for each chapter.
  • Being used to the publisher’s Word template and what was required in them. There are so many rules about fonts, spacing and expectations around how the chapter needs to be laid out, that you seem to be just expected to know and first time round caused a lot of pain and time wasted.
  • A supportive employer, Xtravirt, who were interested in what I was doing, rather than one who previously was not.
  • A big thank you to our Technical Editor, Matt Boren, who left no stone unturned in making sure my code was not only accurate, but also some outstanding suggestions to help me improve. Some of which I have taken forward for everything PowerShell based that I write now. In fact he liked the experience so much, he even got roped into contributing some of the book content himself.
  • I actually had a reasonable idea about the technologies involved……;-)

 

Last time round I guessed how much time I spent on my contribution. Being curious, I decided to track it this time. It’s also something I get asked about now and again by people considering writing a tech book, so now I can refer them to some data 😉

Originally I was going to track the data by week, but since it went on so long it ended up being by month. I’m not sure if writing a book with code samples requires more or less effort than a tech book without them. On the one hand you almost have to produce double the content; explanatory written text and create code to make stuff happen. On the other hand at least the code uses up some pages when printed in the book 🙂

BookHours

In total I spent 330.5 hours (that extra 0.5 was crucial) of my own time on the effort for this book between Jan and Nov 2015, and remember there were 5 other authors. So if you are thinking about getting involved in a project like this then consider carefully whether you are able to commit to the level of time you may have to give up for it. I’m not sure how representative this is for authors of other tech books, but I never talked to one who didn’t say something like “you have no idea how much effort it takes”. Similar to last time, I pretty much worked on it most evenings Mon – Thu and stayed away from it over the weekend so that it did not impact my family life too much. (Although my children are now 5 years older and consequently don’t go to bed so early – which meant starting later and finishing too late regularly) It did mean I didn’t see my friends much during weekday evenings, which is when I tend to catch up with them, for about 6 months, but something has to give. I think my blog and podcast content suffered a fair bit too.

You’ll see that most of the original effort was in the early months either producing content for new chapters, or updating content for existing ones. I was actually originally expecting to be pretty much done by April, but for various reasons I took on a bit more than originally planned and April – June was spent producing more content and some early feedback reviews. August – November was pretty much taken up with reviews.

Reviews are something which I think vary from publisher to publisher, but for Wiley it pretty much follows this:

 

Author Review

If you can recognise your original text after it has been through all that process, then well done 🙂

So at least three responses required per chapter, multiply that by the number of chapters you commit to and then try finding time to write new chapters and respond to reviews both on similar deadlines. Pro tip: get your new content out of the way early on so you can do the reviews in peace –  see Jan / Feb in the chart 😉

With all that effort though, it is a personally (not financially) rewarding experience. I think both times I’ve said “Never again!” too many times, but I suspect it may happen one day. I’m confident that if it’s something you are interested in and you buy a copy that you will find it useful. I genuinely would happily buy a copy with my own money for the content I have seen the other guys produce for it.

Final comment: Writing in a foreign language also brings some difficulties. Having your own English ‘corrected’ to the US version requires a fair amount of patience. Not necessarily spelling differences, but phrases that are commonly understood this side of the Atlantic, but not the other. I’ll just leave this here……

Update 03/02/2016: Due to some issues mentioned in the comments around downloading the sample files, I have attached (most of) them here:

 

Onyx for the vSphere Web Client

The original Project Onyx (also here) was a VMware Fling which intercepted calls between the C# vSphere client and vCenter and could translate things you did in the GUI client to code in your choice of languages; SOAP, C#, PowerCLI, and JavaScript. This was really useful if say a thing you needed to do was not yet covered by a PowerCLI cmdlet, or you wanted to try and make it faster by using Get-View, or you didn’t know how to code it in JavaScript for vRO.

An issue arose however with the release of the vSphere Web Client, since Project Onyx did not support that method of intercepting calls. When VMware started moving new functionality only into the Web Client, that meant you could not use Project Onyx to figure out the code for those new features.

Enter a new VMware Fling – Onyx for the vSphere Web Client!

Some smart guys have now brought this same functionality to the vSphere Web Client. Let’s take a look at an example of how it works.

First of all follow the instructions on the Fling site to get it installed. Currently it supports vCenter 6.0. Note the warning about only using this Fling in your test environment (that’s where you write and test all of your code anyway right? 😉 )
WARNING: This fling replaces core Web Client files and may cause issues with stability and
patching of future versions of the web client, please only continue with this installation if you are
using a test or dev environment.

Once installed and the vSphere Web Client service has restarted, you will see a new menu item for Onyx.

Onyx01

Currently there is only PowerCLI.NET available as a code output choice. I’m hoping that since it is a drop down menu, there are more to come. I’ve already put in a Feature Request for JavaScript for handy use with vRO 😉

Onyx02

You can either Start and Stop recording from this area or very handily they have added the same buttons in the top right hand corner so that you can Start and Stop recording from anywhere in the Web Client.

Onyx03

So start the recording and navigate to the thing that you want to do. For this example we’ll turn on HA for a cluster.

Onyx03b

Once complete, stop the recording.

Onyx04

Now head back to the Onyx section of the Web Client and observe the code required to make that change.

Onyx05

$spec = New-Object VMware.Vim.ClusterConfigSpecEx
$spec.dasConfig = New-Object VMware.Vim.ClusterDasConfigInfo
$spec.dasConfig.vmComponentProtecting = 'disabled'
$spec.dasConfig.enabled = $true
$spec.dasConfig.admissionControlEnabled = $true
$spec.dasConfig.vmMonitoring = 'vmMonitoringDisabled'
$spec.dasConfig.hostMonitoring = 'enabled'
$spec.dasConfig.HBDatastoreCandidatePolicy = 'allFeasibleDsWithUserPreference'
$spec.dasConfig.admissionControlPolicy = New-Object VMware.Vim.ClusterFailoverLevelAdmissionControlPolicy
$spec.dasConfig.admissionControlPolicy.failoverLevel = 1
$spec.dasConfig.defaultVmSettings = New-Object VMware.Vim.ClusterDasVmSettings
$spec.dasConfig.defaultVmSettings.vmComponentProtectionSettings = New-Object VMware.Vim.ClusterVmComponentProtectionSettings
$spec.dasConfig.defaultVmSettings.vmComponentProtectionSettings.vmReactionOnAPDCleared = 'none'
$spec.dasConfig.defaultVmSettings.vmComponentProtectionSettings.enableAPDTimeoutForHosts = $true
$spec.dasConfig.defaultVmSettings.vmComponentProtectionSettings.vmStorageProtectionForAPD = 'disabled'
$spec.dasConfig.defaultVmSettings.vmComponentProtectionSettings.vmTerminateDelayForAPDSec = 180
$spec.dasConfig.defaultVmSettings.vmComponentProtectionSettings.vmStorageProtectionForPDL = 'disabled'
$spec.dasConfig.defaultVmSettings.vmToolsMonitoringSettings = New-Object VMware.Vim.ClusterVmToolsMonitoringSettings
$spec.dasConfig.defaultVmSettings.vmToolsMonitoringSettings.failureInterval = 30
$spec.dasConfig.defaultVmSettings.vmToolsMonitoringSettings.maxFailures = 3
$spec.dasConfig.defaultVmSettings.vmToolsMonitoringSettings.maxFailureWindow = 3600
$spec.dasConfig.defaultVmSettings.vmToolsMonitoringSettings.minUpTime = 120
$spec.dasConfig.defaultVmSettings.restartPriority = 'medium'
$spec.dasConfig.defaultVmSettings.isolationResponse = 'none'
$spec.dasConfig.option = New-Object VMware.Vim.OptionValue[] (0)
$spec.dasConfig.heartbeatDatastore = New-Object VMware.Vim.ManagedObjectReference[] (0)
$spec.dasConfig.hBDatastoreCandidatePolicy = 'allFeasibleDsWithUserPreference'
$modify = $true
$_this = Get-View -Id 'ClusterComputeResource-domain-c22'
$_this.ReconfigureComputeResource_Task($spec, $modify)

This is super cool. I really like the way it’s been implemented and looking forward to further development of this Fling 🙂

 

PowerCLITools Community Module: Now on GitHub

Over the last few years I have built up a number of functions to use alongside the out of the box functionality in PowerCLI. I’ve posted some of the content before on this blog, but have at last got round to publishing all of the content that I am able to share, in a module available on GitHub – I’ve named it the PowerCLITools Community Module in the hope that some others might want to contribute content to it or improve what I have already put together.

octobiwan

This took a fair amount of effort since it is not possible for me to share everything that I have as part of my locally stored version of this toolkit. Some of it was developed by others I was working on projects with (and are not as necessarily so keen to share certain parts of their work) and some can’t be shared for commercial reasons. However, I found some time recently to split out everything that could be shared into a new module and also updated some of the code – typically to add some nice features in PowerShell v3 and later which weren’t available when a lot of the code was developed during PowerShell v2 days.

Since the content has been developed over a few years, consistency and standardisation of approach may not be 100% there. A quick look back over them showed some looking a bit dated – I have spent a bit of time tidying them up, but part of the reason for sharing them  was to take feedback and some prompting on where they could be improved. If I left them until I thought they were just right I’d probably never end up publishing them. So your feedback is the impetus I need to go and improve them 🙂

A lot of the functions are there to fill in gaps in cmdlet coverage with PowerCLI and there are a few which I made more for convenience where I have bundled together a few existing cmdlets into one function. These don’t particularly add a lot of value, but maybe demonstrate how you can tighten up your scripts a bit

Pre-Requisites

Ensure that VMware PowerCLI is installed. Functions have been tested against v5.8 R1.

Installation

1) Download all files comprising the PowerCLITools module. Ensure the files are unblocked and unzip them.
2) Create a folder for the module in your module folder path, e.g. C:\Users\username\Documents\WindowsPowerShell\Modules\PowerCLITools
3) Place the module files in the above folder

So it should look something like this:

PowerCLIToolsModule01
Usage

The below command will make all of the functions in the module available

Import-Module PowerCLITools

To see a list of available functions:

Get-Command -Module PowerCLITools

Add-vCenterLicense
Get-ClusterAverageCpuMemory
Get-CurrentVIServer
Get-SnapshotCreator
Get-vCenterLicense
Get-VMCreationDate
Get-VMDiskData
Get-VMHostAlarm
Get-VMHostDumpCollector
Get-VMHostiSCSIBinding
Get-VMHostLicense
Get-VMHostNetworkAdapterCDP
Get-VMHostSyslogConfig
Get-VMIPAddressFromNetwork
Get-VMSCSIID
Install-vSphereClient
Install-vSpherePowerCLI
New-vCenterPermission
New-vCenterRole
New-VMFromSnapshot
Remove-vCenterLicense
Set-VMHostDumpCollector
Set-VMHostiSCSIBinding
Set-VMHostLicense
Set-VMHostSyslogConfig
Set-VMHostToCurrentDateandTime
Test-VIServerConnection
Update-ESXiSSL
Update-VMNotesWithOwner
Update-VMScsiDeviceOrder

Nested Modules

You will note that each function is itself a nested module of the PowerCLITools module. In this blog post I describe why I make my modules like this.

VI Properties

If you take a look inside the PowerCLITools.Initialise.ps1 file you’ll notice a number of VI Properties. Some of these are required by some of the functions in the module and some are just there for my convenience and make using my PowerCLI session simpler. You can add and remove VI Properties as to your own personal preference, but watch out that some are actually needed.  You can find out more about VI Properties here.

 Feedback

I really hope people find these functions useful. I have a number of ideas on where some can be improved, but please provide your own feedback as it’ll be the nudge I need to actually go and make the changes 🙂

Get-Task: ID Parameter is Case Sensitive

There aren’t many occasions when you trip up in PowerShell because of something being case sensitive, it generally doesn’t happen since most things are typically not like that. I was working with the PowerCLI cmdlet Get-Task and in particular the ID parameter to do something like:

Get-Task -Id 'task-task-2035'

I had originally found the ID via:

Get-Task | Format-Table Name,ID -AutoSize

However, I received the error that no tasks of that ID were found :

Get-Task : 24/02/2015 20:51:57 Get-Task The identifier task-task-2035 resulted in no objects.
At line:1 char:1
+ Get-Task -Id task-task-2035
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (:) [Get-Task], VimException
+ FullyQualifiedErrorId : Client20_OutputTracker_ReportNotFoundLocators_LocatorNotProduced,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetTask

Turned out that making the task ID match the exact case worked:

Get-Task -Id 'Task-task-2035'

Apparently the IDs are case sensitive by design 🙂

One to watch out for anyway…..

PowerCLI is now a Module!

We’ve been waiting for this a long time, but with the 6.0 release PowerCLI is now available as a module. Microsoft changed the way itself and third-parties should deliver PowerShell functionality back in PowerShell version 2 by offering modules. Previously in PowerShell version 1 additional functionality was available via snap-ins.

It’s not fully there yet, but some of the functionality is now available in a module. 6.0 will be a hybrid release, with the rest to follow later.

Notice how the Core functionality is in both lists since this is a hybrid release.

Get-Module *vmware* -Listavailable

PowerCLIModule01

Get-PSSnapin *vmware* -Registered

PowerCLIModule02

I believe there was significant effort in making this leap, so many thanks to Mr Renouf and his team 🙂

Automating Disk Zeroing on VM Deletion

A requirement for a project I had was to zero the VMDK of all VM disks at the time of VM removal.

smash_hard_drive

One consideration was to SSH into the host where the VM was located and use vmkfstools like the below on each vmdk to zero the disk.

vmkfstools –w /vmfs/volumes/<…>.vmdk

Looking for alternatives I found that the PowerCLI cmdlet Set-HardDisk has a ZeroOut parameter. Note the text from the help (version 5.8 R1):

Specifies that you want to fill the hard disk with zeros. This parameter is supported only if you are directly connected to an ESX/ESXi host. The ZeroOut functionality is experimental.

The points to note are:

  • You will need to connect PowerCLI directly to the ESXi host that the VM is registered on. So you will most likely first of all need to connect to vCenter to find out where the VM lives.
  • The functionality is ‘experimental’. A quick scan back through releases showed this had been the same for some time. From my observations the functionality appeared to work fine. There have been many things in vSphere over the years which have been ‘experimental’, but have usually worked fine.

So once you have identified where the VM is and connected to the ESXi host in question, it’s a case of simply looping through all of the disks and zeroing them (with a bit of logging thrown in) – note it will likely take a fair amount of time to zero each disk!

$VM = VM01
$HardDisks = Get-HardDisk -VM $VM

foreach ($HardDisk in $HardDisks){

$HardDisk | Set-HardDisk -ZeroOut -Confirm:$false | Out-Null

$Text = "Zeroed disk $($HardDisk.Filename) for VM $VM"
$Text | Out-File -FilePath C:\log\zerodisk.log -Append
}

vCO Custom Workflow Icons

While at VMworld Europe 2014 I noticed in some of the slides in a vCO session that the presenter was using some icons for workflows that did not appear to be standard. I was quite curious how to do this, but couldn’t find much information about it until I stumbled on the below training video.

If you don’t want to watch the video, here’s how to do it.

1) Get your images into vCO

Supported:

  • GIF
  • JPEG
  • PNG

Not supported:

  • BMP
  • TIFF

Navigate to the Resources tab and you will probably want to create a folder to store them in.

vCOCustomIcons01

 

Then Import Resources

vCOCustomIcons02

 

vCOCustomIcons03

 

Fill up the folder with the rest of the icons that you need.

 

2) Update the Workflows with the Custom Icon

Navigate to the workflow you wish to set a Custom Icon for and edit it. On the General tab select the button for the Workflow icon.

vCOCustomIcons04

The dialogue box will let you select from within Resources with a handy search box to find what you need:

vCOCustomIcons05

The workflow is updated with the custom icon.

vCOCustomIcons06

OK, you can’t really see much of a benefit here. The difference appears when you embed your custom icon workflows into a parent workflow, which can now look something like this:

vCOCustomIcons07

 

 

Pretty cool I think; I could have so much fun with this 🙂

I’ll be presenting some automation at the June 2014 South West VMUG

VMUG

Those great guys down in the South West of England, @mpoore, @jeremybowman,  @virtualisedreal and @simoneady have kindly invited me down to their next VMUG to present about automation. So I will be talking about some of my experiences in automation projects from the last few years and particularly how to write your own code in a generic way so that it is portable across different projects and systems.

It looks like there is plenty of other good content lined up that day so I’d suggest you get down there too.

VMUG02

 

 

 

PowerCLI In The Enterprise: Breaking The Magician’s Code & Function Templates

At today’s #UKVMUG I presented on the topic: PowerCLI In The Enterprise: Breaking The Magician’s Code. Below are the slides from the session:

During the session I discussed breaking your PowerShell code down into functions and modules. To aid with this I am posting the 6 function templates I use which cover many of the typical scenarios I write a function for and enable the rapid creation of these functions since much of the code is typically repeated. These are:

  • Get-Something
  • Set-Something
  • Get-VMSomething
  • Set-VMSomething
  • Get-VMHostSomething
  • Set-VMHostSomething

and can be downloaded below. I hope you find them useful 🙂

I am also posting an XML based snippets file by which you can add all of the above functions to your PowerShell ISE User-Defined Snippets. Simply copy this file to your $home\Documents\WindowsPowerShell\Snippets folder (you may need to create this folder since it is not there by default). Rather than then having to copy and paste one of these template functions, you can insert them via Ctrl-J. This post will explain to you more how to use snippets in the ISE.

ISESnippet11

ISESnippet12