Using Git, Stash and Dropbox to Manage Your Own Code

Sometimes I’m asked how I manage my own (PowerShell) code, in terms of version control, backups, portability etc. In this presentation I demonstrated how my PowerShell code is typically broken down into functions and then placed into modules. This allows me to make very generic code for granular tasks, typically either to plug a gap missing from the out-of-the-box cmdlets or maybe stringing a few of them together. As a consultant this enables me to build up a toolkit of functions for particular scenarios gained over various different experiences and use them in a modular fashion where needed for each particular project. However, once these number in the hundreds how do you manage them effectively? I need them to:

  • be easily available depending on where I am working
  • be backed up
  • track changes via version control, useful even if you are not working in a team developing code together – mostly so I can remember how or why I changed something :-)

So I’m going to run your through the system I have found that works for me. It uses the following components:

  • A Dropbox account to sync the code between different machines, be available to download via a web browser and also store the code outside of my home lab. This means I can get access to my functions pretty much wherever I am and whether I am using my own or a customer machine.
  • A Linux VM in my homelab to run Git and Atlassian Stash for version control – $10 for a 10 user license (free to try out for 30 days)
  • Atlassian SourceTree. Free Git client for Mac or Windows

DropboxCode Stash I discovered Stash via a previous customer and found it to be a very useful and easy to use add-on to manage Git repositories. It’s possibly overkill for my current needs, but I have worked on shared code projects in the past and it could be useful to have an easy way to do this in the future. If you are thinking “why not use GitHub?”, there are reasons for this. While I share a lot of my code via this blog and possibly via GitHub in the future, there are some commercial and other reasons why I’m not able to share everything. GitHub has private repositories for this, but they start at $7 a month, so this approach with a home hosted Stash does me fine for now. Setting Up Stash I have a 1 vCPU, 1GB Ubuntu VM, installed with the Minimal Server option. The first thing to do is install Git. The below command will install Git and any dependencies.

sudo apt-get install git-core

Stash01 Check the versions  of Git and Perl. The version of Git should be 1.7.6 or higher. The version of Perl should be 5.8.8 or higher. (Note that out of the box RedHat / Centos currently do not include a version of Git that supports Stash.)


git --version
perl --version

Stash03 Check the version of Java and install if necessary. The version of Java should be 1.6.0 or higher. The below is the easiest way I found to do this.


java -version
sudo apt-get purge openjdk*
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
java -version

Stash04 Create an install folder, download the Stash installation files (obviously change the version number for that which you wish to download) and extract the download.


sudo mkdir /var/stash
sudo mkdir /var/stash/install
sudo mkdir /var/stash/home
cd /var/stash/install
sudo wget http://www.atlassian.com/software/stash/downloads/binary/atlassian-stash-2.11.4.tar.gz
sudo tar zxf atlassian-stash-2.11.4.tar.gz

Create a user account to run Stash under.


sudo /usr/sbin/useradd --create-home --home-dir /usr/local/stash --shell /bin/bash stash

Set STASH_HOME in setenv.sh to /var/stash/home.


sudo vi atlassian-stash-2.11.4/bin/setenv.sh

Stash05 Change the ownership of the stash folder and below to the stash account


sudo chown -R stash /var/stash

Start Stash.


/var/stash/install/atlassian-stash-2.11.4/bin/start-stash.sh

Stash06 Navigate to the Setup Wizard at http://hostname:7990 Take the option for an Internal database Stash07 Enter your license key Stash08 Enter Administrator credentials and Go to Stash (I am not using the accompanying Jira product) Stash09 This will take you to the login page where you can authenticate with the credentials you just created. Stash10 You should now see the Stash Welcome Page. Stash11 Connecting the SourceTree Client to Stash Download the SourceTree Client for Mac or Windows and get it installed. Once installed, fire up the client. You may be prompted to install Git for your local client to issue Git commands, go with the embedded version if so: Stash12 Say No to Mercurial (unless you wish to use that as well) Stash13 Enter your information – if you have a GitHub account and wish to use repositories there it can be useful to add the associated email account here. Stash14 Accept the recommendation to use Putty Stash15 If you want to use an SSH key then enter that here, I’m not going to for this tutorial. Stash16 Enter your credentials for the Stash website (and others if you wish to) Stash17 Once complete, the SourceTree client will open. They are pretty regular at providing updates for it, so you may be prompted to install further updates before first use. Stash18 Stash19 Stash20 Now we need to create a project and a repository. First of all through the web interface to Stash create a relevant project. (I liked the way it shortened this one to POW – reminds me of the old Batman series) Stash20a Stash20b Now create a repository Stash20c Stash20d Further down this page you will find instructions on how to upload existing code, which in my case is what I want to do. Stash26a From the SourceTree client open up a Git terminal and run the above commands: Stash26 Stash27 Stash28 At this point if you navigate back to your Stash website you should see your files have been uploaded. You can view inside the code too via the webbrowser. Stash31 Now I need to configure the SourceTree client to be aware of this repository. Click the button for Clone / New and choose Add Working Copy. Enter the path to your working folder and Add. Stash29 It is now available in the SourceTree client. Stash30   So now when I either:

  • Edit code directly on my laptop
  • Or copy code edited elsewhere and pop it in the Dropbox folder

Following synchronisation with the Dropbox folder, the SourceTree client will show that files have changed and require committing. In the below example a couple of minor spelling mistakes have been corrected. SourceTree shows the file has been updated and also what has been changed since the previous version. Red line was removed, Green line was added. Stash32 Right-click the file and choose Commit. Stash33   Enter an explanation for what was changed in this version and it also makes sense to select Push commits immediately to origin so that you create the commit and also submit it in the same action. Stash34 Assuming there were no errors then SourceTree should display no files to commit and navigating to the Log / History tab you will notice that a history of your changes will start to build up. Again, useful if you need to track back and see what changed and when. Stash35

Networking for VMware Administrators: Book Review

NetworkingForVMwareAdministrators
I was fortunate enough to receive a preview copy of Networking forVMware Administrators, so bear that in mind when reading this review. However, I found it to be such a useful source of information that I would more than happily purchase my own copy. Having arrived at VMware virtualisation a few years ago from a background in Enterprise IT, this is a book I wish had been available back then.
The book takes the approach of giving the VMware team enough information about networking to be able to have a productive conversation with the network team and vice-versa for virtualisation based networking conversations with the VMware team. Often times these can be difficult conversations if one or both parties is not familiar with the requirements and challenges of the other team’s specialist areas.
traceyblogimage-meeting-small-size1
Back in the day when I first started with virtualisation all of a sudden I needed to know about virtual switches, VLANs, CDP, network security requirements etc, having previously only needed to patch a few cables in, some basic TCPIP and firewalling, and some networking stuff around Microsoft Clustering and NLB. This book will provide you with all of the info for those conversations with some excellent practical examples of each topic.
More than that though, throughout the book the authors will pitch in with design elements of virtual networking to give you more background and insight into why to do things in a certain way, not just how. This is followed up with excellent design based chapters at the end of the book covering different scenarios based on how many and what speed nics are available in each compute device which I’m sure I’ll be referring back to.
I also found the chapters on Storage based networking (covering both iSCSI and NFS) great, since having spent the early part of my virtualisation experience with Fibre Channel storage a recent foray into the world of NFS for vSphere was a different challenge. Again, armed in advance with the knowledge from this book would set you well down the path of designing virtual networking for storage.

The book has an enjoyable reading style with the authors’ humour filtering through well, which sometimes isn’t always the case with tech books since tech publishers seem to think that the dryer a book the better. It’s a no brainer to me that you’ll learn more from a book if its a pleasure to read and you can empathise with some of the authors own experiences which I did with this book and some of their practical tales.
I did skip some of the content in the early chapters on general networking info, however not as much as I was expecting to since I found the content filled in some gaps in my basic network knowledge, often around the why rather than the how.
In summary if you work in any way with a vSphere environment then I think you would gain knowledge from this book.

vCAC: Error requesting machine. (Exception from HRESULT: 0x8004D025)

While setting up vCAC 6.0 in my home lab, I encountered the following error when trying to deploy a machine from a blueprint.

Error requesting machine. Inner Exception: Exception has been thrown by the target of an invocation. Base Exception: The partner transaction manager has disabled its support for remote/network transactions. (Exception from HRESULT: 0x8004D025)

vCACSQLDTC00

I found a VMware KB article for an earlier version along similar lines which requires configuring some MSDTC settings on the SQL server holding the vCAC database. (It appears that this is a reasonably common issue for SQL based applications)

The recommendation from the VMware KB article is to configure MSDTC with the following:

Go to Start > Administrative Tools.
Open Component Services.
Expand Component Service > Computers > My Computer > Distributed Transaction Coordinator.
Right-click Local DTC and click Properties.
Click the Security tab.
Select the Network DTC Access option.
Select Allow Remote Client and Allow Remote Administration options.
Select the Allow Inbound and Allow Outbound options
Specify NetworkService for DTC Logon Account.
Click OK.

vCACSQLDTC01

Following those changes the MSDTC service needs to be restarted.

vCACSQLDTC02

My SQL server is 2012 running on Windows Server 2012 and is easy to change because it was for a homelab. Depending on the environment you might be deploying into, some consideration will need to be taken in advance into whether the above can be changed (if necessary) on a Production SQL server or whether a dedicated SQL server might be required.

VCAP5-DCA: Cmdline Quick Summary Sheets

As part of my preparation for the VCAP5-DCA  I needed to sharpen up a bit in some of the cmdline areas that I wasn’t overly familiar with. Listed in the blueprint for the exam are areas such as creating storage claimrules with esxcli, using vscsiStats or creating ESXi Image builds.

Some of these cmdline tools are more discoverable than others and while you aren’t required to memorise how they all work since you have the documentation available in the exam, you won’t really have time to look it up. I also found the quality of the built-in help for the various commands to vary quite significantly, so relying on that route could be problematic too. For instance if you need to add a storage claimrule the help at that point has a bunch of handy examples which you can copy and amend where necessary, but others aren’t so good.  In the example below we need to mask a storage path and there is a very similar example in the help.

esxcli storage core claimrule add -r 500 -t location -A vmhba35 -C 0 -T 1 -L 0 -P MASK_PATH

The best thing to do is get as much hands on experience with each tool as possible. However, I still found it useful to note down examples of how to use each one as I worked through the various study guides and videos, so that when re-visiting a topic as an exam refresher I could quickly get my brain in gear for what was needed.

Similar to how I used to memorise the Configuration Maximums for the early VCP exams, I pinned each sheet up around my study area after working it through in the lab. Then over the next few days I would test myself on each topic and see how quickly I could remember the commands to run through to say add a claimrule and make it active, or create a custom esxtop config.

So I’ve zipped my cmdline quick summary sheets up and attached them to the bottom of this post in case you might find the same approach useful. There are summaries for:

AutoDeploy
ESXi Image Builder
Esxtop Export and Replay
LUN Masking
Mark As SSD
PSA
VMA
vscsiStats

The point I need to stress is you can’t memorise the commands for every possible scenario, but being able to quickly remember which ones to use and the syntax for how they work can be helpful in the exam when time is of the essence.

“Now how do you mark that disk as an SSD again……..?”

large_prison-wall

VCAPCmdlineQuickSummarySheets

Office 2013 Performance Issues in VMware Fusion 6

I was experiencing some issues running Office 2013 64-bit inside a Windows 7 64-bit VM in VMware Fusion 6.  In particular Word crashing more than Word in Office for Mac (which is saying something) and Excel barely usable, crashing often less than a couple of minutes after being started.

OfficeFusion02

Some research led me to this VMware Communities post and other similar suggestions around turning on Disable hardware graphics acceleration and turning off Use subpixel positioning to smooth fonts on screen. I made those changes and the improvement was drastic (particularly so for Disable hardware graphics acceleration) - no crashes in either application since making the change (most of the rest of the day, compared to crashing every few minutes) and hopefully going to stay that way.

OfficeFusion03

Using More Than One External Monitor in VMware Fusion 6

I needed to connect two external monitors to my MacBook Pro and use them with a Windows VM inside Fusion 6. Initially when I connected both, Windows would only recognise one of the external monitors, seemingly dependent on which was connected first.

A communities posting revealed a similar issue along these lines. Essentially the automatic detection of the monitors was not working correctly and the suggestion was to power down the VM and update the VM’s vmx file with the below:


svga.autodetect = "FALSE"
svga.numDisplays = "3"
svga.vramSize = "134217728"
svga.maxWidth = "8192"
svga.maxHeight = "4096"

This will turn off the automatic detection of the display limits, and allow the VM to use up to 3 displays. You can change it to 4 if you think you might connect a 4th display (using both Thunderbolt ports and HDMI). The absolute maximum number of supported displays is currently 8.

Powering back on resulted in the two external monitors being available, although it still seems a bit flaky. I eventually settled on powering on the VM with no external monitors connected and then once booted, connecting them in the order I wanted them positioned.

MultipleMonitor01

Using PowerShell Aliases in a Module

Creating your own aliases in PowerShell is pretty straightforward. The New-Alias cmdlet allows you to create handy shortcuts for existing cmdlets or functions you have created yourself.

I write a lot of functions for my own modules, so having shortcuts for some of these functions would be pretty useful when using them via a module. However, when I first added some to one of my modules they weren’t available for use. Let’s have a look at an example to see what you need to do to make them available.

We have a module TestModule containing one function and one alias.

ModuleAlias01

The contents of the module are:


function Write-HelloWorld {

Write-Host "Hello World"
}

New-Alias -Name HelloWorld -Value Write-HelloWorld

However, when I import the module the function is available, but the alias is not.


Import-Module TestModule
Write-HelloWorld
HelloWorld

ModuleAlias02

To make the alias available, we need to add the following line to the module file , since by default a module will only export functions.


Export-ModuleMember -Alias * -Function *

ModuleAlias03

Now if we re-import the module (note we need to use the -Force parameter since the module is already loaded) the HelloWorld alias is available.


Import-Module TestModule -Force
HelloWorld

ModuleAlias04

Synology DSM: Enabling iSCSI Multiple Sessions

During setting up an iSCSI LUN on my new Synology box (DSM 4.3), I encountered an issue where the first ESXi box would connect successfully to the iSCSI target, but not the second. So no devices showing on the second node after configuring the iSCSI adapter, despite no apparent errors during the config process.

SyniSCSI00

Looking at the Synology box showed the first ESXi node connected (172.20.0.210), but not the second.

SyniSCSI01

I noticed on the above screen Multiple Sessions – Disabled. I hadn’t seen any mention of that while setting up the iSCSI LUN so looked into the config.

SyniSCSI02

So it appears that multiple sessions are disabled by default, accompanied by a warning that your OS must be cluster aware. Comfortable that ESXi meets that requirement, I decided to proceed.

SyniSCSI03

Multiple sessions are now available.

SyniSCSI04

Rescanning the iSCSI hba and the device now appears.

SyniSCSI05

Now the Synology box shows multiple sessions connected.

SyniSCSI06

 

VCSA: ‘Active Directory Enabled’ Fails During Setup Wizard

During the setup wizard of the vCenter Server Appliance I experienced an error at the step to make it Active Directory Enabled.

VCSA01

In this instance I received the below error:

Failed to execute '/usr/sbin/vpxd_servicecfg 'ad' 'test' 'sunnydale\vcentersvc' CENSORED 'sunnydale.local'':
VC_CFG_RESULT=309(Error: Invalid hostname. FQDN is required for joining a domain.)

VCSA02

Initially I thought it may be something to do with how I was specifying the AD domain name or username. Didn’t get very far with that, so I skipped it for the time being and decided to come back to it later.

This setting can later be configured from vCenter Server, Authentication . Attempting the same from there results in a similar error:

AD Authentication settings
Error: Invalid hostname. FQDN is required for joining a domain.

VCSA03

What the error (obviously) actually means is that the hostname specified on the Network page is the full FQDN and not a standalone hostname. So I had not specified that correctly as part of the initial setup.

Not this:

VCSA04

 

Rather this:

VCSA05

 

Also, make sure you have created a DNS A record for this server.

Once that was resolved I had some further issues with the formatting of the Domain and Admin User, in the end the below worked, i.e. username in the format [email protected]  . (Note I had been experimenting with a change to the administrator account rather than the vcentersvc service account which is why that is different below, I don’t think that was the cause of the issue)

VCSA06

PowerShell Tail Equivalent

Saw this tweet yesterday about an equivalent of tail -f in PowerShell

Tail01

(I think Simon meant Get-Content)

In PowerShell v3 the Get-Content cmdlet received an additional parameter -Tail. This means rather than pulling back the whole of a text or log file, you can just get the last x lines, e.g.


Get-Content .\vmkernel.log -Tail 10

Tail02

Nice, but how can we make this more friendly to those used to typing tail -f?

PowerShell ships with some built-in aliases to help shorten your typing when working at the command line and / or to decrease the initial learning curve. For example cat is an alias for Get-Content, so we could type:


cat .\vmkernel.log -Tail 10

but that just looks a bit weird!

So, since it is possible to create your own aliases with New-Alias, we can combine that with the below Get-ContentTail function to get an experience similar to tail. Stick both in your PowerShell profile and it’ll be available anytime you open a session.


New-Alias tail Get-ContentTail

As Simon also points out above, we can use the -Wait parameter to get a similar experience to -f in Unix. So with my modified version you can do this:


tail .\vmkernel.log 10 -f

Tail03

Now if I simulate (via Wordpad) an application adding lines to that file

Tail04

you’ll see the lines appear in the PowerShell session.

Tail05


function Get-ContentTail {
<#
 .SYNOPSIS
 Get the last x lines of a text file

 .DESCRIPTION
 Get the last x lines of a text file

 .PARAMETER Path
 Path to the text file

.PARAMETER Lines
 Number of lines to retrieve

.INPUTS
 IO.FileInfo
 System.Int

.OUTPUTS
 System.String

.EXAMPLE
 PS> Get-ContentTail -Path c:\server.log -Lines 10

.EXAMPLE
 PS> Get-ContentTail -Path c:\server.log -Lines 10 -Follow

#>
[CmdletBinding()][OutputType('System.String')]

Param
 (

[parameter(Mandatory=$true,Position=0)]
 [ValidateNotNullOrEmpty()]
 [IO.FileInfo]$Path,

 [parameter(Mandatory=$true,Position=1)]
 [ValidateNotNullOrEmpty()]
 [Int]$Lines,

[parameter(Mandatory=$false,Position=2)]
 [Switch]$Follow
 )
 try {

if ($PSBoundParameters.ContainsKey('Follow')){

Get-Content -Path $Path -Tail $Lines -Wait

}
 else {

Get-Content -Path $Path -Tail $Lines
 }

}
 catch [Exception]{

 throw "Unable to get the last x lines of a text file....."
 }
 }