VMware vSphere 4 supports a range of hot storage management technologies:

• vStorage VMFS Volume Grow
• Hot Extend for Virtual Disks
• Hot Virtual Disk Add/Remove

With these capabilities, if space gets tight in your vSphere environment, it is easy to be proactive and address the issue before anyone notices.  The process goes something like this:

• Allocate additional physical hard disk space on your SAN to an appropriate LUN
• Grow your VMFS datastore onto the newly added free space
• Extend, or add new, virtual disks for the VMs that need more storage
• Expand the volume inside the guest to create more usable space

All without a reboot: zero downtime.

Not All Virtualization Platforms are Created Equal

Don’t jump to the conclusion that all hypervisors offer the same flexibility.  Perhaps you are wondering about Hyper-V capabilities?

First, let’s take a look at some Microsoft System Center Virtual Machine Manager 2008 R2 (SCVMM) marketing statements:

The What’s New page announces:

Hot addition/removal of Storage: Allows the addition and removal of storage to virtualized infrastructure without interruption. Additionally, “live” management of virtual hard disk (VHDs) or iSCSI pass through disks, allows administrators to take advantage of additional backup scenarios and readily use mission critical and storage-intensive applications.

The Top Benefits list proclaims:

Hot addition/removal of storage: With this capability, administrators can quickly and efficiently respond to changing storage requirements of virtual machines. This ability to hot-add additional storage eliminates the previous need to take the host down to upgrade storage thus increasing business continuity for end users and reducing complexity for administrators. Additionally it allows administrators to confidently deploy mission critical applications (in which up-time is of paramount importance) that may have rapidly changing storage requirements such as web, database or other business applications.

An IT decision-maker just might get the impression that both ESX and Hyper-V have essentially the same features. They do not.

You may be surprised to find out that all of the descriptions above merely refer to adding a new virtual disk to a VM — providing the conditions are right.  You cannot grow an existing VHD, and you can’t safely remove a VHD with SCVMM.

Plan Ahead

First things first.  If your Hyper-V VM does not have a virtual SCSI adapter — templates and VMs from Hyper-V R1 do not — you won’t be able to hot add a new VHD until you correct that shortcoming.  Hello downtime.

SCVMM can add a new blank virtual disk to a VM or it can copy an existing one across the network from the Library — if you copied it there beforehand, but there is no way to add a VHD that may already be present on your SAN — even if it is already sitting right next to the destination VM.

Removal?

Up-time may be of paramount importance, but preventing data-loss was evidently not part of the original design.  Removing a VHD with SCVMM results in the immediate deletion of the underlying VHD file.  Ouch!  Thankfully, a recent patch improves administrator job security by throwing up a warning before this happens, providing an option to cancel.  There is no way to simply disconnect a VHD using SCVMM.

It turns out that if you really want to take advantage of those “Additional Backup Scenarios” by hot adding and removing virtual disks, you need your trusty Hyper-V Manager utility.  Still think System Center is a single pane of glass?  And that’s not the only task that requires administrators to switch between Hyper-V interfaces.

Conclusion

Microsoft is trying hard to ride the coattails of VMware ESX.  The latest release of Hyper-V R2 still does not have capabilities enjoyed by VMware administrators since the ESX 3.5 days.

Don’t believe the obfuscated marketing literature.  VMware vSphere is for real.

Related posts:

1. Easy recovery from a full VMware ESX datastore
2. Responsible Thin Provisioning in VMware vSphere
3. vSphere Thin-Provisioned Disk Performance
4. Snapshots that shoot back

Thin disk provisioning is a fully-supported feature in vSphere 4 that can save tons of storage space on your SAN by allowing virtual disks to consume storage space as needed instead all at once.

Under the supervision of VMware vCenter Server, ESX 4 thin provisioning is safe and reliable even for production workloads thanks to advanced storage accounting and built-in monitoring.  And even if the worst does happen — an unexpectedly full datastore — recovery is simple.

Now that you are convinced that VMware vSphere thin provisioning has a place in your data center, you may be wondering about performance tradeoffs.

It turns out that thin disks perform just about as well as thick disks.

New Performance Results

In this brand-new VMware vStorage Thin Provisioning performance study, VMware performance engineers compare thick and thin disk performance.  This must-read document covers several important topics, such as:

• I/O-intensive benchmarking with a 16-node ESX cluster
• File copy benchmarking with 2 ESX hosts
• Fragmentation impact
• Thin provisioning affect on co-located thick disks

Now, go forth and provision… thinly.

Related posts:

1. Responsible Thin Provisioning in VMware vSphere
2. Finding thin-provisioned virtual disks with PowerShell
3. Easy recovery from a full VMware ESX datastore
4. VMware Update Manager Performance and Practices

About a year ago, I wrote about the complex requirements for enabling PRO Tips in System Center Virtual Machine Manager 2008 (SCVMM).  That was the previous release — what has changed now that SCVMM 2008 R2 is available?

Not much.

First Things First

Before you can use PRO to Live Migrate virtual machines between hosts (which is sort of the whole point), you need to first configure Hyper-V, failover clustering, and Cluster Shared Volumes — something eWeek found to be significantly more complicated than VMware ESX.

After Hyper-V and Live Migration are up and running, the next step is to install SCVMM 2008 R2 so you can integrate it with your System Center Operations Manager 2007 (SCOM) environment.

What’s that?  You don’t have SCOM deployed?  Go ahead and set it up.  Come back in a few days when you’re ready to go to the next step.

Integration Time

After your environment has both SCVMM and SCOM up and running, it’s integration time! If you have ever thought that the next-next-finish Microsoft wizards were boring, then this is the task for you.  You’ll get a chance to make configuration changes to various systems and possibly even Active Directory — haven’t you always wanted to learn about the SetSPN utility anyway?

But don’t let me spoil the excitement by giving away the ending — pick up your own copy of the 34-page integration guide and experience the thrill for yourself.

Second Opinion

Some of you are probably thinking that this is an exaggeration, it can’t be all that bad.  Let’s take a look from another perspective, presented in the Infrastructure Planning and Design (IPD) guide for SCVMM 2008 R2:

As you can see from the above excerpt, integrating SCVMM with SCOM clearly increases the time, effort, and complexity of your virtualization deployment.  And don’t forget that SCOM is required for monitoring your Hyper-V infrastructure, too.

Unlike the Microsoft virtualization conglomeration, VMware DRS is an integral part of vCenter Server — purpose-built for virtualization.  Talk about additional layers…

Related posts:

1. PRO Tips: pros only, please
2. Managing VI3 with SCVMM considered harmful
3. Say it ain’t so, PRO!
4. Save $14,970 on VMware ESX management

Unbelievable.  That old tale about VMware adding an extra layer is still being told in the Microsoft Virtualization Comparison Brochure.  [Update: The document is currently unavailable for download; MSFT has pulled it for editing.] Pretty surprising, and not at all honest — using the term layer implies a particular order or hierarchy, not merely quantity.  And what on earth is the virtualization layer doing on top of applications?

The case Microsoft is really trying to make here is that if you don’t have VMware virtualization, then you don’t have VMware.  True enough, I suppose.   But you still have the same number of layers.

I took the liberty of making some corrections to the Microsoft collateral and am pleased to present three more appropriate ways to depict these virtualization layers.

The Objective Comparison

Truth be told, both virtualization platforms have approximately the same high-level depiction of layers:

Nothing surprising there — that is how any truly technical content would describe the arrangement.  Notice that virtualization is actually not above the application layer.

The Proportional Correction

VMware ESX and Microsoft Hyper-V both function as a hypervisor, but it turns out that since ESX has a much smaller footprint, one might envision the stack of colored boxes more like this:

Combining Layers — The Right Way

Is virtualization more like hardware or more like software?  If I had to choose, I would pick hardware.  Perhaps those are the two layers that should be combined, like this:

As long as the hypervisor is not tied to a general-purpose operating system, this makes good sense.  However, since Hyper-V is merely a role in Windows Server, it would be a real stretch to picture the same combination for Microsoft virtualization.  Now that I’ve reasoned it through, maybe three layers isn’t so bad after all.

How do you like them layers?

Related posts:

1. Would you buy a hypervisor from these guys?
2. A very flashy hypervisor: Hyper-V Server R2
3. Managing VI3 with SCVMM considered harmful
4. What is VMware ESXi?

PC World recently published an article on Hyper-V R2 competing with VMware vSphere.  I found this statement from Bob Kelly, Microsoft Corporate Vice President, Infrastructure Server Marketing, to be remarkable:

“That’s why I still maintain that at some point, you’ll be at 40 percent to 50 percent virtualized and you’re 50 percent physical, and that’s an important thing to recognize. If you recognize that, you can set the strategy. It’s not, ‘Oh God, we thought the world was going to be only virtual.’ That, fundamentally, is why I think VMware is in trouble,” he said.

In related news, Gartner recently predicted that 50% of all x86 server workloads will be virtualized by 2012.  Still no word on what happens in 2013, after global virtualization limits have been reached.  [Better start looking for some investors that can help me get VM cap-and-trade started.]

But seriously, many companies have already instituted “virtualization first” policies and deploying new apps on physical hardware requires sign-off from upper management.  If Bob Kelly is correct, half of all provisioning requests seek waivers from this policy?  Incomprehensible.

Bob Kelly also explained the true cause of server sprawl in that same article.  If you’re like me, you probably thought it was because of DLL Hell — complex product dependencies that made it extremely risky to manage multiple applications on a single Windows NT system — and SMP scalability problems.  Nope, it turns out that the real issue was the need for sufficient capacity to accommodate peak demand… wink, wink.

Which 50% will you virtualize?

Related posts:

1. VMware ESX 4 can even virtualize itself
2. Single Pane of Glass — Hyper-V Edition

It can be difficult to set up a complete Hyper-V environment.  There are a lot of user interfaces needed to perform the various configuration tasks — especially during initial deployment.  It’s great to see that the mainstream technology media has arrived at the same conclusion.

Jason Brooks, Executive Editor at eWeek, recently reviewed Windows Server 2008 R2 and had the following to say about Live Migration and Cluster Shared Volumes (CSV):

Live Migration was very easy to use, but I found the process of configuring and using Live Migration in Windows Server 2008 R2 significantly more complicated than with VMware ESX server and VirtualCenter. In contrast to VMware’s product, where all tasks are gathered together in a purpose-built interface, the tasks required to configure Cluster Shared Volumes in Windows Server involve visits to various existing and new Windows utilities.

It’s true.  The Microsoft virtualization solution is a conglomeration, and configuring all of the various layers, roles, and features on top of the general-purpose Windows OS is complicated.

You may need Windows to run parts of your business, but that doesn’t mean you need Hyper-V.  Take control of Windows — by running it on rock-solid, purpose-built, enterprise-class VMware vSphere — before it takes control of you.

Simplify, don’t complexify.

Related posts:

1. Hyper-V Console Disconnects During Live Migration
2. SCVMM/PRO Complexity: High
3. How to see if Quick Migration is right for your workload
4. Hands off that CSV!

Today is the big day — Windows Server 2008 R2 is generally available at last.

Please, take a moment to acknowledge this release.  No, really.  I mean, if Microsoft thinks there is a single person left in the free world that hasn’t heard about R2 and free Live Migration, they might be forced to produce another wave of announcements.

They may even resort to Tupperware-style home launch parties for the server next time.  Have the neighbors bring blade servers over for that one — you know, power and cooling issues in the living room.

If you count beta and RC releases, this thing seems like it has been announced a dozen times already this year.  And considering that they’ve been doing battle against VMware with those unreleased products all along, counting them would not be unfair — especially considering the fanfare that accompanied each pre-release.

We’ve got it.  It’s here, it has free Live Migration, and it’s super fun.  According to some reports, R2 has even been known to whiten teeth and freshen breath.

So, please, acknowledge the release of R2 today.  Do it for the children.

Related posts:

1. Windows Server 2008 R2 — Windows 7?
2. Wild SCVMM 2008 R2 rumors and speculation
3. Hello, SCVMM 2008 R2
4. Microsoft System Center Virtual Machine Manager released

Tomorrow is the big launch of Windows 7 and Windows Server 2008 R2, the next major Microsoft operating system releases. A significant change this time around is the simultaneous release of the client and server.  However, I may have inadvertently uncovered the truth today:

Windows Server 2008 R2 will not be version 7 at all!

Actually, Windows Server 2008 R2 will be version 6.1. Will it still be binary compatible with Windows 7?

Sorry about that folks… don’t panic, Windows 7 is version 6.1, too.

Related posts:

1. Microsoft announces Windows Server 2008 R2. Again.
2. Nice writeup on Windows Server licensing
3. Wild SCVMM 2008 R2 rumors and speculation
4. VI Toolkit (for Windows) 1.5 released

I launched VCritical back in September 2008 — hard to believe it’s been a year already.  Thanks to the initial support from top bloggers like Duncan Epping, I was off to a solid start.  I really appreciate all the visits, links, and comments from the global virtualization community.  I was also pleased to add eG Innovations as the first sponsor and may consider adding additional sponsors going forward.

Content

Based on feedback and subscriber/reader metrics, the current blend of technical and competitive articles seems to be about right, so I plan to generally keep doing things the way I have for the foreseeable future.

I welcome any comments on the type of content you would like to see on VCritical — feel free to leave your thoughts below.

Traffic

As the following chart illustrates, I have been fortunate to experience month-over-month growth since inception.

My current policy is to not publicly disclose traffic statistics, but for perspective, each y-axis tick represents multi-thousands of page views.  I use several methods to measure statistics and this WordPress Stats module is the most conservative — it only measures actual blog page views from real users.

Blog Infrastructure

I started out with self-hosted WordPress as the platform for VCritical and have not regretted that decision once.  I did, however, need to change hosting companies twice in the past year.  That was painful, but now I’ve settled on what seems to be a sufficiently robust provider — you get what you pay for.  Thankfully, the cost is more than covered by Google ads and sponsorship.

Thanks

The virtualization space is really starting to heat up — I look forward to the next round.  Stay tuned.

Thanks for reading, subscribing, and following me on Twitter!

Related posts:

1. Vote VCritical — for change
2. Got Gravatar?
3. Thinking about blogging?
4. Not another virtualization blog!

Previously I showed how the architecture behind VMware vSphere clustering is so robust that even a remote virtual machine console remains connected as a VM migrates with VMotion from one host to another.  VMware vSphere is designed from the ground up as an integrated virtualization solution — cleanly abstracting virtual machine operations from the underlying aggregate pool of physical hardware.

Microsoft Hyper-V R2 derives HA clustering capabilities and Live Migration from the additional Windows Failover Clustering layer that can be added to the hypervisor.  With all that integration, you might think a remote console connection would seamlessly fail over along with the VM during a Live Migration.

Sorry, not in this release. Is it an omission, or just a bug?  Time will tell.

Now that System Center Virtual Machine Manger 2008 R2 (SCVMM) PRO tips can be used to automate Live Migration — if it is integrated with System Center Operations Manger 2007 (SCOM), that is — administrators connected to a remote VM console may find themselves suddenly cut off.

See it in action in this short video:

Oh, sure it’s not hard to go back into SCVMM, find the VM, and reconnect — but should you have to do that manually?  VM remote consoles may not be needed for day-to-day administrative tasks, but is that reasonable justification to ignore usability and integration?

Because the Microsoft Virtualization solution is a conglomeration of various components — Windows Server, hypervisor role, Failover Clustering, SCVMM, SCOM, etc. — it is much more difficult to provide a seamless experience.  This remote console issue is really just an indicator.

Related posts:

1. Live Migration “significantly more complicated”
2. SCVMM/PRO Complexity: High
3. vSphere Console Stays Connected During VMotion
4. Choose any two: Hyper-V, HA, Linux

VMware administrators may not have much day-to-day need for a virtual machine console, but when it is needed — it is needed.

You can think of the VM console like a remote KVM for virtual machines  — similar to HP iLO on a physical machine.  Most of the time there are more efficient network-based remote access techniques like RDP or SSH.  However, there are some circumstances — like network misconfiguration — that absolutely require the use of a remote console.

Have you ever wondered what would happen if you were connected to a VM console while it automatically migrated with VMotion to another physical host?  If you are a long-time VMware admin, you would probably expect things to keep on working.  And that is exactly right — the console connection is seamlessly transferred to the new VMware ESX host.  This behavior is especially important when VMware DRS is dynamically moving virtual machines to optimize cluster resources.

Need to check the network settings of a VM?  No problem — you may not even notice if that VM migrates to another host while you are connected.

Here is a quick video of the situation in action:

Does Microsoft Hyper-V and System Center Virtual Machine Manager 2008 handle this scenario the same way?  Not exactly.

Related posts:

1. Hyper-V Console Disconnects During Live Migration
2. VMotion from physical ESX 4 to virtual ESX 4
3. Thanks for all the port groups!
4. Storage VMotion Q&A

The Windows You Know™ formerly assigned letters of the alphabet to storage volumes.  With the introduction of Cluster Shared Volumes (CSV), drive letters are no longer used — relying instead on a 128-bit globally unique identifier (GUID).  In other words, what used to be called the “F drive” is now known as \\?\Volume{d5ad02f4-4e30-11ed-b1db-ca8c6df4064b}\.

While in some contexts the difference is minimal, one obviously affected area is monitoring with System Center Operations Manager 2007 (SCOM), which exposes volume GUIDs in various user interfaces:

In related news, an interesting report recently published by a top medical journal found that those in a relatively new information technology position known as “Hyper-V Administrator” exhibited strongly enhanced memory recall capabilities. This correlation is thought to be a result of the rigorous, albeit unintentional, mental training these personnel undergo in the course of their daily responsibilities.

“It appears to be a classic case of the Von Restorff effect,” said Dr. Sedgwick McCaskey, primary contributor to the research and author of the best-seller Don’t Forget IT.

The report also went on to explain that some “Hyper-V Administrators” have found mnemonic devices to be a great help in their jobs. “Instead of trying to remember all of those random letters and numbers, I sometimes make up songs or funny stories,” said one anonymous participant in the study.

It’s great to see nimble professionals that are able to quickly adapt in the face of change. Especially considering the fact that every new Windows Server 2008 R2 system has a hidden 100 MB partition created automatically during installation by default.  And since that volume has no drive letter, it shows up in SCOM with – you guessed it – the volume GUID:

Ouch.  Pass the Ginkgo.

Just to be clear: three paragraphs of this article are satire; the rest is factual.  If you can’t figure out which is which, feel free to ask.

Related posts:

1. Hands off that CSV!
2. SCVMM/PRO Complexity: High
3. Two thousand?
4. Hyper-V Linux integration components no longer Connected

This is the third article in a series on VMware vSphere thin-provisioned virtual disks.  Now that we’ve covered:

• Full accounting for all provisioned storage space and complete monitoring
• Automatic response to a nearly-full datastore through vCenter Alarms, PowerShell, and Storage VMotion

You may be nearly convinced to start using thin provisioning, but still wondering…

What happens if a datastore fills up?

When a datastore runs out of space, thin-provisioned virtual disks can no longer dynamically grow to accommodate additional storage demand.  When VMware ESX detects this condition, virtual machines in need of additional storage are instantly paused to prevent guest operating systems from failing. Conversely, VMs that that read and write to existing allocated storage blocks will continue running without issue — not all virtual machines will be paused just because a datastore is out of space.

If you ever find yourself in this situation, it’s not hard to fix.  Here is one simple approach, step-by-step:

1. Free up some space by deleting or moving files — ISO images or powered-off VMs would be perfect
2. Resume one of the paused VMs
3. Use Storage VMotion to move the disks for that VM to another datastore
4. Resume the remaining VMs

Watch the procedure in action:

Depending on the size and storage demand of each VM, additional migrations may be needed.  An alternative resolution would be to add additional space to the SAN LUN and grow the VMFS volume.

The Experiment

To simulate a sudden storage demand by the thin-provisioned VMs in the above video, I simply copied a large file from a network share to each Windows Server 2003 VM simultaneously.

For the curious, below is a PowerShell script for the task.  Run it from anywhere — it uses Sysinternals psexec to remotely initiate a file copy on each VM from a network share.

# VM naming convention combines this string with 2-digit number
$vmPrefix = "VM2003e-"
 
# UNC path to a large file that will be copied into each VM
$iso="\\fileserver\ISO\large.iso"
 
# credentials to download above file from \\fileserver
$user="domain\username"
$pass="pa55word"
 
# pass this function a list of numbers
function createVmList ($series) {
    $vmList = @()
 
    foreach ($id in $series) {
        $vmList +=  %{"$vmPrefix{0:00}" -f $id}
    }
    $vmList
}
 
# make sure Sysinternals psexec is in your path
function psexecOnVm ($cmd, $vmList) {
    $vmList | % {Invoke-Expression "psexec.exe \\$_ -d -u $user -p $pass $cmd"}
}
 
psexecOnVM "cmd /c copy /Y $iso c:\" (createVMlist (31..39))

VMware ESX is Resilient

You may have been surprised at how easy it is to recover from a full datastore — without so much as a guest OS reboot.  It’s a testament to the rock-solid architecture behind VMware ESX and VMFS.  No other virtualization platform comes close.  Try for yourself.  See what happens if a group of thin-provisioned Hyper-V virtual machines suddenly run out of storage — it’s not going to be pretty.

Related posts:

1. PowerShell Prevents Datastore Emergencies
2. Responsible Thin Provisioning in VMware vSphere
3. Finding thin-provisioned virtual disks with PowerShell
4. Using SCVMM to attach ISO images to VMware ESX VMs

In my previous post on VMware vSphere thin provisioning, I pointed out the new datastore alarm feature.  You can take advantage of this feature to respond to a sudden storage demand and automatically take action before end users notice.

When triggered, vCenter Server alarm actions allow several options, including the ability to run an arbitrary command such as a VMware PowerCLI PowerShell script.  Please see Carter Shanklin’s in-depth article for more details on how this works — note that he uses a different technique to launch the scripts.

Storage VMotion to the Rescue

When a datastore is about to run out of space, the fastest resolution may be to simply migrate virtual disks to another datastore.   VMware Storage VMotion provides that capability with zero downtime for VMs and no disruption to end users.  Fortunately, PowerCLI can perform this feat with ease, thanks to the Move-VM cmdlet.

Let’s take a look at a functional prototype PowerCLI PowerShell script:

Add-PSSnapin VMware.Vimautomation.Core
Connect-VIServer localhost
 
$vmToMove = get-vm -Datastore $env:VMWARE_ALARM_TARGET_NAME | select-object -first 1
 
$destDS = Get-Datastore | where {$_.FreeSpaceMB -gt 50000 -and $_.Accessible -eq $true} | select-object -first 1
 
if ($destDS) {
	move-vm -VM $vmToMove -Datastore $destDS -RunAsync
}

This script is a proof-of-concept that is not ready for your production environment as it is — it just picks an arbitrary VM from the nearly-full datastore, finds another datastore with at least 50GB free, and moves the VM disks. More comprehensive selection logic and error checking are needed for a critical task like this.

Save your script on the vCenter Server system somewhere, such as C:\scripts\datastore.ps1.

Create the Datastore Alarm

Create a new alarm at an appropriate level in the vCenter hierarchy, such as a datacenter, and configure like this:

On the Triggers tab, add a “Datastore Disk Usage (%)” trigger to alert at a reasonable percentage — I opted for 93.

Run PowerShell Directly from vCenter Server

For whatever reason, PowerShell.exe does not do well when launched directly by another process — it tends to hang instead of exiting when it is finished.  As a workaround, it can be launched from cmd.exe as long as it receives something on standard input.  To do all that, the necessary code looks like this:

"c:\windows\system32\cmd.exe" "/c echo.|powershell.exe -nologo -noprofile -noninteractive c:\scripts\datastore.ps1"

For an alternate approach, take a look at the intermediate batch file solution described by Carter Shanklin in the link above.

On the Actions tab, add a “Run a command” action and supply the appropriate command.  You also need to decide whether to run one time or repeat the action.

Action!

To test the alarm, either fill up the datastore or temporarily lower the alarm threshold.  When the alarm fires, a Storage VMotion should be seen in the vSphere Client:

Note the “Initiated by” column — that’s the machine account for this vCenter Server.  The PowerCLI script is kicked off from vpxd.exe, which is running as LocalSystem.

Additional information is available by looking at the Tasks & Events tab for the datastore.  Here you can see a sample sequence of events, newest on top:

The Last Resort

This automated Storage VMotion recovery alarm is a safety valve that could help you avoid suddenly running out of space on a datastore.  It should not take the place of more proactive storage management, but it sure beats VM downtime.

In case you are wondering: No, you can’t do the same thing with Hyper-V because Hyper-V does not have zero-downtime Storage VMotion.  Just another reason to choose VMware vSphere — as if you needed another reason.

Have you used vCenter alarms to automate any recovery processes in your environment?

Related posts:

1. Easy recovery from a full VMware ESX datastore
2. Finding thin-provisioned virtual disks with PowerShell
3. VI Toolkit (for Windows) 1.5 and the PowerShell prompt
4. Responsible Thin Provisioning in VMware vSphere

A cost-saving feature introduced in VMware vSphere 4 is fully supported thin-provisioned virtual disks. Thin-provisioning decreases demand for SAN storage space by permitting virtual disks to consume just the space they actually use — and grow as needed — instead of pre-allocating all space up front.

What’s new is not necessarily the technology — it’s the management.  In fact, veteran VMware ESX admins have been creating thin provisioned virtual disks for years — for controlled scenarios — by way of the vmkfstools command.

Before vSphere and ESX 4, however, thin disks came with risk — there was no simple way of accounting for the overcommitted storage on each LUN.  Even with multiple gigabytes of free space, a small gang of thin-provisioned virtual machines could quickly quickly grow to exceed datastore capacity during a sudden demand spike.

Complete Storage Accounting

Now in vSphere 4 there is a new element in the capacity section of the datastore summary tab that shows total provisioned space — the maximum potential growth of all virtual machines if thin provisioned disks were fully utilized:

Virtual machines with snapshots have the potential of consuming even more datastore space, so vSphere accounts for this condition, too.  Take a look at this VM Summary tab, where the total provisioned storage includes:

• Virtual hard disk (40GB)
• Snapshot (another 40GB potential, worst-case)
• VM swap file (1GB — sized according to RAM in theVM)

As you can see, a VM with a 40GB virtual disk can actually consume up to 81GB of space on your SAN because of a forgotten snapshot!  Use the  snapshot alarm to stay in control.  And don’t forget that VMware vSphere snapshots are perfectly suitable for production.

New Datastore Alarms

New alarms in vSphere prevent out-of-space surprises.  Administrators can monitor not only the free space on a datastore, but also the percentage overallocated — making it easy to adhere to policies concerning thin provisioning aggressiveness in your environment.

Flexible Virtual Disk Re-configuration

When creating a new VM, opting for thin-provisioned disks is as easy as checking a box.  If you change your mind later, you don’t have to start over — during a Storage VMotion operation, administrators can opt to change the VM disk format on the fly (see below).  It is also possible to inflate thin disks into thick via a new menu in the Datastore Browser.

Danger!

You’ve probably heard the now-famous quote by Tom Bittman from Gartner:

Virtualization without good management is more dangerous than not using virtualization in the first place.

That goes double for thin-provisioned virtual disks.  Without comprehensive accounting and monitoring in place, your virtual infrastructure may be heading for disaster.  This level of insight is only available with VMware vSphere 4 — and it’s built right into the platform.

What about Microsoft virtualization?  Hyper-V R2 thin provisioning — known as “dynamic disks” — is not a best practice.  Perhaps due to the lack of accounting and monitoring of storage overcommitment — especially critical now with Cluster Shared Volumes and multiple VMs per LUN.

Are you using vSphere thin provisioning?

Related posts:

1. Easy recovery from a full VMware ESX datastore
2. Finding thin-provisioned virtual disks with PowerShell
3. VMware vSphere 4 has a Snapshot Alarm
4. PowerShell Prevents Datastore Emergencies

This article is part of a series on Incoming Google Traffic (IGT).

It is very clear that the VMware community loves the ability to run virtual instances of VMware ESX 4 — this has been the most popular VCritical article ever.  Take a look at these keywords that account for hundreds of searches over the past month:

esx-on-esx
esx on esx
esx inside esx
esx in esx
install esx on esx 4
how to virtualize esx

Amazingly, VCritical is currently the number one result when searching Google for “ESX 4″.  I cringe at what the Bing results might look like, though.

Related posts:

1. VMotion from physical ESX 4 to virtual ESX 4
2. IGT Part 6: Cluster invari-what ID?
3. MS Virtual Server VM found to perform poorly while running inside an ESX VM
4. IGT Part 5: Hyper-V snapshots are not gone until the VM is powered off

A VMware ESX 4 (and ESXi 4) virtual machine can have up to 8 virtual CPUs.  Even though the underlying physical hardware has multiple cores per socket, each virtual CPU presented to the guest OS appears as a separate socket.

This normally does not matter because guest operating systems treat cores as CPUs anyway.  However, there is one significant area that is affected — licensing.  Some operating systems and applications are licensed by physical CPU, regardless of the core count.  If this license is enforced only by a legal agreement, and not by technology, then there is still no problem — configure a four vCPU VM on a system with a quad-core processor and you are in the clear.

The issue arises when software enforces the CPU socket limitation.  One good example of this is an operating system like Windows XP, which supports a maximum of two sockets — there is no way to use four cores if each is presented as a socket even though such an action would still be allowed within the license agreement.

To address this, vSphere 4 introduced a new undocumented VM configuration option called cpuid.coresPerSocket.

If you have experimented with the feature — Duncan Epping already wrote about it a while back — you may have wondered if there is a method to see how many cores and sockets a Windows system has.  After all, Device Manager simply shows each core as a CPU.

One of the more recent Sysinternals tools from Mark Russinovich does that very thing.

Acquiring Sysinternals, and gaining Mark Russinovich, was an exceptionally wise move by Microsoft.  So far, so good — knock on wood — Microsoft has not done anything ridiculous like restricting the tools to be part of a Software Assurance (SA) incentive.

The Sysinternals utility is called Coreinfo.  In these examples I am using it with two flags to show cores and sockets:

coreinfo -c -s

Here is a default VMware ESX 4 VM with 8 vCPUs:

After changing cpuid.coresPerSocket to 8, like so:

The guest OS sees a single socket, 8-core VM:

Setting the value to 4 divides the 8 cores into two sockets like this:

That last setting is the most legitimate for licensing purposes, as the underlying hardware here is a dual quad-core.

Please remember that this feature is currently not supported by VMware GSS.  If you need to use it in production, please contact your VMware account team so that your use case can be taken into account when planning future releases.

Related posts:

1. Upgrading VMs to vSphere virtual hardware
2. Easy recovery from a full VMware ESX datastore
3. New VMware ESXi Management Kit

Have you noticed the avatar images next to blog comments that make each author more easily identifiable?

On WordPress blogs, these images are generally made possible through a service called Gravatar.  Most of the Top 20 Blogs at vSphere Land use WordPress — if you plan on commenting on the VM blogs, why not take a moment to set up a Gravatar for yourself?

I’m pointing this out in case folks are under the impression that they must have a special account on VCritical to display an avatar next to their comments — it’s all handled by Gravatar.

Do you have a Gravatar?

Related posts:

1. Thinking about blogging?

Cluster Shared Volumes (CSV) is the new Microsoft Hyper-V R2 feature that allows virtual machines to be saved as plain text files and then imported into a spreadsheet or database for editing.  Wait… that’s a different kind of CSV.

Actually, CSV is a layer on top of NTFS shared storage that provides some of the functionality of a cluster filesystem — multiple hosts can access a single LUN simultaneously.  No more one VM per LUN jokes, please.

Unlike VMware VMFS, CSV relies on a single coordinator node for all metadata updates to the LUN — interesting.

The Secret Sauce

It turns out that CSV is a delicate feature and files on such a volume should never be managed directly.  In an effort to protect Hyper-V administrators from themselves, Microsoft has taken an interesting non-technical approach to preventing CSV misuse.

After installing Windows Server 2008 R2, configuring storage LUNs, enabling Failover Clustering, and adding cluster nodes, CSV must be enabled on the cluster.  In the process of doing so, a dialog box is presented that looks much like an end-user license agreement:

The text inside the box states the following:

The Cluster Shared Volumes feature is only supported for use with Windows Server 2008 R2 Hyper-V role.  Creation, reproduction and storage of files on Cluster Shared Volumes that were not created by the Hyper-V role, including any user or application data stored under the ClusterStorage directory of the system drive on every node, are not supported and may result in unpredictable behavior, including data corruption or data loss on these shared volumes.

In order to proceed, administrators much check a box indicating that they have read the notice.  Note: If you are the one that checked the box in your organization, please be sure to pass the warning on to your coworkers.

Backups?  Someday.

What about backup (”reproduction”) of virtual machines stored on the CSV LUNs?  You cannot simply take backups of VM files like you would on a normal volume. In fact, if you attempt to back up any files on a Cluster Shared Volume by using the native Windows Server Backup tool, the following error is thrown:

Which leaves the Hyper-V administrator with just one option at this point: in-guest backups only.  Eventually, System Center Data Protection Manager will support backing up Hyper-V VMs, but not right now.

Check out this hilarious ZDNet article bashing VMFS — they didn’t happen to mention any of these CSV advantages.

Is this an enterprise-class solution that is ready for your production workloads today?

Related posts:

1. Hyper-V Administrators Exhibit Advanced Memory Skills
2. Storage vendors unanimously applaud SCVMM innovation
3. What would things be like without VMFS?
4. IGT Part 2: SAN Misers

Microsoft lists a System Center Virtual Machine Manager 2008 Client License on the pricing and licensing page.

[Updated information based on reader feedback follows]

This new SCVMM Client License is exclusively for VDI environments — Hyper-V running desktop VMs (only).  There is a VDI license suite that includes SCCM/SCOM Server Management licenses and SCVMM Client Licenses.  Perhaps since SCVMM only manages VMs, Microsoft decided to call that one a Client license.

In summary: this is not an additional license, nor is it an alternative way to license SCVMM for server workloads (there are already at least three ways to do that…).

Related posts:

1. Wanna see a million ESX licenses?
2. Hello, SCVMM 2008 R2
3. Wild SCVMM 2008 R2 rumors and speculation
4. What’s New (and What’s Vapor) in SCVMM 2008