Archive for July, 2009

Again, I was having a bit of day off from work – and didn’t really succeed. I think I might be a work-a-holic! I learned two things today. Firstly:

Hot-add might not be so hot after all:

In my book on vSphere4 I covered hot-add of CPU/RAM to the VM – knowing that that support for this featue was very much dependant on the Guest Operating System (aka Windows & Linux) – I set up a VM using the very latest version of Windows and the highest level – that is to say Windows 2008 64-bit Enterprise (OK, I didn’t have an .ISO of DataCenter Edition). I had pretty good experience.

But it has come to my attention via David Davis and Jason Boche (behind Jason’s name is link that takes you to his testing on hot-add) that hot-add might not be so hot after all. You see I just check the 2nd vCPU was just present in Task Manager. I forgot to check Device Manager and to see if cycles are actually scheduled on the new vCPU. According to Jason Boche – they are not – and in fact it seems like all the various flavours of Windows will require some kind of reboot.

Of the course, the really funny thing about hot-add is… wait for this… it’s not enabled by default AND you have to power down the VM to enable hot-add (Ahem…)… Anyway, I think there maybe a good reason why VMware chose not to enable hot-add as default. It’s incompatiable with VMware Fault Tolerence.

Anyway, not content to just take Jason’s word for it (not that I don’t trust/respect the guy) I want to give this hot-add business another try, and see what gives…

Installing EMC PowerPath for VMware:

For the last couple of days I’ve been playing with EMC PowerPath for VMware. As you might know ESX4 vStorage allows for storage vendors to add their own PSPs (Path-Selection-Plug-ins) and extend the functionality of the ESX beyond the standard PSP supplied by VMware (MRU, Fixed, Round-Robin). I was inspired by Chad Sakac recent blog post and video.

http://virtualgeek.typepad.com/virtual_geek/2009/07/howto-download-install-and-license-emc-powerpathve.html

The nice thing for someone like me – is that I was able to download the PowerPath software for free (the same couldn’t be said about the PDFs – hint-hint!) and Chad was kind of enough to allow me some NFRs that last a year. It does appear to be the case that theres some kind of “conflict” between storage views and PowerPath, Chad’s told me that VMware are working on patch to fix this issue – nothing major sounds like just a bit wonky xml.

Anyway, this is likely to end up in my book on vSphere4, but in case it doesn’t – I wrote a little PDF to explain how it is done. (or you could watch Chad’s videos)

Following on from this weeks series – today I learned 3 new things – and I wasn’t even supposed to be working! After getting up at 4.00am to take my partner and her daughter to the airport, I thought I’d take it easy by going back to bed, and then later catching up with some telly on me Sky+ box… But even just by doing as little as that I learned a couple of tidbits – mainly by keeping a watchful eye on my email and the forums.

HA Split Brain (Split Brian?):

Split brain is HA situation where an ESX host becomes “orphaned” from the rest of the cluster because its primary service console network has failed. As you might know the COS network is used in the process of checking if an ESX host has suffered an untimely demise. If you fail to protect the COS network by giving vSwitch0 two NICs or by adding a 2nd COS network to say your VMotion switch, underdesired consequences can occour. Anyway, the time for detecting split brain used to be 15 seconds, for some reason this has changed to 12 seconds. I’m not 100% why, or if in fact the underlying value has changed – or that VMware has merely corrected its own documentation. You see its possible to get split brain in Vi3.5 happening if the network goes down for more than 12 seconds, but comes back up on the 13th, 14th or 15th second. I guess I will have to do some research on this one. Of course, the duration can be changed – and split brain is trival matter if you take the neccessary network redundency steps…

USB “Support”:

This is an odd one. You can in a VM on ESX4 add a USB Controller. The VM must be hardware level 7 for this to be the problem. The trouble is – it doesn’t do anything. And no amount of adding or loading drivers at the COS will allow you to plug a USB device into an ESX host, and have it appear in the VM. You might ask if it doesn’t work does that mean its bug. No, its not a bug. It’s just there, but you can’t owt with it. NICE! The best we can say at the moment is that its been put there for some future technology which may or may equally may not become available. As ever I personally feel some USB-over-IP device is the best way to go.

Fix Broken OVF Exports:

This happened to me twice now. I power down a perfectly functioning virtual appliance (in my case the UDA 2.0 which is currently in beta) and export it to be OVF Template using the vCenter4. To check the export has been successful I re-import it only to get to 100% complete, but end with an unpleasant error message “Failed to deploy OVF package: The remote server returned an error: (500) Internal Server error”

The last time this happened to me I twittered about it – finally some helpful guy who follows me on twitter helped me fixed the problem. [I only wish I could remember his name because he deserves the credit for this fix!] It’s not clear precisely why it happens, but appears that occasionally the Export function of vCenter4 creates an OVF file that misreports the size of the virtual disk. A quick edit to the OVF file is all that is needed. So below is my “bad” OVF File:

<?xml version=”1.0″ encoding=”UTF-8″?>

<!– Generated by VMware VirtualCenter Server, User: Administrator, UTC time: 2009-07-21T13:53:10.761609Z –>

<Envelope vmw:buildId=”build-162856″ xmlns=”http://schemas.dmtf.org/ovf/envelope/1″ xmlns:cim=”http://schemas.dmtf.org/wbem/wscim/1/common” xmlns:ovf=”http://schemas.dmtf.org/ovf/envelope/1″ xmlns:rasd=”http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData” xmlns:vmw=”http://www.vmware.com/schema/ovf” xmlns:vssd=”http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”>

<References>
<File ovf:href=”uda20-beta-build5-disk1.vmdk” ovf:id=”file1″ ovf:size=”207287808″ />
</References>

<DiskSection>
<Info>Virtual disk information</Info>
<Disk ovf:capacity=”1” ovf:capacityAllocationUnits=”byte * 2^30” ovf:diskId=”vmdisk1″ ovf:fileRef=”file1″ ovf:format=”http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized” />
</DiskSection>

The bad part is highlighted in bold. To correct this I found out the actual size of the -flat.vmdk by using ls at the Service Console:

I then modify the .OVF file accordingly:

<?xml version=”1.0″ encoding=”UTF-8″?>

<!– Generated by VMware VirtualCenter Server, User: Administrator, UTC time: 2009-07-21T13:53:10.761609Z –>

<Envelope vmw:buildId=”build-162856″ xmlns=”http://schemas.dmtf.org/ovf/envelope/1″ xmlns:cim=”http://schemas.dmtf.org/wbem/wscim/1/common” xmlns:ovf=”http://schemas.dmtf.org/ovf/envelope/1″ xmlns:rasd=”http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData” xmlns:vmw=”http://www.vmware.com/schema/ovf” xmlns:vssd=”http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”>

<References>

<File ovf:href=”uda20-beta-build5-disk1.vmdk” ovf:id=”file1″ ovf:size=”207287808″ />

</References>

<DiskSection>

<Info>Virtual disk information</Info>

<Disk ovf:capacity=”2040109056” ovf:capacityAllocationUnits=”byte” ovf:diskId=”vmdisk1″ ovf:fileRef=”file1″ ovf:format=”http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized” />
</DiskSection>

Er, that’s in hits not in pounds or dollars! Almost (but not quite) without me noticing it RTFM crossed over into having more the 1M hits.

http://www.sitemeter.com/?a=stats&s=sm3icm

I first started using sitemeter.com back in the very late 90′s when RTFM was just a bit of free space that I got with a now defunct ISP called freeserve.co.uk (remember them?). So its taken sometime to get to this point. Of course, the biggest thanks goes to those people in the VMware Forums who first started supporting me and my work by downloading my free PDF guides. Anyway, here’s to the next 1 million!

Well, carrying on with my theme of reporting the new stuff I learn each day. Here’s what I learned in the last couple of hours.

ESX4i doesn’t have Web-Access:

Not that this really matters too much… But ESX4i doesn’t have web-access front end – where you can do VM Management – but ESX4 does. To be honest if your using this functionality its best delivered from the vCenter web-pages because, well, its Active Directory “aware” in the first place. Interestingly, in the place of the link for the web-access page – VMware has put download links to the vSphere Remote Command Line

Roles and Help from VMware:

Normally, if I create a new role – I use an existing sample – clone it and then modify it from that clone. In fact I’ve never ever create a new role from scratch. In vSphere4 if you create a new role and select a couple of privileges the system will offer suggestions of additional privileges that will be probably required for the whole task to be completed. Personally, I will be sticking with my cloning method – but its nice to know. What I would like to see VMware do is take this idea a step futher and make some kind of “delegation wizard” (I’m borrowing a term for Microsoft AD) to guide people thru the steps of some of the main permissions tasks – something that would really, really benefit VMware SRM customers!

Standard & Distributed Portgroups – Drag & Drop:

Say you want to move a whole bunch of VMs from portgroup to another – well in the past you would have edit each VM (tedious) or used some whizzy bit of PowerShell like this:

get-vm | get-networkadapter | sort-object -property “NetworkName” | where {‘VLAN10′ -contains $_.NetworkName} | Set-NetworkAdapter -NetworkName ‘VLAN11′

I still like that. But what I learn today is if your using vCenter4, you can in the “Network View” drag-and-drop many VMs to their correct portgroup. You can do a similiar kind of task using the “migrate” wizard from the DvSwitch…

Ah, following on from that previous blog post – last week I learned that SVMotion in vSphere4 does NOT need VMotion to be configured first! That’s right, you can do SVMotion with having a VMotion switch enabled. In the screen grab below – you can see that I haven’t got VMotion enabled – its dimmed in the dialog box – whilst trying to move a VM. You cannot move a VMs files and move its ESX host location at the same time – because the VM is powered on (do this need you would need to do a cold migration). BUT, SVMotion moving the VMs file whilst the VM is powered is available. There isn’t a terrific benefit to knowing this little tidbit – accept it demonstrates that SVMotion is core feature of vSphere4 with no direct dependencies to the VMotion part of the product. So if SVMotion is built-in and for free – I wonder how long VMotion will be as well (hee-hee-hee)

It may as some suprise to the readers of my blog, that I don’t know everything about VMware (sic). Only last week I was on the VMware Trainers forums asking a question – and someone said “I’m surprised you don’t know that, Mike – but it at least it makes you more human in my eyes”. Quite what it means to be “more human” is debatable – perhaps I’m extrahuman! In fact one of the occupational hazards of being the self-proclaimed “Mr RTFM” is the impish delight that others take in correcting my mistakes and misunderstandings. So it goes.

Anyway, what I like to do during my training course is admit that EVERY DAY I learn something new about VMware which I’ve never come across. It goes to remind everyone that old dogs like me do learn new tricks everyday. So with that spirit in mind I want to try every week tell you what I learned about VMware that week. You know the kinda of thing like tidbits that make you go “ahhhhh, I didn’t know that”. Of course, you free to comment “Jesus, Mike! Didn’t you know that – its on page 28 of the admin guide…”

RDMs and the 2TB Limit:

OK, I’m ashamed to say that I didn’t know that RDMs like virtual disks (.vmdks) are limited to being 2TB in size. I ass-um-ed that ESX4 had smashed thru this limitation. But no, it still there. To be more accurate the limitation is 2TB – 512B = 4294967295 blocks (512B per block). The limitation doesn’t come from VMFS but from the fact that VMware Logical Volume Manager still uses CHS (cyclinders-heads-sectors as the method enumerating LUN size, rather than GPT (GUID Partition Table). Despite the fact that the GPS method is supported in Windows (since Windows 2003 SP1), it is not supported by VMware. Of course there work-arounds if you do need >2TB but the restriction remains…

Some Nice Bits of PowerShell:

Whilst helping some people on the forums I came across some sweet piece of powershell. I’m collecting sample of common tasks – so I can call them up from a personal library in my lab environment. So here’s what I found last week:

Find VMs on Local Storage:

Get-Datastore |where {$_.Name -match “store|local|storage”} |Get-VM |Get-HardDisk |select Filename |Export-Csv c:\LocalVMs.csv

Force an ESX into Maintenance Mode:

Get-VMHost -Name esx1.rtfm-ed.co.uk | Set-VMHost -State maintenance

List all VMs with their IP Address:

Get-VM | select name, @{Name=”IP”; expression={foreach($nic in (Get-View $_.ID).guest.net) {$nic.ipAddress}}}

vApps on a stand-alone ESX host:

In case you don’t know vApps in vSphere4 is glorified resource pool – which allows you to gather a bunch of related VMs into a single object (called the vApp). From there you can set resource settings (like resource pool) but also do funky stuff like start-up/power-down orders and different methods of allocating IP address (say if you were an ISV delivery a multi-tier application in the .OVF format). NOW, if you create a vApps on a stand-alone host, and then subsequently try to add to a DRS enabled cluster, you will have a problem (see graphic below) as the vApp is destroyed during the process. Moral of the story? Create VMware Cluster first, then create vApps…

Manual invention required in Update Manager if vCenter4/VUM run in a VM:

I’m big advocate of running vCenter/VUM and other VMware Infrastructure components in a virtual machine. In fact I’m often horrified to discover that people physicalize those roles. Anyway, I don’t want to get into that debate – instead flag up anolmaly in VUM in vCenter4. If you go to do a remediate on the ESX host which is running the virtualized vCenter4/VUM instance you will see the error below. To resolve it you must manually move the VMs to different ESX host in the cluster. Normally, I run my virtual infrastructure components in a different ESX host (esx4) in a different environment to esx1,esx2 and esx3 but last week to keep everything up and running and patched to the SAME level. I joined ESX4 to the DRS enabled cluster and so the warning:

Over on VIOPS, Bob Stevens of VMware has put a best practises for performance together – four chapters covering:

• Chapter 1, “Hardware for Use with VMware vSphere”
• Chapter 2, “ESX and Virtual Machines”
• Chapter 3, “Guest Operating Systems”
• Chapter 4, “Virtual Infrastructure Management”

http://viops.vmware.com/home/docs/DOC-1571

Some more information has come out how de-duplication has been impleamented in the new vDR appliance:

http://blogs.vmware.com/uptime/2009/07/vmware-data-recovery-taking-advantage-of-vsphere-4.html

Its looks like I will be going to VMworld this year in San Fran. So I will be RSPV on this “underground” meeting that’s been cooked up over here:

http://party.vmunderground.com/vmworld-2009-sunday-night-extravaganza

VMware updated a product this week, and released two new ones.

The release Lab Manager 4.0 which is the first high-level management product to be compatiable with vSphere4. That’s a good sign as many people (wrongly in my opinon) were belly aching that no every single high-level management product was compatiable with the new platform. New to Lab Manager 4.0 is cross-network fencing. A network feature that’s been long discussed since VMworld TSX in Canne in 2007. Basically, its like a PVLAN impleament – in that you can have VMs with the same IP/same name on the same network without conflicting with each other. I’m really interested in this feature – because it might be away forward for DR solutions where re-IPing VMs and making sure there not unintend network conflicts during the test of recovery plans… It also seems that a prediction many people (including myself) made about 2 years ago has come true. That “Stage Manager” and “Lab Manager” have jumped into bed, got each other pregnant and had a shot-gun wedding. Yep, they’ve come together as one product. It was always quite tenious argument separating the two, and even harden VMware Vets like me stuggled to understand why there was a Lab Manager AND a Stage Manager.

I remember the Register asking – what was next – a manager manager to manage all your managers? Sounds like story about the NHS…!

http://www.vmware.com/products/labmanager/

VMware also released vCenter Chargeback. Despite the vCenter monikor it is not included in any of the standard SKU’s for vSphere (so its not in Foundation, Standard, Advanced, Enterprise or even the much derrided Enterprise+), but as separate downloadable product. Priced at nearly $750 per socket one of the main costs of chargeback will be the chargeback software. I DO LOVE THAT IRONY!

To be honest I’ve come a lot of US software vendors adding chargeback modules to their software – Citrix did to their Presentation Server product. I dunno why, but in EMEA when you chat chargeback to customers/clients/student they just seem to glaze over. It just doesn’t seem like burning issue to folks on this side of the pond. Maybe I’m wrong – and not talking to the right people. Perhaps it is a US thang. You tell me. Do you rub your hands in glee with the thought of raising internal invoices for VMs you folks create?

http://www.vmware.com/products/vcenter-chargeback/

The other product is to me more intriguing – vCenter AppSpeed. Again its not included in any of the standard SKUs. I love it when vendors bring out a new performance optimiser or analyser – some how you know the marketing folks are never gonna call the product – AppSlowness, App Little Bit Quicker But Nothing Your Users Would Notice or App Monitoring Tool That Duplicates Data That Should Be In the Core Product. Anyway, you probably assuming from my comments that I don’t rate this thing – in truth I have little/no idea of what does and how it works beyond the normal guff that surrounds a product launch. But I am intrigued…

http://www.vmware.com/products/vcenter-appspeed/