Archive for the ‘vCenter’ Category

One of the most hotly contested issues debated on my training courses is whether you should or should not virtualize vCenter. Some folks are convinced that it should remain physicalized citing somewhat out-date recommendation to this affect back in VMWorld 2005. Well, there’s nothing like keeping you best practice policies up-to-date is there. As for me I’ve always been strong advocate of virtualizing vCenter – and I often pretend I don’t know how to install Windows to physical box to remind people – that VMware’s management server – benefits from virtualization just like any other workload. However, its not without caveats, not least an interesting feature of vSphere which now makes virtualizing vCenter trickier than it was in previous releases… It does seem to be the case that people have an ideological opinon on this before they have even thought of the technical merits… Read on McDuff

Inspired by an exchange between two bloggers – Scott Lowe (EMC) & Steve Chambers (Cisco) – I’ve stepped in to weigh in the balance – the desire for greater consolidation ratios against the dangers of putting too many eggs in one basket. Every generation seems to demand a new panacea – virtualization; the cloud – while the same old problems caused by what I call “stupid IT” persist.

The article is in two parts on the techtarget website:

How to simplify your IT and stop being stupid

Virtualisation is not a panacea and neither is the cloud

Here’s one I came up against in the last week and half. I’ve been playing with Win7-64-bit as part of my work on the new View 4.5 beta program. I’m beginning the move away from Windows 2003 Enteprize 32-bit as my standard – and everything I do is going over to Windows 2008 R2 64-bit and Windows 7 – 64-bit. Basically, its good bye to everything 32-bit in my lab environment – and a big hello to needing more disk space and memory.

Anyway, a buddy of mine from EMC ping me an email saying his Guest Customizations weren’t joining the domain for pooled desktops – he was a bit ahead of me. But sure enough when I got there. I had the same problem. I’ve seen this problem in both Windows 2008 R2 and Windows 7. Basically, if you complete the desktop like this – it doesn’t work. The guest gets renamed, and its IP set – but it just doesn’t join the domain.

However, if you complete the dialog in this way – it works just dandy

I have a feeling this an undocumented or obscurely documented version of sysprep… but I’ve not found any source that confirms or denies if this a bug or feature…

This week I sat in on a TTT (Train-The-Trainer) session for a new VMware course called “Troubleshooting”. It’s part of my job as in instructor that I must attend these events to be able to deliver a particular course. So you know this new course has some hands on labs where you do some advanced configuring/reporting using various CLI tools, and it also contains PowerCLI scripts that the instructor runs which screws up vSphere4 environment, and students run about have to try and fix the problems. Its something I call “Instructor Revenge” – what it means is if you are an instructor and you have had “a***” on your previous course you can get your own back! What precisely is an “a***” on a course? An a*** on a course displays these annoying habits:

• The a*** tells the instructor how to do his job
• What ever you say, the a*** always contradicts it
• The a*** rambles on and on about problems he had, and how he resolved them – isn’t he just great!
• The a*** introduces himself by saying that he really shouldn’t be on this course, because he knows ALL this stuff already – and he is only doing it because of certification reasons
• The a*** interrupts the instructor every 10 seconds with a question – generally the very thing your about to say (because he thinks the instructor is too dumb to know it already)
• The a*** thinks instructor knows nothing about the “real” world
• The a*** only knows 20% of the product, but thinks he knows 100%

Anyway, you get the picture – fortunately most students are really nice and helpful – and a***s are few and far between. But when you get them, the troubleshooting course will be sweet revenge, because you can cripple his environment beyond repair and watch him struggle for four days. Revenge is dish best served cold, my friend.

Well, yesterday last week I had a real problem. Not a very serious one. So I left until today this week to resolve it. This is live blogging, by which I mean what is about to unfold is a journalistic account of how I fixed (or didn’t!) the problem. The idea of this to show how you troubleshoot a real problem – and the only way to do that realistically is to have one you haven’t seen before. I hope to learn as much as about this process and you do. So join me for this ride. The real thing I want to get across is the not the problem, but how I handle it. I might not handle it very well. Sorry about that. I do the best I can with the limited skills I have. But perhaps you can learn from my mistakes as well as my successes.

Let me start with a overview of what the problem is, and what I was doing when it happened – and what I think caused it.

(more…)

In this article, we eschew the normal blue-sky 2010 technology predictions for something a bit more everyday that will affect your daily virtual life.

Scale up, up, up and away
Firstly, it’s no surprise that by mid-year there is likely to be a major rerelease of vSphere4 with a strong emphasis on increased scalability. Building on top of vSphere4′s current scalability I wouldn’t be surprised to see the number of vCPUs a single ESX host can support go beyond the 128 core range. I think it’s likely that by the end of 2010 or the beginning of 2011 we will be looking at more than 8 vCPUs to a VM – with VMware pushing the amount of RAM per-VM into the 512 GB to 1 TB range and the ESX host supporting 1 TB or 2 TB of physical RAM.

Read on…

Following from my popular (according to me!) blog post theme – of “what I learned this week…” a irregular round-up of stuff that seems really obivious – but actually has escaped me.

vCenter DB Retention Policy:

A couple of weeks ago I made a point about how the vCenter DB grows in size day-by-day and week-by-week – and that wasn’t away to control it size or FIFO its data. I turned out I was wrong. If you look in the vCenter Server Settings. In vCenter 4.0 there is actually something called the retention policy.

What I was hoping for was the ability to FIFO the performance data. As far as I know the only way to clear performance data is with SQL Transact commands to delete the redundant data.

VMware Tools won’t upgrade:

I’ve a couple of VMs who’s VMware Tools won’t upgrade. I think it maybe because the uninstall data was deleted – erm, by me. Opps. Anyway, I think I might have found a work around in this KB article.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001354

ESXi lacks RPMs for VMware Tools for Redhat Linux 64-bit:

This is little bit of an obscure one – and it was drawn to my attention by one of my students. If you create a Red Hat Linux 64-bit (RHEL5) on an ESX ‘Classic’ host you will find you have both the ability to install VMware Tools with either an .RPM or extract it from a .TGZ file. NOW, if you try the SAME thing on ESXi host you will find the RPM version isn’t there. Presumably this is done by VMware to reduce the footprint of ESXi.

Find out if your virtual disks are the right type for VMware Fault Tolerance:

This one came thru twitter. Specifically from William Lam. Thanks William this is great find. William has a great resource of scripts on his website:

http://www.engr.ucsb.edu/~duonglt/vmware/

My previous method of finding this out was pretty lame. It involved using vmkfstools and vmkernel log files – and looking at very obscure values therein. This is MUCH easier; MUCH clearer, and MUCH simpler.

Why is this important. Well, let say your VM has the ordinary thick type of disk (technical term is zeroedthick) or you have used the thin type – AND these disks are quite large (let say 2TB). To convert these disks the Fault Tolerance format (eagerzeroedthick format) requires a power off of the VM, and it can take a long time. That’s got to be factored into your maintenance window.

Of course none of this matter if the vSphere Client/vCenter actually showed the correct format. Unfortunately, both the zeroedthick and eagerzeroedthick format are both referred to as “thick” in the GUI.

Here’s how you do it.

Just cat & grep the last vmware.log file of the VM which you can see in the directory where the VM is located. So here’s a VM with three types of disk. Thin (scsi0:0), EagerZeroedthick (scsi0:1) and Thick (scsi0:2). In the screen grab you can see scsi(0:0) is thin

If you then PuTTy to a ESX host, and the type the following command:

[root@esx1 mail03]# cat vmware-1.log  | grep  ‘FT enable’

This will produce the output of:

Dec 02 00:38:08.505: vmx| VM has thin disk scsi0:0; FT enable will be disallowed
Dec 02 00:38:08.507: vmx| VM has zeroedthick disk scsi0:2; FT enable will be disallowed

Alternatively, if you use

[root@esx1 mail03]# cat vmware-1.log  | grep  ‘scsi0:0′

[root@esx1 mail03]# cat vmware-1.log  | grep  ‘scsi0:1′

[root@esx1 mail03]# cat vmware-1.log  | grep  ‘scsi0:2′

This will print out much detailed information. The difference between each output is the “allocation type” field. These are are as follows:

allocation type: 0 – EagerZeroedthick Virtual Disk

allocation type: 1 – Zeroedthick Disk (Default)

allocation type: 2 – Thin Virtual Disk

This week I was asked in an email the following question:

[By the way I hate WordPress - even when I put paragraph returns in this post it ****ing well removes them!]

Hope you been doing well. I have been busy, busy, busy.

Got a question for you. Preparing to start a new vSphere farm and trying to decide do I use stand switches or start with the DVS. Especially when I see this: http://www.ntpro.nl/blog/archives/1283-vSphere-DvSwitch-caveats-and-best-practices!.html So the main question is do we think this is mature enough to go forward with yet? Can I use Host profiles with DVS safely?

Just appreciate your thoughts.

Apologises for my tardiness in getting back to you. Snowed under. You get the picture. So I’ve written you this lovely long email to make up for that. Gee, this could be blog post! [How ironic - it did become a blogpost!]

A couple of things come to mind. Firstly, Eric’s blog post – I do respect Eric’s blog (and others). Things have gotten so big in VMware – that it would impossible to know of all the known/unknown issues – after all no man is an island, and no-one ever see’s all the possible configurations and screw-ups….

BUT. (you could tell that was coming). Untimely, ripping an ESX host out of vCenter (right-click disconnect/right-click remove). Isn’t the recognised procedure. Yes, I know tardy/slapdash admins might have done this in vCenter1/2 but isn’t something that’s recommended or recognised as the right way. It should always be regarded as the LAST resort.

So if people first removed ESX from the DVS, before removing the ESX host this problem would NOT happen. Basically, what Eric is saying is if you don’t step A followed by Step B followed by Step C then problems can occur. But problems will always occur if you not aware of relationships.

So Eric post would not put me off using DVS for Service Console/Management Network traffic or vmkernel traffic for that matter. I think there’s tendency when people read blog post about new technology that we focus on the bad news, rather than the good. Also people forget (rather conveniently) about what was awful about the previous technology (in this case Standard vSwitches) which they moaned and complained about before the advent of the new. So there’s tendency to re-write the past as some golden period before all this new stuff came about….

I’ve used host profiles with DvS without a problem. For me the bigger question is not whether they can be “depended on” or “safely” used in production- but the reality of what they are like to use on daily basis…. Do they make your admin/configuration any easier. Whilst I love DvSwitches, I’m not at all armoured of Host Profiles. Neither are helped by being shackled to the most expensive SKU of vSphere you can buy!

I like DvS… I especially like them for virtual machine networking. But I see them as being less useful for Management and Vmkernel Ports (not bad, just less useful) because you still have to configure the IP configuration of the ESX host.

Of course, VMware’s answer to that would be to use host profiles. For me – I look at the different ways I might get that IP configuration on to the ESX host. I know the more I manually have to do stuff, the more errors I’m likely to make thru fat-finger-syndrome. Also, If I have to hand the build process to someone less experienced I want to automate as much possible to prevent accidental errors…. This is also tempered by what type of ESX the customer is using. If they using ESXi it rules out kickstart %post scripting and using my UDA appliance…

So lets say I’m doing bulk admin. Building 100′s of ESX hosts, do I really want to sit there apply the host profile to each and every host. As I applied the host profile to N host, it would stall as it asked for my management, VMotion, iSCSI, FT logging, HA Address IP addresses. If you think about the number of IP’s you need for an ESX box (if you follow to the recommendations to letter you end up with quite a lot). Five IP addresses (and their associated subnet masks!)

So when I think of bulk admin & host profiles –  I don’t think they add up. Have I ever created an IP conflict by applying the host profile to many ESX hosts? Yes, an I only have 4 bloody ESX hosts!

I have PowerCLI scripts which automates EVERYTHING I would want to do (and more) than Host Profiles are current capable off. It did take some time to learn PowerCLI and test my scripts. But time is something I have plenty of when I’m not teaching. I wouldn’t say I was the most gifted of scripters – but I don’t care as long as it works, and works reliably.

My PowerCLI script will add a host into vCenter and completely configure it – include things that host profiles cannot do… such as iSCSI and DPM settings. The other thing I don’t like about host profiles – is the fact the host has to be maintenance mode to apply them.   So they can’t really be used as a reconfiguration tool – because who wants to move 20-30 VMs just to add a port group – when something like PowerCLI could do that infinitely quicker with just a couple of lines of script. That more or less relegates the host profiles to being a “consistency” check too – like ConfigControl without the licensing (another product that should be included in the core vSphere4 product!!!).

What’s irritating about host profiles – is that I think if you did have 100′s of ESX hosts you would want something that was more automated than host profiles. The smaller/medium shops won’t buy enterprise+ so they won’t benefit. When I teach VMware, it’s at this moment that I mime to my students – taking a gun from the wall – loading it up – and then firing it at my own foot. It always makes my students laugh.

Can you see what I’m doing here? I’m not arguing against DvS/Host Profiles per se. I’m looking at the technologies and what I might trying to achieve with them – and seeing if they help or hinder. Like anything it depends on the environment and the skills of the person involved. So here’s what I’m recommending in the book:

• New to Product/Admin with no time/aptitude for Scripting
  ESXi+vCenter4+DvS+Host Profiles. Essentially, drag-and-drop management strategy….
• Admin with plenty of time/aptitude for scripting/100′s of Servers
  ESXi or ESX “Classic”+vCenter+DvS+PowerCLI. I know this guy would not be disappointed with PowerCLI. They are adding another 60 cmdlet in time for vSphere4.1….
• ESX 2/3 Admin who has an existing kickstart scripted environment” –
  Don’t re-invent the wheel. Use ESX Classic+KickStart (UDA)- with a view to moving over to PowerCLI, ready for the demise of the COS in vSphere5 (circa 1012?).

  R.I.P. COS

I don’t know what category you fall into. You see host profiles could benefit everyone – the more ESX hosts – the more benefits they are. But then you have to buy this damn + product to get them. It’s just madness.

As for DvS. You could take a hybrid attitude on that. This appears to be the consensus view at the moment. That you use Standard vSwitches for management & vmkernel – and leave DvS just for VMs. This seems to be because Standard vSwitches are more of a “known quantity” than DvSwitches. I had hoped that host profiles would see of end of scripting and scripted installation, but nothing seems offer the same flexibility of things like the EDA/UDA and PowerCLI…. If you really mad about scripting then PowerCLI is the way to go. Hence my recent move away from “Classic” to ESX4i, and using PowerCLI.

And Finally…. I’ve been using DvSwitches for everything since the beta. I’m firm believer in using new technology in lab environments like my own – the more exposure you get to technology the more comfortable/confident you become with it. Plus it gives you the edge on folks who might be more production/conservatively minded. I’ve had no problems until I make an admin error. Like the day I removed vmnic0 from DvSwitch0. But hey, I could have done that with Standard vSwitches as well.

I now know how to fix both it they get broken, and that does impress my students.

Well, the licensing part of this took me the better part of day to work out. That’s because I’m more or less crap at decoding the API SDK referrence guide:

http://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/index.html

My favourite part of the license script is the $licassman variable. It could be used to describe a number of people I’ve worked with…

I want learned today is the add-host cmd-let has new parameter it didn’t have in the beta/RC1 of vSphere4. The -force switch is used to accept the default/untrusted SSL SHA certificate that comes from clean installation/first boot of an ESX host.

$vmhost = “esx4.vi4book.com”

add-vmhost $vmhost -location (get-datacenter -name ‘SanFran DataCenter’ | get-folder -name ‘AMD Hosts’ | get-cluster -name ‘AMD Cluster 2′) -user root -password password -force: $true

$targethostMoRef = (get-VMHost $vmhost  | get-view).MoRef
$si = Get-View ServiceInstance
$LicManRef=$si.Content.LicenseManager
$LicManView=Get-View $LicManRef
$licassman = Get-View $LicManView.LicenseAssignmentManager
$licassman.UpdateAssignedLicense($targethostMoRef.value,”YOUR LIC KEY”,”vSphere4 Enterprise Plus (1-12 cores per CPU”)

Well, I have been working on how to do this for a couple of days (believe it or not). I find navigating the VI API/SDK a bit a struggle. In the end I google-wacked an obscure part of my powershell that was giving me troubles, and found this community thread:

http://communities.vmware.com/message/1301276

$si = Get-View ServiceInstance

$LicManRef=$si.Content.LicenseManager
$LicManView=Get-View $LicManRef

$license = New-Object VMware.Vim.LicenseManagerLicenseInfo
$license.LicenseKey = “YOUR LICENSE STRING HERE”
$license.EditionKey=”esxEnterprisePlus”
$LicManView.AddLicense($license.LicenseKey,$null)

I found out the editionkey value by using the QueryAssignedLicenses method which is part of LicenseManager. Anyway, I kinda got sidetracked by this – because what I really want to do is license an ESX host after its been added to the vCenter with the add-vmhost cmdlet. This should be a good start to working it out.

This afternoon (well my evening actually) I got a sneaky peak at NetApp’s new plug-in to vCenter dubbed the NetApp RCU 2.1. I was joined in the web-ex by fellow blogger Scott Lowe, and by Eric Forgette, the key architect/developer for RCU – in case you don’t RCU stands for Rapid Cloning Utility. Put simply the RCU leverages the functionality of a NetApp array to quickly duplicate VMs, with the strongest usage case being VDI of course. NetApp has been doing this sometime – and the early videos of them creating numerous VMs for VDI via script has gone down in youtube history… Anyway, that functionality of cloning is available to all NetApp customers free of charge, without the need to learn any fancy scripting. Let me give you a walking tour of the new RCU. Full details will be available when it is offically launched at VMworld this year.

(more…)