Using vCops in the real world. A few findings along the way.

Ok, I have been using vCops for quite a while and I have mostly come across posts that are showing how to leverage a few of the features but they what I would call the “prescriptive lab” where everything is setup to show you a situation. I think that these posts are great and I know that I used them a lot when I first got into the product. Now, this could just be me being a bit simple but I found that when I scratched below these examples things got complex and there are so many settings and metrics that are not always that clear on how to use them and what to look for. So with that in mind here is a post with a few of my findings. I hope that they may help someone else out there!

Also, I have been a bit cheeky and lifted my notes straight into this post, so it may be a little rough around the edges!

VIEWS

When using non trend views set a lower interval as a VM or Host or Datastore for example must exist for the entire period that you have set to appear in the view and be taken into account.

Non trend views are summary views for example and are controlled globally in the settings and apply to all views.

vcops tips 1

Just a little bit of extra info on intervals. If you select Daily for example with an interval of 12 that would show the last 12 days of data. So interval is determined by the interval type selected. So I thought that if you selected daily and 12 intervals it would show you 2 hour samples through out the day. And like wise, select monthly and 12 and you get 12 points of data collection rolled up and presented. D’oh, talk about over complicating that one!

Demand or Allocation models

In the case of allocation and demand settings there are a few things to know. A container in the case of vCops is a cluster and not the folders that you pull from the VM and templates view. If you select to do both demand and allocation is will show the most restrictive case for the interval that you have set.

vcops tips 2

What we are seeing below is that our effective demand is more containing than allocation as our over allocation rule is set to 5:1 for CPU.

vcops tips 3

When we look at the usage and host usage we see and average usage of 38%, so how can that be? You must remember that is an average so we have to dig a bit deeper and look at the same report at a per host level.

Host 1 – Over by 5

 vcops tips 6

Host 2 – Neutral

vcops tips 5

Host 3 – 1.6 VM capacity remaining

 vcops tips 6

Host 4  – Over by nine

vcops tips 7

Operations Tab

Using the operations tab you can get a breakdown of the hosts real-time work. Looking the CPU chart you can actually see that most hosts are showing a larger bar line for CPU demand when compared to usage.

At this point lets put a quick definition on demand. Demand is basically what the VMs want and would use providing that there was not any contention. So when demand is high and usage is below we have contention occurring. Now this could be from just one or two larger VMS and we can drill down to find out what the cause is. So to use another point in our case in these charts the VMS are looking for more than is available.

So this could be that a number of VMs have too many CPUs assigned and they are showing more demand but having to wait so high co-stp and cpu ready times, or it could be an in balance of VMs that DRS cannot fix or just lots of hungry applications!

You may have noticed that one of the hosts (02) was showing no data and this is due to a problem that has yet to be resolved….

** Note that capacity risk report also shows this info in a nice way. Less detail, more high level.

General Tips

You need to have a policy in place that says what your over commit policy is. This is used to show in summary views how much remaining capacity you have based on that rule.

We have ours set to 5:1 that shows a neutral view to vCentre in terms of number of VMS that we have.

In terms of average VM spec vCops uses its trending information to determine a Small Med and Large profile for our environment

Setting a work week avoids backup traffic for example being classed as an everyday trend.

vCops polls vCentre every 20 seconds for its real-time data. It does not look back at its historical data.

Error after moving Citrix Configuration Logging DB to new server

I have recently had to move all of my Xenapp 6.5 databases from an old SQL2005 server to a new SQL2008 server. For the most part it went well but I did encounter the following error, that is pretty simple to solve.

After moving the CL database I went and reconfigured the Configuration logging settings, using the wizard in the Appcentre. That all went well and it was happy talking to the new DB. I then went and made a simple change to an application, so that I could query the change log. What I found when I hit the get log button was the following error came up.

“The EXECUTE permission was denied on the object”

Citrix CL DB Move error

Ok, so its access denied. Seems pretty simple. I went back to my DBA and we checked that the Stored Procedures were there (Which they were) and we re-applied permissions to the Citrix service account. It seemed that no matter what we did the error persisted.

After a quick SQL trace we could actually see that when you hit “get log” it uses the logged on user account to query the DB. So adding the correct AD group to the CL DB on the new server fixes this issue. Simple stuff, but it may help someone else if need be.

To further back this up you can read some of the public docs on the CL DB permissions.  http://support.citrix.com/proddocs/topic/xenapp6-w2k8-admin/ps-maintain-define-db-perms.html

A few tips for moving your Citrix Xenapp 6.5 Database

I dont intend on reinventing the wheel here as there is already a great blog post detailing this process which you can find here My thanks to Terrance for his work on this blog. I just wanted to add a couple of findings based on my experience of the process.

I do agree with Terrance that the Citrix docs are not great and that is why you should follow his post, much as I did.

Lets start with a little bit of background on my setup. I have 9 Xenapp servers in my single farm, with one data collector. I had to try to do this move during the day as my DBA was not available out of hours for the usual maintenance window, and we could not wait or defer the move until he was. Catch 22 then! Typical….

So here was my plan.

  1. Disable logons the day before the move to the data collector
  2. The following day I will stop the IMA on that server
  3. The IMA services and 8 other XA servers will all be left with logons enabled serving users
  4. The DBA will back up and restored the DB to the new server
  5. We will then take the original DB on the old server offline, cutting off the 8 live XA servers
  6. At this point I would test that I could access a few applications – This would test that the local cache was working
  7. I will then repoint the data collector to the new DB server using Terrances steps
  8. After the IMA service restart I will test an application launch and change from the data collector server
  9. I will then have a maintenance window in the evening to perform the process on the other 8 XA servers

First question to answer is. Did I follow this plan and the answer is yes. And for the most part it all went well, there are just a couple of gotcha’s that I didn’t fully think through, so I will list them out for your consideration. They all make perfect sense when you consider that they have no real-time access to the DB anymore.

  1. I found that Citrix shadowing didn’t work whilst the live XA servers were running in cached mode. Support staff were unable to enumerate users.
  2. Opening the App Centre on a cahced server would prompt a discovery to run, which would fail. You could only run App centre on the server that was migrated
  3. You could not control the logons to the servers running in cached mode
  4. My last point is just for clarity, you can move the DB without stopping all of the IMA services on your XA servers

As per usual I hope this helps someone else out there. None of this is rocket science but sometimes it just helps to read about someone elses experience!

Error – Internal View Composer Error. Contact your Administrator

I came across this little error a few weeks back. I could log into my admin console and no matter what I clicked on in relation to a desktop, or a pool for example I would see the following error message.

Internal view comp error

It’s not a very helpful error and I am the admin, so there was no one else to turn too! I spent a while checking the basics and could see that all servers and services were up, virtual desktops were running but we could not make any changes to pools or carry out any actions, such as a recompose.

I bit more digging and there really was nothing jumping out as being wrong, so I turned to Google. There were not a lot of options/fixes on offer and everything that was listed didn’t fit the bill.

So I thought I would post the fix for this issue as it was pretty simple in the end. The user account that our View services are using for some reason had a password that was set to expire. Clearly this is a bit dumb and was an oversight. A quick AD update to the user account, a close and re-open of the View admin console (WI) and we are in business.

So there you have it. Simple fix.

Reflections on hosted telephoney

This past few weeks have been really interesting as I have been digging around in the world of hosted telephony & UC solutions. As a company we have had little mandate to look for hosted services but I am looking to change that ethos and move some of our non core ( and by that I mean core to the business not IT) to hosted or “cloud” services.

Another key driver being imposed on me and my team is to gain flexibility of being able to scale up and down quickly. Great I thought my goals and business goals are fully aligned and we set about speaking to vendors.

So far I have spoken to two UC providers, one who uses MyTel and the other Cisco. The story is the same, minimal CAPEX costs (handsets mainly) and a OPEX cost model, as you would expect.

At this stage I am less concerned with features and feature parity, I think they are all similar and I know they will for the most part do what I need. What is of interest are the terms, duration of contract and dealing with the variability. All things that I just thought would be simple, however maybe I have been a little wet behind the ears here but I found a few things out that may help others who are looking for hosted UC and telecoms.

It is pretty evident that nearly every vendor that I have spoken with have a hosted offering through one of many channel partners. You can scale up as much as you like but if you want to scale back then you had better think again. As the vendor licensing program’s do not seem to support doing that as the channel partner has to by X licenses and put them aside for your business, so why would they foot the bill if you decided two months in to drop say 30% of your users off their platform.

The best deal I have found to date is one channel partner would allow us to flex down by 15% but in return we had hiked monthly rates and had to commit to a five-year contract!

To date I have now spoken to the following vendors (Avaya, MyTel, Cisco, Arista and Gamma) and some of their channel partners hosting their respective platforms and the story of scale up but not down is unanimous. Yet they all tout a consumption based model in the cloud is what they do. I disagree and think that they all need to get up to speed here, as there must be other businesses of a similar size who are looking for the same flexibility.

There is however some hope…

I wrote that first section a few weeks back and I never got around to publishing it. Since then I actually discovered two channel partners one of which is using Avaya and the other is on….

They both offer a true consumption based offering that you can scale up and down as you need too. The Avaya partner is interesting as they are pioneers in some respects, as they were the first partner to convince them to offer this type of license model and as such they have worked with them to fine tune this model.

I do not want to plug their business on my blog, but please message me in the comments if you would like to speak with them.

Transitioning from techie to manager

This posts wont be for everyone but its something that I wanted to share some thoughts on and it will the first in a series of posts around management and my experience of making the step change.

I have been working in IT for 14 years and during that time I have led project teams, been part of a team and worked alone, but the one common theme was that I was full-time on the tools. You know how it goes, Clicking buttons, tearing your hair out late at night over cryptic error messages or taking phone calls from users who just don’t get it at all.

About two years ago I was offered a new role to lead a small team of 5 infrastructure engineers. It was still considered a hands on role, but I would also coach, mentor and be the escalation point for the team. I figured that this was  a chance to try something a bit different, and so my career as a manager took its first steps.

Over the past two years I have learnt a hell of a lot about myself and people in the work place… and I have learnt a lot the hard way as well. Which got me to thinking that I could actually write a few posts on some of the step changes that I have seen since being responsible/accountable for more than just my own work load and maybe share some of the tips that I have around the following subjects.

How your day changes

Mentoring and Coaching

Giving good and bad feedback

Becoming the boss of peers

Project Management

Managing Upwards, sideways and down

Interviewing

One to Ones

Goal setting

Performance reviews

Appraisals

and the list goes on. Some of you maybe thinking that is all easy stuff but I didn’t find it all that easy and maybe I can help someone else deal with the step change a bit quicker whilst making fewer mistakes in the process! On top of all that I still have to keep my hand in on the technical aspects but at a slightly higher level, which is good as I still like to click the odd button and flex the technical muscles!

For those thinking about making the change I would say go for it, especially if you like working with people and helping them to achieve their goals. One of the biggest change for you as a new manager is realising that your success is no longer installing and configuring a new vSphere Cluster, or migrating your user base from Office 2010 to 2013, your success is realised though others doing these things. You help them to do it and advise, guide and teach along the way. Sure you will get the chance to click a few buttons but you wont be measured on that any more, or maybe not so much.

VMware don’t want to build long term relationships with Customers!

This is my first post in while and its sad to say that its a bit of a rant!

I don’t like to bash vendors and I will be the first to admit that I really like the VMware products and community. I guess that my enthusiasm for those is why I am so disappointed with the recent experience I had with VMware licensing direct.

Now, I won’t disclose who I spoke to in the licensing team and I will say that a certain set of circumstances led to my VAR putting me in direct contact with VMware.

As most of us VMware admins know, VMware is not a cheap platform but I still sell it to my business as a platform that continues to deliver a robust feature rich platform that suits our business model and use cases perfectly.

Cutting to the chase, as our conversation developed it became quite clear that all Bob (not his real name) wanted to do was get was a quick sale on the books and was not interested in trying to resolve my issue with a longer agreement.

What I was trying to ask for was a 3 year deal but at a price that was quoted the previous month, the net difference £1500 pounds. On top of that I illustrated our future VMware growth plans and my reason for asking for the price to be honoured. To which he replied and I quote..

“We are not interested in what you could do, and my managers won’t honour that quote based on possible growth.” He was unwilling to even ask his manager and said all he could offer was a 3 year deal at the higher price, to which I said my hands were tied and I could not get agreement to get the increase.

There is a major internal political thing so don’t bash me for asking for such a small amount of money, or why we had gone over the 28 days quote period. But if you ask me I would have thought that most vendors are interested in trying to help a customer, especially given previous history and future business!?

So I was forced down a 1 year deal to keep my costs under budget. VMware were happy to get one more years business. Has it gotten that bad VMware that you need to land grab rather than look at the bigger picture.

I know that VMware are in business to make money but I guess I was a little disheartened to hear that attitude first hand from a vendor that I hold in high regard. I don’t think that I was asking for a lot and I thought that this would be a no brainer, but a chat with my VAR afterwards said that she had seen this hard-nosed approach before with licensing and it was a real shame as they are left holding the baby with the customer.

So am I alone here, did I just have a licensing rep who lived by his monthly target and couldn’t see value beyond the here and now? Or are others seeing major inflexibility when dealing with VMware licensing and pricing policy’s.