Upgrading UCCX 9.0(2) to 10.6(1)

If you’re wondering about upgrading UCCX from 9.0(2)SU2 to 10.6(1)SU2, and would like that information with a side of snark, then this is the post for you.

Fair warning, this is one of those your-mileage-may-vary entries, definitely do not take my notes as gospel because I promise *I* won’t be the one restoring your UCCX server from backup at 3am.  Always read the release notes and upgrade guides in their entirety, skipping pages earns you nothing but (more) pain and suffering.

That being said, there’s a few important compatibility matrices you will want to check for this upgrade.

Pay close attention to whether your UCS can handle later versions of ESXi without UCS upgrades. For instance, 10.6 no longer supports ESXi 4.X, so you might have some pre-upgrade UCS work to be done before the real party gets started. If you are unsure about your UCS/ESXi version compatibility, try this link for checking.

UCCX 10.6 upgrade from 9.0(2) does require a .cop file, I used ciscouccx.refresh_upgrade_v1.9.cop.sgn.

Before beginning this process, I highly suggest taking a screenshot of your licensing, sacrificing a couple of chickens, and making sure you have ordered your upgrade license through the PUT tool. Voice engineering breeds much in the way of paranoia, so I also recommend downloading the scripts and prompts, just in case you get the additional fun of rebuilding the server from scratch. It should go without saying, but I’ll say it anyway, be sure to check that your backups have been running AND that they have been running successfully.

The order of operations goes something like this for an HA setup, note that there is no Finesse deployed in my setup, strictly CAD.

  • Confirm the primary is active and all services show IN SERVICE – don’t skip this, I’ve never tried to upgrade in an active failed over state, but I imagine it’s like crossing the streams and would end in much badness.
  • Install .cop file on the primary, reboot, grab coffee, wait for services to come back up
  • Install .cop file on the secondary, reboot, grab moar coffee, wait for services to come back up
  • Install upgrade file to primary, drive to a different county to get coffee, don’t panic when the server reboots during the installation, and do not reboot after install.
  • Install upgrade file to secondary, switch to vodka. Question life choices to get involved in voice engineering.  Do not reboot after install.
  • Switch versions on the primary. There’s more than enough time at this stage to continue questioning your life choices. All of them.
  • Server (finally) boots to new version. Wait for services to start. The docs say this could take up to 30 minutes. Shouting profanities at the server will not shorten this interval significantly, but you’ll likely try anyway.
  • Log into the server, install license file, note the error message about OVA template issues. Shut down the server because seriously, who needs that kind of negativity in life? Or shutdown because you need to make changes to the OVA. Whichever…
  • Modify OVA template for RAM, OS, and vNIC changes*
  • Power on server, wait for services. Yes, again.
  • Switch versions on the secondary, repeat the process above, pour another glass of whatever is left.
  • Once the primary and secondary are both online with all services show IN SERVICE, check that replication status is good. 
  • Run the client configuration tool and test your queues.  Buy a lottery ticket if you haven’t had to call TAC,by this point.**

The above list is strictly an overview, but gives you a reasonable idea of what to expect during the upgrade. A whole lot of proper planning will result is a whole lot of waiting for things to happen, but not much else.  An uneventful voice upgrade is an awesome voice upgrade.


*Things got interesting with this. During the planning stages, TAC sent me a link to the procedure for altering the OVA, related to Bug ID CSCut04158. This detailed the process to change the vNIC to vmxnet3. When I presented the process to my virtualization guru, he concluded it would not work for our configuration, we would have to use some PowerCLI magic instead. And by we, I mean he. He used PowerCLI magic, I just threw another chicken onto the alter. The code went something like get-vm MyServerName | get-networkadapter | set-networkadapter -type “vmxnet3” – use this suggestion at your own risk, I am not a virtualization expert, nor do I play one on TV. 

**I will point out that in the upgrade guide, an error about unaligned partitions is called out as a potential issue – it sounded like a whole lot of no fun resulting in rebuilding from backup, and I was quite relieved I didn’t hit that one. Did I mention read the docs?  Definitely do that…

Published 08/16/2016











Posted by on 2016/08/16 in UCCX, Upgrades, Upgrades


Tags: , , , , ,

2016 Cisco Live, US – Geeks for life.

Putting together a wrap up post on Cisco Live US always makes me smile, and 2016 is no exception.  As many of you know, this CLUS marked 5 years since Tom and a small group of engineers first bonded over networking nerdiness and an addiction to 140 characters.

We’ve followed and helped each other through upgrades, outages, career changes, certifications, and a plethora of engineering challenges. We’ve commiserated with the suck that sometimes is our jobs, we’ve championed the hard-fought successes of our peers, and as a bonus, we’ve managed to provide an ever flowing stream of hilarious commentary along the way.

I’m thrilled to be part of a networking community that just keeps growing larger, and I’m continually impressed with the engineering talent this group represents.  I’m also pleased to have been part of Tech Field Day at CLUS as well, which continues to do an amazing job bringing great content and engineers together.

Just check out this awesomeness. Could there be anything better? No, no there could not.


Tags: ,

8841s and line selection woes…

Just a quick post in case there is at least one other person in the world that didn’t know this particular trick for the 8800* series phones that always go off hook on the last line used.  In case you weren’t aware that these phones had this behavior, don’t worry, as soon as you deploy one to a receptionist that handles more than a single phone line, you WILL be made aware. You may not survive the encounter, but you will know…

Should the user let you live long enough to research a solution, you will find that TAC doesn’t offer you any hope in restoring the requisite amount of receptionist happiness to keep oxygen flowing to your lungs. TAC will politely inform you that the phone is operating as designed and that you might want to buy the receptionist some flowers instead.  Okay, maybe that last part was my idea…

Technically TAC is right, you cannot force the phone back to selecting the primary line after hanging up a call that was answered or placed on any of the other lines. However, as my awesomely brilliant coworker discovered, you can change the behavior of the phone so that when calls come in they are DISPLAYED on the primary line.

This way, when the user hangs ups, the phone still has the primary line selected, even if the last call received was for one of the other lines on the phone. This mimics a behavior users have come to know and love of the older generation of phones, a behavior users who manage multiple lines desperately want back.

You can set this option on the phone configuration page, just search for Show All Calls and change the feature to Enabled.

Show All Calls on Primary Line

This won’t make the remembering-the-last-line-selected-for-off-hook behavior go away, but it does make its impact far less noticeable to end users who might otherwise want to plot your demise.

* In this case we were working with 8841 phones, but my coworker has used this trick in the past on 9971 phones, so I’d be inclined to believe this would work on pretty much any SIP phone that has this behavior. 

Published 04/11/2016


Tags: , , , , , ,

Tom’s Corner – Cisco Live Twitter community 5 years later…

If you’re an annual attender of Cisco Live like I am, you might be noticing there’s some pretty cool stuff taking shape for 2016.

This year will be the 5th anniversary of Tom’s Corner – the first ad hoc gathering location for us geeky, twittering engineers*.

Hard to believe, but in 2011 we were just a smallish group of network nerds, some with quite a bit more hair in those days, and of course, the one random photo-bomber guy on the left. Still nobody knows who that guy was…

The Twitter Group 2011

The amazing amount of talent represented and the camaraderie of the network engineering community is what makes Cisco Live the place to be.

This year I’m also looking forward to witnessing some epic Engineering Deathmatch action. If you aren’t familiar with Engineering Deathmatch you gotta watch this video.  You won’t be disappointed.

And if that weren’t enough Cisco Live goodness, I gotta admit I’m pretty excited about hearing Keyser Söze, err Kevin Spacey, speak.

Other useful details for attendees:

  • The scheduler will be available on  April 26th for NetVets, May 3rd for the rest of us.
  • The band this year is Maroon 5, and the CAE is going to be at the shiny new T Mobile Arena
  • Newbies to Cisco Live can get a super cool mentor to help them figure the conference thing out, just sign up here
  • Tech Field Day will be at Cisco Live this year as well, look for fabulous content from them. If you’re interested in being a delegate for this or any event, apply here
  • Awesome blogs to watch for new Cisco Live content and to read past posts for excellent conference advice: @fryguy_pa, @scottm32768, @mrtugs@net_introvert, @networkingnerd, @someclown, @aconaway, @danieldibswe, @gingmarCisco Live Attendee Blogs
  • The annual twitter list is out, sign up here!

*Tom’s corner even had it’s own check in on Four Square. In case there was any doubt we were and still are total nerds…

Published 04/11/2016

Leave a comment

Posted by on 2016/04/11 in Cisco Live


Tags: , ,

Cisco Live Europe: NBASE-T progress report

The Ether

Robert Metcalfe –May 22nd, 1973 – first memo describing Ethernet. Photo Credit:

If you ever get a chance to watch Peter Jones present on a topic, I highly encourage you to take it! He’s passionate about multigigabit and the evolution of Ethernet technology, and it shows.

In this CLEUR Tech Field Day Extra session, Peter outlines significant progress that has been made on the 2.5G and 5G standardization process, which is pretty impressive considering the NBASE-T Alliance was only formed in October of 2014.

In addition to announcing some Cisco products that will be capable of multigigabit magic, Peter also lays out some additional interesting use cases, besides just the hotly debated timeline of the impending 802.11ac Wave 2 data rate apocalypse.

A few highlights from the product announcement pieces of the presentation, be sure to consult relevant data sheets for accuracy, caveats, and any insomnia issues you might have:

  • 48 port blade for the 4500E, 12 ports will do multigigabit
  • 3850 24 or 48 port switch, half of the ports will do multigigabit; note this is an entirely new SKU but stacking is compatible with previous 3850s
  • 3560-CX compact switch, two ports will support the multigigabit
  • All multigigabit ports are supporting UPOE (60 Watts)
  • No SFP+ available – Peter talks briefly about the heat challenges around this form factor, interesting geeky stuff
  • 3800 series AP

While I agree that there’s certainly some questions as to which devices and their data rates will bring current copper infrastructure to its knees, realistically getting more mileage out of already installed cabling makes a good kind of sense. Hardware refresh cycles are generally much shorter intervals than building wiring replacement schedule. Finding ways to cope with aging copper cabling and eking out a few more years of use from such a significant investment is a reality for engineers.

To leverage 2.5G or 5G magic, both devices – what’s getting plugged in and what’s being plugged into – will need to support the multigigabit technology.  Coordinating equipment refresh cycles and budget to make this happen could be quite the challenge as well.  Some number crunching would be appropriate to determine cost effectiveness for your own environment, but multigigabit Ethernet could be that additional tool in the belt when it comes to maximizing cabling investments already made.

Also of nerdy note, Peter’s short discussion of how the multigigabit magic filters out the noise using advanced coding and signal processing is quite fascinating. These brief comments led to me down a rabbit hole and to this interesting article,Crosstalk problems are back, which discusses the issues of noise when dealing with high data rates across copper wiring.

I highly recommend checking out these posts from other awesome engineer bloggers, good stuff:

And for more learning on the topic, definitely check out these:

Fundamentals of NBASE-T


Standard Disclaimer: Tech Field Day covered my expenses at Cisco Live Europe, but I am a redhead, any thought that my opinions could be bought or dictated is just crazy talk. 


Tags: , , , , ,

Updating DNS server IP addresses on Cisco voice servers

This post focuses on something that went right when making voice changes – shocking, I know!  Yes, I did open two different TAC cases before making the changes, and yes, I did have to talk to TAC once during the actual change process, but in my book, that’s a total win.

My mission, which the sys admin gave me no choice but to accept, was to change the DNS ip addresses each voice server was pointing to.  Now in sys admin world, this is hardly a big thing, but in voice world, no change is so small and insignificant that cannot be made intricately cumbersome to complete.

If you live in the CUCM 8.x universe, you are probably familiar now with the concept of a license mac. Changing any of the components that went into generating the precious license mac for the server invalidates all the pretty, pretty licenses you spent hours (likely even days) of your life trying to get in the first place. Changing primary DNS *is* one of those components in 8.x, so accepting your fate of needing to rehost licenses for ALL THE THINGS is step one in 8.x universe. Step two likely involves some heavy drinking.

But I currently live in 9.1.2 world – so some of this misery is offset.  The CUCM and Unity Connection (UNCX) teams at some point decided this whole invalidating-the-licenses-for-minor-changes deal was a suckfest, at least I’m assuming that was their thought process, so with version 9.x, changing primary DNS server ip address for CUCM and UNCX servers doesn’t upset the license mac gods. Cisco IM&P now leverages CUCM licensing in 9.x, so no rehosting required for those servers, either.

UCCX, though. Of course UCCX still cares. Because UCCX.

So enough background and onto the process – which is pretty straightforward considering it’s voice stuffs. Standard disclaimer, I put this list together from various calls with TAC asking for documentation and clarification on the process for each application and what to expect. Your mileage may vary, don’t take my word for it, always have a good backup, and certainly don’t blow your voice servers up. Check the docs, check with TAC. Note that for each application, the changes are made on the publisher/primary server first, then any subscribers or secondary servers.

For CUCM and UNCX servers:

-In the GUI of the License server, remove the CUCM or UNCX server instance from the License server.  Yes, I know the trepidation of deleting anything from a voice server – especially involving licensing, but it is strongly recommended to do so. If you forget or don’t do this, PROBABLY nothing will happen, according to my conversations with TAC.  But no way I’m taking that chance, you decide for yourself…

-At the CLI of the server, issue the following commands:

set network dns primary X.X.X.X
set network dns secondary X.X.X.X
show network eth0 – confirm the changes

-Add the instance back to the license server and synchronize. When adding the server back, remember it’s the OS username and password that you want to be using.

-Additional step for CUCM highly recommended by TAC: restart the Cisco Tomcat service at the CLI with the command utils service restart Cisco Tomcat

-This is also the process for stand alone license servers, but of course you don’t have to remove any instances from the license server or perform the Tomcat restart.

For IM&P servers:

-At the CLI of the server, issue the following commands:

set network dns primary X.X.X.X note you will still get an error message that rehosting is required, but I confirmed later with TAC that this is just a holdover error message from the 8.x days.
set network dns secondary X.X.X.X
show network eth0 – confirm the changes

For UCCX servers (note I have an HA environment):

-Take note of the current server license MAC, put it in a safe place.  I copied the contents of the license files to my desktop and took a screenshot of the current license configuration page. Because it’s voice and the paranoia with licensing runs deep.

-Sacrifice a chicken or two, and then at the CLI of the server, issue the following commands on each server:

set network dns primary X.X.X.X
set network dns secondary X.X.X.X
show network eth0 (confirm the changes)

-Reboot the primary, wait for what feels like an eternity for the primary to come back up and get it’s services started, then reboot the secondary.

-Take note of the new license mac and request a rehost, provide licensing with the new and the old license macs.

-Load the new license file and start happy dance.  Unless you hit an issue like I did and the new license won’t load. Then try another reboot of the pair, attempt license load once more with fingers more tightly crossed than before, and then proceed to happy dance.

Last, but not least, my good twitter friend and awesome voice guru Ryan Huff pointed out to me this Answer File Generator tool which can be used to predict your new license mac so you can request a rehost in advance. I decided that the 30 day grace period for UCCX would be enough for this project, but it’s fantastic to know that such a thing exists. Especially if you are going to be invalidating a lot of licenses, have a very small change window, and want to go ahead and get off grace period licensing as quickly as possible*.


*my rehosted license file for UCCX was generated in under 15 mins. Impressive. Still a PITA, but at least a quick PITA…

Published 09/09/2015










Tags: , , , , , , , , , ,

Cisco Live 2015 – Community rocks.

FullSizeRender (22) The San Diego sun has set on another Cisco Live, closing out an amazing week of learning and community. As usual, Cisco Live 2015 was one jam-packed, over-before-you-know-it, week full of opportunities to learn and network with incredible peers.

Once again, I got to be part of Tech Field Day, attending presentations by both OpenGear and Netbrain.  I highly recommend checking out the videos for both, you won’t be disappointed. Netbrain even announced a DevOp version of their product, which allows you 10 free nodes. Super cool stuff, the demo will knock your geeky socks off.

And, of course, I got the hang with so many fabulous engineers, some I have come to know well over the years, and others were new shiny faces. Now I shall subject you to my camera roll, because awesome. Enjoy.

A Canadian and a Texan walk into a bar... @ghostinthenet @that1guy_15

A Canadian and a Texan walk into a bar… @ghostinthenet @that1guy_15

Oh, the priceless expressions of Canadian @Rob_Coote. It doesn't get any better. @sharpnetwork

Oh, the priceless expressions of Canadian @Rob_Coote. It doesn’t get any better. @sharpnetwork

The extremely intelligent, very kind @jamieljones, a total honor meeting her. She is an inspiration to anyone working to get their digits.

Wireless guys loving on wireless. @jsnyder81

Wireless guys loving on wireless. I find it’s better not to ask… @jsnyder81

Awwww, these guys... @tonhe @ciscovoicedude

Awwww, these guys… @tonhe @ciscovoicedude

The fabulous @pilotmike, oddly without his Blackberry. @denisefishburne

The fabulous @pilotmike, oddly without his Blackberry. @denisefishburne

FullSizeRender (15)

Anti-social social dining. @avalonhawk @scottm32768 @radzima

Because network engineers ARE superheros. @scottm32768

Because network engineers ARE superheros. @scottm32768

Behold @vcabbage, the self proclaimed pretty, pretty princess.

Behold @vcabbage, the self-proclaimed pretty, pretty princess.

Some look better in skirts than others, just saying... @kathleenmudge @wifijanitor @subnetwork @ucgod @fryguy_pa @lauren @denisefishburne

Some look better in skirts than others, just saying… @kathleenmudge @wifijanitor @subnetwork @ucgod @fryguy_pa @lauren @denisefishburne

Yeah, what can I say about this? ;) @grinthock @Rob_Coote

Yeah, what can I even say about this? @grinthock @Rob_Coote

The incredibly smart, incredibly witty @drjmetz.

The incredibly smart, incredibly witty @drjmetz.

The expressions are priceless. @kathleenmudge @lauren @ucgod @wifijanitor

The expressions here are priceless. @kathleenmudge @lauren @ucgod @wifijanitor

Usually it's his backside people are posting... @networkingnerd

Usually it’s his backside people are posting… @networkingnerd

These two crack me up. @scottmorrisCCIE @denisefishburne

These two crack me up. @scottmorrisCCIE @denisefishburne

I love that @samplefive is just having a normal conversation in the background. @wifijanitor @fryguy_pa

I love that @samplefive is just having a normal conversation in the background. @wifijanitor @fryguy_pa

Social media gurus @kathleenmudge @lauren

Sweet and amazing social media gurus. Thanks for all you do. @kathleenmudge @lauren

These two are fabulous. But don't tell them I said so... @fryguy_pa @subnetwork

These two are fabulous. But don’t tell them I said so… @fryguy_pa @subnetwork

Kilts.  There were kilts. (Why??!!) @subnetwork

Kilts. There were kilts. (Why??!!) @subnetwork

It's importance to have balance in your networking career.  @denisefishburne @nmarus

It’s important to have balance in your networking career. @denisefishburne @nmarus

FINALLY this guy decided to join us for a Cisco Live. About time... @JTIE_6EE7

FINALLY this guy decided to join us for a Cisco Live. About time… @JTIE_6EE7

Because UC engineers rule.  @mlundbom1 @ucgood

Because UC engineers rule. @mlundbom1 @ucgood

The awesome ginger princess @jay25f with the fantastic @denisefishburne.

The awesome ginger princess @jay25f with the fantastic @denisefishburne.

Awwww...thanks @drjmetz for this pic. @networkingnerd

Awwww…thanks @drjmetz for this pic. @networkingnerd

The @ucpappy trying to collaborate on some new fangled device. @ucgod

The @ucpappy trying to collaborate on some new fangled device. @ucgod

The caffeinated side of the social media hub.  @fryguy_pa @bbaize @wifijanitor

The caffeinated side of the social media hub. @fryguy_pa @bbaize @wifijanitor

The non-caffeinated side of the social media hub. @bcjordo @hankito @tonhe

The non-caffeinated side of the social media hub. @bcjordo @hankito @tonhe

Completely impressed with the technology aid @CiscoTACOPS brings to disaster situations. @aconaway @bcjordo @densaer

Completely impressed with the technology aid @CiscoTACOPS brings to disaster situations. @aconaway @bcjordo @densaer

Networking - it's about community.

Networking – it’s about community. And this community is incredible.

How we all felt by the week's end. @lauren

This sums up perfectly how we all felt by the week’s end. @lauren

Published 6/19/2015


Posted by on 2015/06/19 in Cisco Live, Cisco Live 2015


Tags: ,