Tom’s Corner – Cisco Live Twitter community 5 years later…

If you’re an annual attender of Cisco Live like I am, you might be noticing there’s some pretty cool stuff taking shape for 2016.

This year will be the 5th anniversary of Tom’s Corner – the first ad hoc gathering location for us geeky, twittering engineers*.

Hard to believe, but in 2011 we were just a smallish group of network nerds, some with quite a bit more hair in those days, and of course, the one random photo-bomber guy on the left. Still nobody knows who that guy was…

The Twitter Group 2011

The amazing amount of talent represented and the camaraderie of the network engineering community is what makes Cisco Live the place to be.

This year I’m also looking forward to witnessing some epic Engineering Deathmatch action. If you aren’t familiar with Engineering Deathmatch you gotta watch this video.  You won’t be disappointed.

And if that weren’t enough Cisco Live goodness, I gotta admit I’m pretty excited about hearing Keyser Söze, err Kevin Spacey, speak.

Other useful details for attendees:

  • The scheduler will be available on  April 26th for NetVets, May 3rd for the rest of us.
  • The band this year is Maroon 5, and the CAE is going to be at the shiny new T Mobile Arena
  • Newbies to Cisco Live can get a super cool mentor to help them figure the conference thing out, just sign up here
  • Tech Field Day will be at Cisco Live this year as well, look for fabulous content from them. If you’re interested in being a delegate for this or any event, apply here
  • Awesome blogs to watch for new Cisco Live content and to read past posts for excellent conference advice: @fryguy_pa, @scottm32768, @mrtugs@net_introvert, @networkingnerd, @someclown, @aconaway, @danieldibswe, @gingmarCisco Live Attendee Blogs
  • The annual twitter list is out, sign up here!

*Tom’s corner even had it’s own check in on Four Square. In case there was any doubt we were and still are total nerds…

Published 04/11/2016

Leave a comment

Posted by on 2016/04/11 in Cisco Live


Tags: , ,

Cisco Live Europe: NBASE-T progress report

The Ether

Robert Metcalfe –May 22nd, 1973 – first memo describing Ethernet. Photo Credit:

If you ever get a chance to watch Peter Jones present on a topic, I highly encourage you to take it! He’s passionate about multigigabit and the evolution of Ethernet technology, and it shows.

In this CLEUR Tech Field Day Extra session, Peter outlines significant progress that has been made on the 2.5G and 5G standardization process, which is pretty impressive considering the NBASE-T Alliance was only formed in October of 2014.

In addition to announcing some Cisco products that will be capable of multigigabit magic, Peter also lays out some additional interesting use cases, besides just the hotly debated timeline of the impending 802.11ac Wave 2 data rate apocalypse.

A few highlights from the product announcement pieces of the presentation, be sure to consult relevant data sheets for accuracy, caveats, and any insomnia issues you might have:

  • 48 port blade for the 4500E, 12 ports will do multigigabit
  • 3850 24 or 48 port switch, half of the ports will do multigigabit; note this is an entirely new SKU but stacking is compatible with previous 3850s
  • 3560-CX compact switch, two ports will support the multigigabit
  • All multigigabit ports are supporting UPOE (60 Watts)
  • No SFP+ available – Peter talks briefly about the heat challenges around this form factor, interesting geeky stuff
  • 3800 series AP

While I agree that there’s certainly some questions as to which devices and their data rates will bring current copper infrastructure to its knees, realistically getting more mileage out of already installed cabling makes a good kind of sense. Hardware refresh cycles are generally much shorter intervals than building wiring replacement schedule. Finding ways to cope with aging copper cabling and eking out a few more years of use from such a significant investment is a reality for engineers.

To leverage 2.5G or 5G magic, both devices – what’s getting plugged in and what’s being plugged into – will need to support the multigigabit technology.  Coordinating equipment refresh cycles and budget to make this happen could be quite the challenge as well.  Some number crunching would be appropriate to determine cost effectiveness for your own environment, but multigigabit Ethernet could be that additional tool in the belt when it comes to maximizing cabling investments already made.

Also of nerdy note, Peter’s short discussion of how the multigigabit magic filters out the noise using advanced coding and signal processing is quite fascinating. These brief comments led to me down a rabbit hole and to this interesting article,Crosstalk problems are back, which discusses the issues of noise when dealing with high data rates across copper wiring.

I highly recommend checking out these posts from other awesome engineer bloggers, good stuff:

And for more learning on the topic, definitely check out these:

Fundamentals of NBASE-T


Standard Disclaimer: Tech Field Day covered my expenses at Cisco Live Europe, but I am a redhead, any thought that my opinions could be bought or dictated is just crazy talk. 


Tags: , , , , ,

Updating DNS server IP addresses on Cisco voice servers

This post focuses on something that went right when making voice changes – shocking, I know!  Yes, I did open two different TAC cases before making the changes, and yes, I did have to talk to TAC once during the actual change process, but in my book, that’s a total win.

My mission, which the sys admin gave me no choice but to accept, was to change the DNS ip addresses each voice server was pointing to.  Now in sys admin world, this is hardly a big thing, but in voice world, no change is so small and insignificant that cannot be made intricately cumbersome to complete.

If you live in the CUCM 8.x universe, you are probably familiar now with the concept of a license mac. Changing any of the components that went into generating the precious license mac for the server invalidates all the pretty, pretty licenses you spent hours (likely even days) of your life trying to get in the first place. Changing primary DNS *is* one of those components in 8.x, so accepting your fate of needing to rehost licenses for ALL THE THINGS is step one in 8.x universe. Step two likely involves some heavy drinking.

But I currently live in 9.1.2 world – so some of this misery is offset.  The CUCM and Unity Connection (UNCX) teams at some point decided this whole invalidating-the-licenses-for-minor-changes deal was a suckfest, at least I’m assuming that was their thought process, so with version 9.x, changing primary DNS server ip address for CUCM and UNCX servers doesn’t upset the license mac gods. Cisco IM&P now leverages CUCM licensing in 9.x, so no rehosting required for those servers, either.

UCCX, though. Of course UCCX still cares. Because UCCX.

So enough background and onto the process – which is pretty straightforward considering it’s voice stuffs. Standard disclaimer, I put this list together from various calls with TAC asking for documentation and clarification on the process for each application and what to expect. Your mileage may vary, don’t take my word for it, always have a good backup, and certainly don’t blow your voice servers up. Check the docs, check with TAC. Note that for each application, the changes are made on the publisher/primary server first, then any subscribers or secondary servers.

For CUCM and UNCX servers:

-In the GUI of the License server, remove the CUCM or UNCX server instance from the License server.  Yes, I know the trepidation of deleting anything from a voice server – especially involving licensing, but it is strongly recommended to do so. If you forget or don’t do this, PROBABLY nothing will happen, according to my conversations with TAC.  But no way I’m taking that chance, you decide for yourself…

-At the CLI of the server, issue the following commands:

set network dns primary X.X.X.X
set network dns secondary X.X.X.X
show network eth0 – confirm the changes

-Add the instance back to the license server and synchronize. When adding the server back, remember it’s the OS username and password that you want to be using.

-Additional step for CUCM highly recommended by TAC: restart the Cisco Tomcat service at the CLI with the command utils service restart Cisco Tomcat

-This is also the process for stand alone license servers, but of course you don’t have to remove any instances from the license server or perform the Tomcat restart.

For IM&P servers:

-At the CLI of the server, issue the following commands:

set network dns primary X.X.X.X note you will still get an error message that rehosting is required, but I confirmed later with TAC that this is just a holdover error message from the 8.x days.
set network dns secondary X.X.X.X
show network eth0 – confirm the changes

For UCCX servers (note I have an HA environment):

-Take note of the current server license MAC, put it in a safe place.  I copied the contents of the license files to my desktop and took a screenshot of the current license configuration page. Because it’s voice and the paranoia with licensing runs deep.

-Sacrifice a chicken or two, and then at the CLI of the server, issue the following commands on each server:

set network dns primary X.X.X.X
set network dns secondary X.X.X.X
show network eth0 (confirm the changes)

-Reboot the primary, wait for what feels like an eternity for the primary to come back up and get it’s services started, then reboot the secondary.

-Take note of the new license mac and request a rehost, provide licensing with the new and the old license macs.

-Load the new license file and start happy dance.  Unless you hit an issue like I did and the new license won’t load. Then try another reboot of the pair, attempt license load once more with fingers more tightly crossed than before, and then proceed to happy dance.

Last, but not least, my good twitter friend and awesome voice guru Ryan Huff pointed out to me this Answer File Generator tool which can be used to predict your new license mac so you can request a rehost in advance. I decided that the 30 day grace period for UCCX would be enough for this project, but it’s fantastic to know that such a thing exists. Especially if you are going to be invalidating a lot of licenses, have a very small change window, and want to go ahead and get off grace period licensing as quickly as possible*.


*my rehosted license file for UCCX was generated in under 15 mins. Impressive. Still a PITA, but at least a quick PITA…

Published 09/09/2015










Tags: , , , , , , , , , ,

Cisco Live 2015 – Community rocks.

FullSizeRender (22) The San Diego sun has set on another Cisco Live, closing out an amazing week of learning and community. As usual, Cisco Live 2015 was one jam-packed, over-before-you-know-it, week full of opportunities to learn and network with incredible peers.

Once again, I got to be part of Tech Field Day, attending presentations by both OpenGear and Netbrain.  I highly recommend checking out the videos for both, you won’t be disappointed. Netbrain even announced a DevOp version of their product, which allows you 10 free nodes. Super cool stuff, the demo will knock your geeky socks off.

And, of course, I got the hang with so many fabulous engineers, some I have come to know well over the years, and others were new shiny faces. Now I shall subject you to my camera roll, because awesome. Enjoy.

A Canadian and a Texan walk into a bar... @ghostinthenet @that1guy_15

A Canadian and a Texan walk into a bar… @ghostinthenet @that1guy_15

Oh, the priceless expressions of Canadian @Rob_Coote. It doesn't get any better. @sharpnetwork

Oh, the priceless expressions of Canadian @Rob_Coote. It doesn’t get any better. @sharpnetwork

The extremely intelligent, very kind @jamieljones, a total honor meeting her. She is an inspiration to anyone working to get their digits.

Wireless guys loving on wireless. @jsnyder81

Wireless guys loving on wireless. I find it’s better not to ask… @jsnyder81

Awwww, these guys... @tonhe @ciscovoicedude

Awwww, these guys… @tonhe @ciscovoicedude

The fabulous @pilotmike, oddly without his Blackberry. @denisefishburne

The fabulous @pilotmike, oddly without his Blackberry. @denisefishburne

FullSizeRender (15)

Anti-social social dining. @avalonhawk @scottm32768 @radzima

Because network engineers ARE superheros. @scottm32768

Because network engineers ARE superheros. @scottm32768

Behold @vcabbage, the self proclaimed pretty, pretty princess.

Behold @vcabbage, the self-proclaimed pretty, pretty princess.

Some look better in skirts than others, just saying... @kathleenmudge @wifijanitor @subnetwork @ucgod @fryguy_pa @lauren @denisefishburne

Some look better in skirts than others, just saying… @kathleenmudge @wifijanitor @subnetwork @ucgod @fryguy_pa @lauren @denisefishburne

Yeah, what can I say about this? ;) @grinthock @Rob_Coote

Yeah, what can I even say about this? @grinthock @Rob_Coote

The incredibly smart, incredibly witty @drjmetz.

The incredibly smart, incredibly witty @drjmetz.

The expressions are priceless. @kathleenmudge @lauren @ucgod @wifijanitor

The expressions here are priceless. @kathleenmudge @lauren @ucgod @wifijanitor

Usually it's his backside people are posting... @networkingnerd

Usually it’s his backside people are posting… @networkingnerd

These two crack me up. @scottmorrisCCIE @denisefishburne

These two crack me up. @scottmorrisCCIE @denisefishburne

I love that @samplefive is just having a normal conversation in the background. @wifijanitor @fryguy_pa

I love that @samplefive is just having a normal conversation in the background. @wifijanitor @fryguy_pa

Social media gurus @kathleenmudge @lauren

Sweet and amazing social media gurus. Thanks for all you do. @kathleenmudge @lauren

These two are fabulous. But don't tell them I said so... @fryguy_pa @subnetwork

These two are fabulous. But don’t tell them I said so… @fryguy_pa @subnetwork

Kilts.  There were kilts. (Why??!!) @subnetwork

Kilts. There were kilts. (Why??!!) @subnetwork

It's importance to have balance in your networking career.  @denisefishburne @nmarus

It’s important to have balance in your networking career. @denisefishburne @nmarus

FINALLY this guy decided to join us for a Cisco Live. About time... @JTIE_6EE7

FINALLY this guy decided to join us for a Cisco Live. About time… @JTIE_6EE7

Because UC engineers rule.  @mlundbom1 @ucgood

Because UC engineers rule. @mlundbom1 @ucgood

The awesome ginger princess @jay25f with the fantastic @denisefishburne.

The awesome ginger princess @jay25f with the fantastic @denisefishburne.

Awwww...thanks @drjmetz for this pic. @networkingnerd

Awwww…thanks @drjmetz for this pic. @networkingnerd

The @ucpappy trying to collaborate on some new fangled device. @ucgod

The @ucpappy trying to collaborate on some new fangled device. @ucgod

The caffeinated side of the social media hub.  @fryguy_pa @bbaize @wifijanitor

The caffeinated side of the social media hub. @fryguy_pa @bbaize @wifijanitor

The non-caffeinated side of the social media hub. @bcjordo @hankito @tonhe

The non-caffeinated side of the social media hub. @bcjordo @hankito @tonhe

Completely impressed with the technology aid @CiscoTACOPS brings to disaster situations. @aconaway @bcjordo @densaer

Completely impressed with the technology aid @CiscoTACOPS brings to disaster situations. @aconaway @bcjordo @densaer

Networking - it's about community.

Networking – it’s about community. And this community is incredible.

How we all felt by the week's end. @lauren

This sums up perfectly how we all felt by the week’s end. @lauren

Published 6/19/2015


Posted by on 2015/06/19 in Cisco Live, Cisco Live 2015


Tags: ,

Voice basics: troubleshooting a failed outbound fax

Faxing is a technology that instead of nuking it from orbit (the only way to be sure), we’ve propped it up and tried to make it part of the VoIP world, resulting in a whole lot of troubleshooting and whole lot of bang-head-here moments for voice engineers.

While time, variances in equipment, and sheer PTSD keep me from exploring all the ways in which faxing can suck go wrong, I thought I’d throw out a recent example of an all too common occurrence – proving your fax machine isn’t the (biggest) offender in an outbound communication failure.

Specifically, this example deals with an XMedius fax server, a Cisco voice gateway with PRI, and a who-knows-what fax endpoint on the other side.  Your mileage in fax troubleshooting may and likely will vary, just keep that in mind and a drink at hand.

The first step in dealing with one of these reported issues (after cursing, of course) is to determine if it’s an isolated incident or possibly a dialing issue.  Besides calling and confirming* a fax machine actually picks up, checking your inbound and outbound logs on the fax server can quickly quell those reports the server is down when someone really forgot to dial a 9 when sending the fax. Happens all the time.

In my case, I had plenty of inbound/outbound successes to determine this was an isolated case.  I also had the packet capture feature of XMedius turned on.**

This feature is brilliant, truly not an understatement.

I opened the packet capture for one of the failed attempts, navigated to Telephony -> VoIP Calls -> and then selected Flow for my call.  When you do this, there will be quite a bit of information presented in graph form.

You should be looking for a few basic things in particular:

  • Do you see the call ever connect?
  • Do you see the sender’s cng (calling tone) packet?
  • Do you see a DIS (Digital Identification Signal) from the remote endpoint?
  • Do you see the sender’s training message?
  • Do you see the remote endpoint’s CFR (confirmation to receive)?

In my flow graph of the not-so-happy fax, I notice that even though I’ve made contact with the (whiny) fax machine on the other side and negotiations have been successful – the remote endpoint never sends a CFR, therefore the server will not send the fax data.

The fax server tries again and again to elicit a response, but there’s only silence from the other side.  I assume because the remote endpoint realized that for every successful fax, a puppy dies.  Well, that’s the rumor I’ve heard (or started).

Here’s an excerpt from the flow graph, definite lack of CFR.


Below is flow graph of a fax that the server sent successfully to another number.  While there are differences, you can see that CFR goodness the flow graph above is missing.

Successful Fax

After reviewing this information, I moved onto finding out if the voice gateway ever sees the CFR and maybe just forgets to send it along.

After working with TAC and doing a PCM capture on the gateway, I was able to confirm that the remote endpoint never sends the CFR, which meant I could declare with some amount of relative certainty that this was a whole lot of not-my-problem.***

TAC even provided me this handy-dandy flow graph built from the captures we took on the gateway, you can see that the fax server tries three times (TCF (9600)) to get the remote end to cough up a CFR, but no dice.

outbound fax flow

While this just scratches the surface, these basics, along with a formidable hammer, should get you started in your fax fighting mission. Just remember to really effectively troubleshoot a fax machine, it’s all in the swing…


Published 4/10/2015

*Do not skip this step. Never assume a user is asking you about problems with a working telephone number.  Always test from outside your phone system to confirm that the phone number in question hasn’t been disconnected or written down wrong by the user. This will save you countless hours and possibly what’s left of your sanity.

**Check your XMedius Administrator’s guide or call their support for steps to turn on this feature, it’s a pretty straightforward process and well worth the time.

 ***Trust me there are no absolutes in fax, unless you’re talking about frustration, that part is guaranteed.


Tags: , , , , , , , , ,

Short and sweet – how to block an incoming call on your voice gateway

Welcome to a quick post on how to block an incoming call when you know the calling number you want to block. Specifically, this is how I would block an incoming call on a Cisco voice gateway with an ISDN PRI attached. Your mileage might vary a little with SIP trunks and will definitely vary quite a bit with MGCP.*

The first thing you need to do is create yourself a voice translation rule, something like this ought to do the trick:

voice translation-rule 9
rule 1 reject /5550005555/   <<keep in mind this is the calling number you want to block, but I like to test initially with an outside number such as my cell phone that I can test with.

Then set yourself up a lovely translation profile that references the rule you just created. Name it something obvious so that the next administrator doesn’t have to beat you to death for your obscurity:

voice translation-profile CALLBLOCK
translate calling 9

To complete the configuration, add these two commands to your incoming POTS dial-peer.  If you aren’t sure what your incoming dial-peer is, use the debug voip dialpeer all command and make a test call.  This is a good idea even if you think you know what the inbound dial-peer is because sometimes life is whimsical, and dial-peer configurations even more so.

dial-peer voice 4445 pots
call-block translation-profile incoming CALLBLOCK
call-block disconnect-cause incoming unassigned-number

There are a few ways to test this.  As I mentioned before, you can use your own cell phone number in the original configuration and confirm that the call blocking works. Then just substitute the to-be-blocked number into the voice translation rule.

You can also run the following command and see what the router *thinks* it will do when it sees the number you are trying to block:

test voice translation-rule 9 /5550005555/
/5550005555/ blocked on rule 1

As with all things voice, there are eleventy-billion ways to accomplish a task, this post just covers one.  If you have another method you prefer, please share in the comments, would love to hear it.

Published 03/10/2015

*The process with SIP trunks is practically the same, your inbound dial-peer won’t be POTS, though.  MGCP will require you to use CUCM 8.0 or later for this, check out this document



Tags: , , , , , ,

Changing your Unity Connection SMTP domain

Changing the SMTP domain on a Unity Connection server really isn’t that big of a deal, but as with all things voice, no change works as initially advertised.  Previously, I had never had cause to mess with the SMTP domain address, but recently one of the major cell providers quit delivering our voice mail message notifications to devices and, not surprisingly, users were none to happy about it.

My guess was that the carrier in question didn’t much like the format of the sender address, since it included a sub domain:  I quickly decided that changing the SMTP domain on the server would easily test that theory and seemed far less painful than opening a ticket with a large service provider*. I did open a TAC case just to see if there were any caveats in making this change I might want to be aware of. That’s a whole lot of voice experience talking…err, writing. The voice paranoia runs deep for a reason.

TAC indicated that not only was this a simple change as thought, but that only one service would have to be restarted – the Connection Conversation Manager, and that wasn’t to be a big deal. Well, finding that hard to believe**, I proceeded to make said change and found that there’s a little more to the story.

First – there are *three* services that have to be restarted, and since two of them are critical services, you experience a failover if you are running in HA. The system does warn you this will happen, and for what it’s worth I did not experience a loss of service doing this. Certainly don’t blame me, though, if you do have an outage and aren’t in a maintenance period when you attempt this change.



Of course, you get to rinse and repeat on the secondary server in an HA environment. Personally, nothing about the warning prompt I got on the secondary server indicates this is not a big deal, but hey, maybe that’s just me…

smtp-connection conversation manager secondary

That being said, once the services were restarted, and my blood pressure returned to normal, I expected to see the SMTP domain updated and a happy dance to ensue.  Alas, that was not the case.  After consulting with TAC, and without the least bit of surprise whatsoever, I found that a reboot was “sometimes”, infer always, required.

This did fix my problem and the delayed happy dance was epic. And thankfully not recorded for the sake of my remaining pride.


Published 1/26/2015

*Almost all levels of hell are more pleasant than opening a case with a carrier. I suspect Satan actually admires the ingenuity of carriers and any future levels of hell are modeled on their expertise and innovation in human suffering.

**There is seemingly far more proof of the existence of the Loch Ness Monster, Big Foot, and the Abominable Snowman than of something involving voice being “easy”


Posted by on 2015/01/26 in Unity Connection


Tags: , , , ,