The analog mixup…

While it is perfectly acceptable to have a shared line that is assigned to both a voip phone and an analog device, there is no guarantee things won’t get wonky.*  Let me demonstrate with a recent example I came across.

Users were reporting that occasionally when calling an extension they would get fast busy.  A quick look up in CUCM showed that the directory number was assigned to several phones and two ATAs. Cue huge sigh.  For those not familiar, ATAs all provide the excruciating pains of troubleshooting analog, in a tiny, fits-in-a-bread-box, device.

Immediately suspecting one of the ATAs, each connected to a classic 5.8 GHz cordless analog phone, I did a quick check to be sure the tiny boxes of evil darkness (TBEDs?) were registered. After confirming ports on both ATA devices were up basic settings in check, I decided my time would be best served with some log file collection.

For those who find themselves needing to examine CUCM trace files, I highly recommend Translator X.  It’s free, easy to use, and super helpful.  After looking through the Call Manager trace file results in Translator X, I could see clearly that each time the busy occurred, the call had been sent to one specific ATA, and logs showed a corresponding Disconnect Cause of “(47) Resource unavailable, unspecified.”

ATA

“Unspecified.” Yep, vague error messages are my favorite. Not able to find anything tangible in the CUCM configuration, I opted for an onsite visit.

My complicated troubleshooting plan was to just unplug the troublesome ATA and let the good times ring in.  After locating and confirming the MAC address of the ATA, I pulled the plug.  Like any good network/voice engineer, I started to make a test call to confirm whether I had indeed fixed the situation without breaking anything else – an art form we constantly strive to perfect.  Fully expecting that all would be well, I was quite disappointed when greeted once again by a fast busy.

Time to pull out Tippy.

photo1

Tippy, (yes, a reference to tip and ring, I am still that much a voice geek), happens to be my reliable test analog phone and every one should have a Tippy.  Reliable test equipment will save yourself time and sanity, and you should always be weary of results from equipment you haven’t base-lined.

Since Tippy and I go way back, I knew after making a successful test call with Tippy plugged into the ATA, that something was wrong with the cordless analog phone itself, or possibly in its wiring back to the closet.

I then found that plugging Tippy into the wall jack of the problem phone resulted in a successful call as well, so my conclusion was that the cordless phone at hand had shuffled off its mortal coil. I opted for moving the other cordless phone to replace the bad one, the models being identical and the other one being in a location that wasn’t being utilized.

Swap complete, time to test again, and still getting a fast busy. A girl cannot catch a break.

On a hunch, I decide to press the handset locator button on the cordless phone base station.  Oddly, the handset on the charger station in front of me was not the handset that was beeping.

Then it hit me, the users had mixed up the handsets.

The base station didn’t have a working handset, but was still alive and would take the call the ATA presented it, but the handset couldn’t communicate because it was around the corner, down the hall, and IN THE OTHER ROOM!

Like I said, wonky.

*Yes, that is in fact a technical term, just ask any voice (or former voice) engineer.  

Bonus material:
If you have never had to manage an ATA, (lucky you), you might not know that typically you can reach the ATA device via a web page by entering ://[the ip address]/dev.  You will see a web gui that I am certain was coded by a developer with absolutely no sense of web design. Seriously, not even a bit.

ATAConfig

Published 6/19/2013

The Mole Truth

Troubleshooting FXO lines can be like trying to tie your shoe laces with your teeth.  It’s not that it’s impossible, but you know there must be an easier way. Let me tell you, telecom carriers are not here to make this cumbersome job any easier.

Take for instance a recent case in which debugs showed that the carrier was dropping calls coming into an FXO port.  The symptoms were ones I had seen before – basically users hear a half ring when a call comes in and then, try as they might, they can’t answer the call before the line goes dead. Then they hear another tormenting half ring, and again silence when answered.

Having ruled out the notion that someone was playing one fabulously entertaining and clever joke on poor unsuspecting users, I spent quite some time placing test calls and running debugs the first time I encountered this issue. After some quality time with TAC, I turned the issue over to the carrier, confident the problem was on their end.

In the case at hand, I spent the same amount of time placing test calls and debugs, spent a little less time with TAC, and then once again turned the issue over to this customer’s local carrier, confident something was a muck on their end.

At that’s when the games began. Again.

Granted, I am going to be a bit facetious with these, but I know I am not the only one who has had this experience more than a few times…

Rule one of carrier technicians- never admit there is anything wrong.
Here’s how it works:
Assigned technician immediately claims there is no issue at all, everything is just fine.
Uh, so when you call this number the call goes through for you? No. Well, you see, that’s the problem…

Rule two of carrier technicians- make a token appearance on site.
Here’s how it works:
Unbeknownst to you, technician arrives on site, makes bold claims that all things carrier side work fine, and leaves.  Technician calls and tells you about it later.
Wait, are you sure?  Did you get dial-tone up to the demarc?  Oh, you didn’t check for dial-tone…yeah, we need you to go back and confirm that…

Rule three of carrier technicians- make second on site visit, use test equipment, resolve issue.
Here’s how it works:
Technician begrudgingly gets out test equipment and uses it to tone out the line.  Finds moles.
I’m sorry, did you say moles?  Yes, ma’am, moles. A whole family of ’em.

Okay, so maybe finding moles isn’t the rule, but it certainly was an entertaining exception.  You see, the half rings were a symptom in both of my cases of damaged copper wires.  In the first case, the wires had been terminated poorly and exposed.  In the second, a family of moles had decided copper wires make the best houses.

What did I learn from these experiences?  One, tests calls and debugs are your friend.  Two, the carrier technician isn’t.  That is stated a bit harsh- I do have it on good authority that carrier technicians are people too and not in fact spawned from evil alien overlords…BUT a good engineer has to have a willingness to be an advocate on their customer’s behalf. Even when told repeatedly everything is fine- just fine.

In my cases, I had to insist the technician tone out the lines – all other indicators on the carrier side were telling the technician things were fine.  However, once the wires were toned and traced, there was no time wasted getting the lines repaired.

So don’t be afraid to ask the technician to take that second look.  Ask for the results of whatever packet capture that has been done or test that has been performed.  Verify that proper troubleshooting was done – but handle it professionally, because you never know when those debugs, tests, and traces are going to show you missed something.  And then it’s your words you get to eat, so make sure they aren’t sour.

Published 04/12/2012

Hey, voice gateway, that’s *my* call!

It’s not often that people ask me to keep calls from being answered by a voice gateway, but recently I got privilege of addressing just that issue.

The scenario here is not all that common, but if you should find yourself in similar jam, you are going to want to send me candy and flowers for this tip.

Say that you have a voice gateway and the only reason it’s there is so that some archaic fax machine can send and receive (die, fax, die!!), and so that outgoing user 911 calls are routed out the local PSTN.

Say that for whatever reason, an FXS card was not used for the fax machine, but instead, wiring wizards split off the PSTN connection at the demarc so currently both the router and the FAX machine share dial-tone from the same line.

Say that you determine not to question the setup so much as focus on making sure that when calls come in from the PSTN, the FAX machine takes the call and the router just sits there, pretending like nothing’s going on.

Here is the magical command you need to make this happen:

voice-port 0/0/3 (or whatever voice-port your PSTN connection is cabled to)
ring number 10

This sweet little command tells the router not to pick up when the call rings in initially.   This gives the FAX machine plenty of time to jump in there and take the call without the router interference.

Granted you won’t see this everyday. If you’re lucky you may never see it.  I, on the other hand, need to have a long talk with karma…

Be sure to check out this support forum, which led me to the above solution: https://learningnetwork.cisco.com/thread/9357

Published 03/26/2012

Runt post: A little VG224, MGCP annoyance…

While I am not a fan of MGCP for general gateway setup, I agree that it’s a good protocol to go with when setting up a VG224 device.  For those not familiar, you can use a VG224 device to connect multiple analog devices to your VoIP network.  Keep in mind that each analog device you add to your IP network makes a baby cry, but if you’re going to do it anyway and have quite a few analog stations, these devices make sense.

Up until recently, I had never had the privilege of configuring one of these guys from scratch.  Like adding any MCGP gateway, though, its a pretty straight forward process, so imagine my surprise when my VG224 wouldn’t register with call manager.

I saw the Unregistered status in Call Manager, and I also saw this on the gateway when I did a #show ccm-manager from the CLI. The status stayed in Registering toward the primary, then would occasionally swap to Registering toward the secondary. What had I forgotten?  Most commonly the mistake is to forget to use the fully qualified domain name as the device name in Call Manager. If “ip domain name” is set on the router, be sure to include the domain name as part of your device name when adding your VG224 to CUCM.  (Pro tip: You can see what the FQDN name should be in the #show ccm-manager output.)

This wasn’t my mistake though.  My “mistake” was simply this – I was trying to get the VG224 to register, but I hadn’t added any configuration to the ports.  My thought was “let’s get this thing registered, then I’ll go back in and add the directory numbers to the ports” – the VG224’s thought, however, was “I’m not registering until this chick defines some ports.”

Alas, after about 20 minutes of checking the configuration against known good configurations, I decided to proceed with adding the directory number information to the ports and worry about the registration at the end.  After I configured the first port, though, the VG224 starting showing as Registered in Call Manager and I proceeded to kick myself for wasting my own time.

Here’s what you *should* see in the output of a properly registered VG224 using MGCP, notice the Domain Name, this is what needs to be added in Call Manager for the device name:

My_VG224_#show ccm-manager
MGCP Domain Name: My_VG_224_my.lab.com
Priority        Status                   Host
============================================================
Primary         Registered               10.10.10.1
First Backup    Backup Ready             10.10.10.2
Second Backup   None                     

Current active Call Manager:    10.10.10.1
Backhaul/Redundant link port:   2428
Failover Interval:              30 seconds
Keepalive Interval:             15 seconds
Last keepalive sent:            14:12:27 CDT Apr 25 2002 (elapsed time: 00:00:14)
Last MGCP traffic time:         14:12:27 CDT Apr 25 2002 (elapsed time: 00:00:14)
Last failover time:             15:33:26 CDT Apr 11 2002 from (10.10.10.2)
Last switchback time:           15:35:01 CDT Apr 11 2002 from (10.10.10.1)
Switchback mode:                Graceful
MGCP Fallback mode:             Not Selected
Last MGCP Fallback start time:  None
Last MGCP Fallback end time:    None
MGCP Download Tones:            Disabled
TFTP retry count to shut Ports: 2