The Mole Truth

Troubleshooting FXO lines can be like trying to tie your shoe laces with your teeth.  It’s not that it’s impossible, but you know there must be an easier way. Let me tell you, telecom carriers are not here to make this cumbersome job any easier.

Take for instance a recent case in which debugs showed that the carrier was dropping calls coming into an FXO port.  The symptoms were ones I had seen before – basically users hear a half ring when a call comes in and then, try as they might, they can’t answer the call before the line goes dead. Then they hear another tormenting half ring, and again silence when answered.

Having ruled out the notion that someone was playing one fabulously entertaining and clever joke on poor unsuspecting users, I spent quite some time placing test calls and running debugs the first time I encountered this issue. After some quality time with TAC, I turned the issue over to the carrier, confident the problem was on their end.

In the case at hand, I spent the same amount of time placing test calls and debugs, spent a little less time with TAC, and then once again turned the issue over to this customer’s local carrier, confident something was a muck on their end.

At that’s when the games began. Again.

Granted, I am going to be a bit facetious with these, but I know I am not the only one who has had this experience more than a few times…

Rule one of carrier technicians- never admit there is anything wrong.
Here’s how it works:
Assigned technician immediately claims there is no issue at all, everything is just fine.
Uh, so when you call this number the call goes through for you? No. Well, you see, that’s the problem…

Rule two of carrier technicians- make a token appearance on site.
Here’s how it works:
Unbeknownst to you, technician arrives on site, makes bold claims that all things carrier side work fine, and leaves.  Technician calls and tells you about it later.
Wait, are you sure?  Did you get dial-tone up to the demarc?  Oh, you didn’t check for dial-tone…yeah, we need you to go back and confirm that…

Rule three of carrier technicians- make second on site visit, use test equipment, resolve issue.
Here’s how it works:
Technician begrudgingly gets out test equipment and uses it to tone out the line.  Finds moles.
I’m sorry, did you say moles?  Yes, ma’am, moles. A whole family of ’em.

Okay, so maybe finding moles isn’t the rule, but it certainly was an entertaining exception.  You see, the half rings were a symptom in both of my cases of damaged copper wires.  In the first case, the wires had been terminated poorly and exposed.  In the second, a family of moles had decided copper wires make the best houses.

What did I learn from these experiences?  One, tests calls and debugs are your friend.  Two, the carrier technician isn’t.  That is stated a bit harsh- I do have it on good authority that carrier technicians are people too and not in fact spawned from evil alien overlords…BUT a good engineer has to have a willingness to be an advocate on their customer’s behalf. Even when told repeatedly everything is fine- just fine.

In my cases, I had to insist the technician tone out the lines – all other indicators on the carrier side were telling the technician things were fine.  However, once the wires were toned and traced, there was no time wasted getting the lines repaired.

So don’t be afraid to ask the technician to take that second look.  Ask for the results of whatever packet capture that has been done or test that has been performed.  Verify that proper troubleshooting was done – but handle it professionally, because you never know when those debugs, tests, and traces are going to show you missed something.  And then it’s your words you get to eat, so make sure they aren’t sour.

Published 04/12/2012

Hey, voice gateway, that’s *my* call!

It’s not often that people ask me to keep calls from being answered by a voice gateway, but recently I got privilege of addressing just that issue.

The scenario here is not all that common, but if you should find yourself in similar jam, you are going to want to send me candy and flowers for this tip.

Say that you have a voice gateway and the only reason it’s there is so that some archaic fax machine can send and receive (die, fax, die!!), and so that outgoing user 911 calls are routed out the local PSTN.

Say that for whatever reason, an FXS card was not used for the fax machine, but instead, wiring wizards split off the PSTN connection at the demarc so currently both the router and the FAX machine share dial-tone from the same line.

Say that you determine not to question the setup so much as focus on making sure that when calls come in from the PSTN, the FAX machine takes the call and the router just sits there, pretending like nothing’s going on.

Here is the magical command you need to make this happen:

voice-port 0/0/3 (or whatever voice-port your PSTN connection is cabled to)
ring number 10

This sweet little command tells the router not to pick up when the call rings in initially.   This gives the FAX machine plenty of time to jump in there and take the call without the router interference.

Granted you won’t see this everyday. If you’re lucky you may never see it.  I, on the other hand, need to have a long talk with karma…

Be sure to check out this support forum, which led me to the above solution:

Published 03/26/2012