A tale of two phones

Manager comes in and asks me if I have any tasks the shiny new intern can observe, my response: I’m about to troubleshoot a Call Forward All issue- if the intern doesn’t have anything more interesting, he’s welcome to watch.  And to my complete dismay, he really doesn’t have anything more interesting and I have an audience.

First step: gather the facts.  Call the user’s extension, the phone rings- super.  Set the call forward all- super.  Call the user’s extension- get busy signal.  Not so super. Okay, Houston, we’ve verified there is a problem.

Second step: check the obvious.  In this case, checking the CSS of the Call Forward All of the Directory Number is the place to start.  If no CSS or the wrong CSS is set, the call that’s supposed to be forwarded to Timbuktu isn’t going to get there no matter how much you will it to do so. So I verify the CSS is correct, and I give a lengthy explanation of CSSs to the intern who, once again to my dismay, has not fallen asleep yet.

Third step: isolate the issue- This is also known as the simplify-the-issue stage or eliminate-all-the-factors-you-can stage.  In this situation, I whittle down the problem to it’s most basic parts- taking the directory number, which was a shared line, isolating it to a known good test phone. Also, I confirm the number I’m forwarding to is an internal, reachable, working number. I repeat the tests. And for the love of Pete, it still doesn’t work.

Soooo, thinking I’d like the intern to think I’m smarter than a potato, I blow through the rest of my bag of tricks, including using the Dialed Number Analyzer to confirm my call is taking the expected path.  I also proceed to rule out any PBX conflicts.  As a side note, if you have customers that run a Cisco voice system integrated with a legacy PBX, it is perfectly acceptable to blame the PBX for all issues.  They are like telco carriers, it’s the right thing to do.

While I still have what is left of your attention, let’s move along.  At this point, I hang my head in shame, admit defeat, and do what all good mentors do – show the intern how to collect logs and open a TAC case.

And TAC’s findings on the source of my malcontent?  Bug ID CSCse19548.  Which, for those who would rather I google that for them, the essence is there’s a counter that is supposed to prevent looping calls and it has been triggered.  This counter tells the Directory Number  “you are a loop!! no call forward for you!!” (call forward Nazi…)

The fix: reset the counter.  You can do this two ways: the take-a-hammer-to-it approach and stop/start the CCM Service, temporarily taking down call processing on your CUCM server, OR the slightly milder approach, you can reset the max forwards counter using the phone itself:

-Go off hook on a phone that is registered to CCM that has the high counter, make sure the CallFwdAll is set
-enter “**##*30 (enables codes)
-Go off hook again
-enter “**##*35 (clears the max hop counters)

By the way, if you are wondering what this issue looks like in the log files, you are looking for something like this:

06/17/2011 11:25:21.637 CCM|Forwarding – processCFA – ERROR –
Forward loop detected.  — Clear the call with USER_BUSY. callKey=

Final thoughts:

The last thing I will mention I learned from this situation is that users rarely tell you all of the story.  If you’ve ever seen House and heard his rant on “all patients lie” it’s similar in concept. In this case, I later found that the directory number in question was once part of a group of two directory numbers in which two very nice, very old ladies, kept call forwarding their phones to each other.  You guessed it, creating a call forwarding loop!  Just shortly before this CallFwdAll issue had been discovered, the hooligans had been assigned a single directory number to share which put a stop to their shenanigans. Reminding me to always, always, ask questions.  And lots of them.