Before we get started, some relevant references:
- Understanding the Basic Configuration of the Adaptive Security Appliance (ASA)
- JDSU - PVA-1000 VoIP Network Analysis Suite
- CAT5E/6 Impedance/Return Loss (PDF)
- Using Wireshark to Troubleshoot (VoIP Deployment)
- Expose VoIP Problems Using Wireshark
Readers: Never, never, never split any data local area network (LAN) pairs.
Splitting any data LAN connection or twisted pair cabling system (CAT 5, CAT 5E, CAT 6, CAT 6A, etc.) violates the premise of structured wiring. Then, the cable is unable to perform ideally for voice over IP (VoIP) applications and under traffic loads because the split pairs degrade return loss and the cable dielectric is changed with the compromise.
You've hit a major find, but you have a site that "tested clean" with JDSU. Keep in mind that sometimes the test isn't going to catch everything -- I'm still not rattled that this network tested "clean" with any network analysis. Keep looking, because if they were smart and cheap enough to cut corners on the cabling, then they've probably cut corners somewhere else. Make sure the switches are managed and support Quality of Service (QoS), Class of Service (CoS) and virtual LANs (VLANs).
While you're at it, take five to 10 minutes to inspect a couple of the physical cables, faceplates (inserts and jacks) and patch panels. Verify the cable jackets are CAT 5 or better and then verify the type of cable they used. Did they use stranded copper wire? IF they did, then your troubles have just begun.
Stranded copper wire is for patch cable use only, and don't let anyone tell you or sell you differently. Unshielded Twisted Pair (UTP) cabling uses solid twisted copper wire for vertical and horizontal concealed wiring. Usually, the stranded wire looks and feels just like a patch cord because the outer diameter is bigger than solid UTP cables. If in doubt, rip it open -- if the copper is stranded or braided, then you need to deploy new cabling.
After you determine what kind of cabling you have, examine everything else -- patch cords, patch panels and faceplate inserts. If it looks cheap or haphazard, replace it. Don't use hand-crimped patch cords, either -- buy factory-made patch cords and cut handmade patch cords in half before throwing them in the garbage. Stick with one brand for the patch panels and inserts for consistency.
Do all of this before you fix or replace the existing cabling, because you are probably in for some more work to add cables, since the initial premise of splitting each cable's pairs was to save money on cabling. Once you have an infrastructure assessment, then you should have a plan to fix the wiring. If you do buy new cable, forget PVC, just use plenum-rated solid conductor UTP (CAT 5E). Teflon-rated cables perform better -- there's a reason Teflon is used, and it's not just because of fire code. (That's a myth.)
Now, you said "hosted." What are the ping times to the host IP address? Do an hourly check for a day at look at the latency. During those same ping tests, do a traceroute to the host IP hourly, checking to be sure that the host is not failing over to India or overflowing traffic to some far-out place.
Next, check the pipe provider for the wide area network (WAN). Is it dedicated? These pooled Internet WAN connections being hawked by bypass and third-party arrangements with landlords really bite. You may have a 1 Mbps or 2 Mbps pipe, but it doesn't perform like it because it's pooled and there are numerous traffic loads on the tenant switch.
Break out the specs on the Polycom phones and talk to the host to make sure that compression schemes match up, too. Your router should have QoS setup for outbound only, giving voice priority over data. You didn't mention how many phones and what kind of WAN connection you have, so you will need to ensure that you have ample bandwidth to support the phones (simultaneous calls) and data needs over that WAN connection.
But your work is only beginning. Set up Wireshark and trace packets. You may need to setup a mirror in your switch. I'll bet you're dropping packets, so if you fix the cabling and get it in shape, maybe your troubles will go away. Be very methodical in your approach and avoid shotgunning, but my guess is that you will have more opportunity once you start unraveling what was installed previously.
Another note: While I've given you several areas to look and told you that splitting pairs will cause you a lot of pain, it's assuming that the host is just like the phone company -- perfect, right? Hosted telephony solutions range so wildly that I would consider the provider, too. Before you do that, though, fix the cabling and then go over the network with a sharp eye.
This was first published in December 2010