Part two of Bulletproof IP telephony deployment by Christian Stegh, IP Telephony Practice Leader with Avaya, Inc. provides solutions to performance issues in Layer 2 – the data link layer.
Read Bulletproof IP telephony deployment, Part 1.
Read Bulletproof IP telephony deployment, Part 3.
Many nagging IPT problems occur at Layer 2 and are difficult to pinpoint. Since sniffers
and network management tools aren't typically optimized for troubleshooting Ethernet, it
usually takes extra time and equipment to analyze L2 detail. Add the fact that CAM and
spanning tree tables aren't as frequently accessed as IP routing tables, and Layer 2's
complexity is compounded.
Speed/duplex:
One of the most basic Layer 2 settings that cause trouble is the speed/duplex setting on a
LAN switch ports. If speed/duplex settings are mismatched on a phone or IPT gateway
and its upstream LAN switch, impairments like gaps in speech, one-way audio, and even
resets may occur. Generally, if an interface can be hard-coded for speed and duplex, it should be. However, some older devices may not be hard-coded, and are only capable of auto-negotiation. In this case, corresponding LAN ports should not be hard-coded for speed/duplex, but instead left to auto-negotiate as well.
If access layer ports are to be used as either phone or data (or combined) ports, then the
setting of the LAN switch port depends on the settings of the end devices that will
connect. If PCs and phones are always set to 100/full, then locking the port to 100/full
makes sense, but if the PCs and phones are set to auto/auto, then the LAN port should be
left to auto/auto. The complexity is greater where mobile users plug in and where they
may have access to change their own network interface card settings. If different IT
organizations handle PCs, phones, and/or networks, there should be universal settings
and standardized NIC and LAN port defaults.
Uplinks between LAN switches should not be overlooked. While most GbE ports are
full-duplex by default and can't be changed, all 100 MBPS uplink interfaces should be set to 100/full on both ends of the wire. It is always a good practice to check the interface
for Frame Checksum (FCS) and Cyclic Redundancy Clock (CRC) errors after connecting
the device to the network. On occasion, some Linux and other machines may require
some manipulation prior to properly settling down.
Figure 3 illustrates some correct and incorrect speed and duplex settings.
Spanning tree:
Spanning tree can be the culprit of one-way or total loss of audio when improperly
configured. If spanning tree is being used to protect from bridging loops, the tree should
be well documented to assure optimal traffic flows and to improve troubleshooting.
Document the forwarding and blocking ports of the spanning tree topology for both
standard and all of the possible failure conditions. Make sure that the optimal path
through the network is provided during the standard condition, and make sure that the
secondary path traverses as few hops as possible. If varying network uplink speeds are in
place (i.e. 1G and 100M) to choose from, decisions should be made on how to manipulate
the port costs, if necessary. If spanning tree is left to its standard
blocking/listening/learning/forwarding algorithm, voice calls in progress will lose
packets during the reconvergence period. A fast-start spanning tree algorithm should be applied to relevant interfaces in the network, like ports that service IP phones or
telephony gateway/server cards, in order to speed up the time it takes for a port to begin
forwarding when a topology change is detected. Testing the spanning tree in a lab to
understand its behavior on IPT is recommended.
Virtual LANs:
VLANs offer many advantages for IP telephony, including improved security, QoS,
manageability, and broadcast control. Media servers, gateways and phones should be
placed in VLANs separate from data machines. Unless deployed in a very small network,
IP phones and servers/gateways need not all be in the same VLAN. Typically, routing
voice packets from a phone, through a router, to a server/gateway provides acceptable
performance. If there are several adjuncts, like CTI, call recording, or voicemail servers,
they can be placed in their own VLAN separate from the gateways and media servers,
provided they are routable to devices/clients that need to access them.
It is a best practice to configure an 802.1Q trunk to allow PCs to plug into IP phones and
share a LAN port, but still separate both into their own VLANs. When doing so, prune
off the unnecessary VLANs from the trunk. Carrying extra VLANs can lead to problems
if rogue traffic from an unknown VLAN is allowed to traverse the trunk. For instance,
ports that interface IP phones should only allow the native data VLAN and the voice
VLAN to traverse the link, not extra data VLANs that may be present on the switch.
Note, that if a phone or other endpoint is tagging their frames with an 802.1Q header, and
the LAN switch port is not configured as an 802.1Q trunk, the frames will be discarded
by the ingress port. This would impact both signaling and bearer traffic, so is quickly
noticeable.
Some have felt that VLANs are more trouble than their worth and have delayed adding
an IPT VLAN at first, but have moved quickly to do so after experiencing voice troubles.
It is true that in most networks, a majority of the time, a separate IPT VLAN is not
necessary to provide better voice quality or stability. However, in the event of an
anomaly, VLANs can provide better stability and help deliver the quality demanded by
voice packets. For instance, in the case where a virus or broadcast repeatedly forces
traffic onto a data VLAN, voice packets in their own VLAN would be sheltered and
continue to flow untouched. Voice VLANs are like an inexpensive insurance policy, and
as with life insurance, they're more effective when implemented early.
ATM:
While often forgotten in the IP telephony arena, some LANs and MANs continue to
leverage ATM. When LANE is being used, proceed with extreme caution into IPT. In a
LANE environment, there is no effective way to implement QoS. Once voice packets are
converted to cells, they are no longer prioritized. Congestion in the ATM portion of the
network could cause jitter or other voice impairments that Layer 3 QoS policies cannot
affect. Additionally, with all of LANE' s logical functions embedded on different
hardware components throughout the network, LANE has too many variables to reliably
support IP telephony.
Of course, ATM is still a viable WAN transport. Optimally, Constant Bit Rate services
are provisioned for Virtual Circuits carrying voice, but if carefully engineered, UBR can
work. If UBR has to be used, bandwidth needs to be engineered conservatively and there
should be no points within the ATM network at which congestion may occur at layer two.
Take for example the topology in Figure 4, an example of what not to do. Even if Layer
3 engine in ATM 1 properly prioritizes voice over data on the way to ATM 3, since ATM
3 is unaware of any priority differences between traffic from ATM 1 and 2, data traffic
from ATM 2 can keep voice traffic in the back of the outbound WAN queue. This is
because upon exiting ATM 3, no QoS from Layer 3 is still applicable. In this example of
both voice and data sent out a UBR virtual circuit, there will be congestion that delays
and potentially drops voice frames.
ATM' s cell size requires no major changes to IP telephony engineering principles like
frame size (the frequency at which an IP telephony interface puts a frame on the wire). A
standard 20ms frame fits into ATM' s 53B cell without additional wasted overhead.
MPLS:
Many enterprises are leveraging the any-to-any mesh capabilities of MPLS. MPLS can
only impact IP telephony in positive ways, of which only a couple reasons are mentioned.
First, most MPLS service providers provide options for sending different types of traffic
into different prioritized virtual circuits. From the edge router, voice traffic is typically
sent to a separate Label Switched Path, which is carefully engineered to provide low
latency, jitter, and loss. Thus, MPLS provides an efficient way to blend voice and data
on a single physical circuit. The only issue that some enterprises may have with current
MPLS services is that they typically only provide one high-priority queue. For
enterprises running voice, video, and streaming over the WAN, this presents a potential
problem, since the carrier can't differentiate between the media flows. The MPLS
network is not aware of the individual voice and video flows, it's only aware that there is
more traffic coming in on a high-priority queue. In this rare case, oversubscription or
Call Admission Control are two available solutions to maintain quality of service.
Secondly, MPLS allows variance from the traditional hub-and-spoke model. This can
benefit enterprises that want to have disjointed facilities communicate directly with one
another, instead of hopping through a central data center. For instance, if a large
marketing branch has a full contact center deployed, and a small marketing branch wants
to tap into it, MPLS can directly connect the two remote sites instead of both coming
through the hub. This simplifies bandwidth engineering and the challenge of enabling
QoS throughout the data center core and multiple WAN links.
Quality of service:
Quality of Service (QoS) begins at Layer 2 in the form of the IEEE standard, 802.1p/Q. Even if
a LAN is equipped with 100Mbps to the desktop and Gigabit Ethernet uplinks, there is
still a chance that voice packets could be queued behind data packets at some point.
Therefore, enabling 802.1p is a prudent thing to do. Without enabling QoS end-to-end, it
is very difficult to support reliably good audio. If there are voice quality issues, they' re
often caused by congestion on the interfaces without QoS.
802.1p uses three bits within the 802.1Q header for setting the "User Priority" of the
Ethernet frame. The resulting eight priority levels allow different types of traffic to be
assigned to a different Class of Service (CoS). The classes are usually automatically
mapped to different outbound queues on the switch ports that are configured for 802.1p/Q.
Depending on the switch, globally enabling QoS on the LAN switches often creates the
queues by default, but it is usually required to manually configure the interface for
802.1p/Q. As a convenient coincidence, 802.1p/Q not only enables prioritized voice
frames, but 802.1Q also allows an IP phone and a PC behind it to be in separate VLANs
yet share the same wire to a LAN switch. The user priority bits within 802.1p allow
voice frames to exit the LAN switch toward the phone before data frames are sent to the
PC. Figure 5 illustrates the 802.1Q header, where it fits into two types of Ethernet frames,
and where the user priority bits and VLAN ID reside within it.
Trust:
Some manufacturers' LAN switches either trust or re-mark the frames entering the switch
interface. If the ingress interface doesn't trust the frame, it can re-mark the user priority
bits based on something else in the frame. The re-marking decision may be based on the
VLAN ID, the ingress port number, or if the LAN switch is Layer 3 aware, information
in the IP header may be used to re-mark the Ethernet frame. Allowing the switch to trust
ingress frames or packets reduces latency and processing power that would be required
for the switch to inspect, remark, and then forward the frame. Trusting CoS is often
preferred over trusting DiffServ Code Points (DSCP), since most network administrators
are more concerned about users and applications tagging priority at layer three than at
Layer 2. This is because some applications (i.e. softphones) are often preprogrammed to
mark their frames with DSCP, but most applications are not able to control the 802.1p
settings by default. Trusting CoS requires mapping Layer 2 values to Layer 3 values so
that upstream routers are aware of the prioritized traffic. This subtle technique is
discussed next.
QoS and the transition between Layer 2 and Layer 3:
As a voice frame leaves an Ethernet segment into a router interface, the router can inspect
the IP header for QoS information. Most phones and gateways can mark their IP packets
with L3 QoS information, but the router may be programmed to not trust the IP marking
(as described previously). If the router doesn't trust the L3 marking, the Layer 2 priority
must be mapped to a Layer 3 priority within the router. If this isn't done, the voice packet
may not be properly prioritized on the router's outbound interface or through
downstream Layer 3 hops.
As a voice packet heads out of a router and onto an Ethernet segment, the router should
be programmed to map the Layer 3 QoS setting into a Layer 2 Ethernet setting. This is
especially important for voice packets coming from a remote site into a data center,
where there are likely multiple Layer 2 hops prior to voice packets reaching their final
destination. If voice frames are sent through multiple L2 hops without having the 802.1p
bits set again, they'll apt to be congested in a LAN switch port on the way to their
destination.
The default mapping behaviors of routers can lead to unexpected QoS markings, so
researching and then manually setting the mapping is advised. The best way to validate
that the QoS policy that you' ve designed is actually being carried out by the hardware is
to put a protocol analyzer inline at both ends of the voice stream. Putting the sniffer
inline allows you to see the Layer 2 VLAN tagging and QoS information. Figure 6
shows where the sniffers should go and where to apply the mappings discussed in this
section.
Read Bulletproof IP telephony deployment, Part 1.
Read Bulletproof IP telephony deployment, Part 3.
About the author:
Christian Stegh is currently Avaya's IP Telephony Practice Leader for the North American Region. Among other responsibilities, he acts as a liaison between Avaya's customers/sales teams and Avaya's development teams, influencing direction of Avaya solutions based on customer input. Prior, he was a Managing Consultant within Avaya Global Services, designing, optimizing, and implementing hundreds of converged networks for customers. He began his IT career as a network engineer for a Fortune 200 manufacturing firm, where he managed worldwide L2/L3 networks. His interests not only include multimedia network performance, but also SIP, converged security, and business continuity. Feel free to provide your feedback to him.
