In a previous tip we looked at a key VoIP environmental variable, the vocoder, which must be included when trying...
to assess call quality. Tools that correlate vocoder type with impairment statistics, especially statistics that focus on burst loss and jitter buffer discards, will provide a very good sense of call quality. But to understand the end-user's opinion of call quality, which many think is the ultimate goal of application or service management, you need a way to measure the impact of how the end user experience changes from good quality to bad, and the reverse, and of how the end-user remembers the disturbances caused by network impairments.
Humans do not experience impairments instantaneously. There is a lag time, generally on the order of five to 10 seconds for a transition from good to bad quality. Interestingly, it takes people a bit longer to recognize the transition back to good quality. That lag time is on the order of 15 seconds. This underscores the fact that there is a difference between instantaneous quality as "seen" by the network and subjective quality as experienced by the end-user.
There is also the issue of how the human auditory memory system works. Generally, auditory memory decays exponentially over approximately a 30-second period. This is typically called the "recency effect", which essentially states that people remember more vividly events that occurred more recently than those that occurred in the more distant past.
Though a fairly intuitive postulation, telephone companies such as AT&T performed tests where subjects were asked to listen to and rate 3 one-minute calls, each with noise inserted in evenly spaced, different points in the call. In call one the noise was inserted near the beginning; in call two in the middle; and in call three near the end of the call.
The call with the earliest nose insertion was rated the best with a MOS (Mean Opinion Score which typically comes from human subjects with scores that range from 1-5 rating) almost 40% higher than the call with noise inserted towards the end of the call. To account for this significant impact on user-perceived call quality, tools must be able to "weight" the impact of impairments according to where they occurred during the call. Without such a weighting, subjective call quality assessments can be off by almost 40%.
Bob Massad is a VP at Telchemy, Inc., a provider of voice quality management solutions.