Voice on WLAN trials 2: QoS matters

By David Newman, Network World, 2/08/05

As described in the previous article, we measured the voice performance of wireless LANs from Aruba, Cisco, Collubris and Chantry (now part of Siemens). In this article we explain our findings on Quality of Service (QoS). Elsewhere on Techworld, we have articles on WLAN QoS, progress towards Wi-Fi QoS standards, and moves to provide it for voice.

Is QoS necessary?
There has been much discussion of whether QoS is necessary, given the low bandwidth required by voice. To test this once and for all, we started with QoS disabled, routing all calls through one access point, then added more calls, and background data to simulate other network traffic on the WLAN.

Because all the vendors recommend enabling QoS for voice traffic, this baseline test gave us a "before" picture to demonstrate the need for voice traffic prioritisation.

With QoS turned off, all four systems tested did fine with only a single call active, with R-values (see previous article) hovering around 78. That is about as good as it gets with VOIP over wireless. The threshold for near-toll-quality voice is generally considered to be around 75, meaning the systems delivered good audio quality for a single call.

Performance for all systems changed across the board when we placed six or seven calls through a single access point and switch, especially when data traffic was active. Yet even without background data, we could not test the Colubris system with seven calls active and QoS disabled - all the calls dropped.

Then we introduced background data, from VeriWave TestPoints (a stream of User Datagram Protocol, UDP, packets at 1 Mbit/s), the results were positively awful without QoS. With only six concurrent calls, R-values for all systems (except Aruba) were generally at or below the point where voice signals were unintelligible or calls were dropped. Sound quality through Aruba's system remained high, roughly the same as with no data, even without QoS.

The Chantry and Colubris systems could not perform the background data test with seven calls (QoS disabled). All calls failed as soon as the VeriWave box began offering background data.

All vendors recommend the use of QoS mechanisms for handling voice traffic, even when no data traffic exists. QoS is a must when handling VOIP traffic over a WLAN.

Adding QoS to the mix
We reran the same five test configurations as in the non-QoS cases: We measured one call with no background data, and six and seven calls with and without the 1M bit/sec background UDP traffic.

We expected much-improved results once we enabled QoS, but only Aruba's system put up consistently excellent results in all the tests with QoS enforcement. Even in the most stressful case (seven calls plus background data), the Aruba system delivered near-toll-quality. With QoS enabled on the Aruba equipment, there was little difference between the least and most stressful test scenarios.

Other vendors' QoS mechanisms did little to protect call quality when background data was present. On the plus side, QoS mechanisms generally did an excellent job when only voice traffic was present.

Audio quality improved for all systems in cases where we used only voice traffic. In tests with six and seven calls (no background data), all systems delivered near-toll-quality results with QoS enabled.

That changed when we added the background data. With six calls and data active, R-values fell below 70 for the Colubris CN1250, meaning that "some users [would be] dissatisfied" according to the ITU R-value specification. The R-value was about 60 for the Chantry switch ("many users dissatisfied").

Beyond the objective R-value scores, we did some subjective spot-checking of call quality when data was present. Sure enough, we heard echoes, dropouts and generally poor voice quality whenever the TestPoints offered a datastream.

Things got worse for Chantry, Cisco and Colubris when we tried seven calls plus data. Chantry's BeaconMaster couldn't handle this test case; all seven calls failed when we added data. Cisco's WLSM posted an R-value of about 50, the bare minimum level at which calls are intelligible. Further, 3 of 7 calls dropped during this test on Cisco's gear. The Colubris CN1250 completed the test, but didn't forward enough voice frames for the test equipment to compute an R-value score. R-value scores for this test were only computed for the calls that remained active during the 30-second test ( in Cisco's case, it was on 4 calls instead of 7).

SpectraLink generally recommends a maximum of six concurrent calls per access point, not the seven we used in our tests. Thus, vendors might complain that our seven-call scenario was an overload test case. That is valid, but only up to a point. First, the Chantry and Colubris systems had trouble even with the recommended maximum of six calls with data. Second, Aruba's system could handle the seven calls with data scenario. Third, our most stressful test came nowhere near overloading the wireless medium. We offered 3 Mbit/s of traffic or less in all tests, including voice and data. That's not even near the amount needed to saturate the wireless channel.

It's possible to run each access point with seven calls and data, provided the system is designed for it. But doing so requires careful attention to timing.

Delay and jitter measurements
Delay and jitter are critical metrics for any application, but are especially important when dealing with voice or video. When delay or jitter rises to 50 to 70 milliseconds (ms), voice quality starts to degrade (see graphic). With six calls and background data, the average delay measured below 50ms for all vendors, but maximum delay and jitter shot up to much higher levels, topping out at more than 250ms in tests of Cisco (six calls) and Colubris gear (seven calls).

An analysis of the logs produced by the TestPoints found several reasons for the voice quality degradation. Anytime jitter exceeds 60 millisec, audio quality begins to suffer. As the maximum delay and jitter numbers rose, R-values fell - and that's for the calls that survived the 30-second test. When delay and jitter rose too high, the calls simply dropped.

Doubling the access points
So would throwing more access points at the problem help? We re-ran all our tests on two access points, with half the phones associated to each access point.

With two access points, the R-values were generally much higher. That wasn't surprising, considering each access point does half the work as in the first set of tests. Average delay also increased, which was expected given the additional component in the traffic path.

While this suggests performance can improve with more access points, it also raises several concerns. Cost goes up, even with thin access points. Second, wireless spectrum is limited, and depending on placement, too many access points will interfere with one another. Third, performance still was not perfect with two access points; we had some dropped calls in the presence of background data.

With a mobile workforce you can't predict how many users will try to associate with a given access point at a given time. Every access point has a saturation point, and our results suggest that point is relatively low when voice is added.

Tomorrow: Roaming is an issue.

This story and associated images are copyright, 1995-2003 Network World, Inc.