NGN Reliability Testing

Testing Business VoIP services is one of many important uses for a test automation framework. Ensuring continuous and reliable VoIP services for business customers is the foucs of today's blog entry.

NGN Relaibility Introduction

US businesses expect high percentage (five-nines 99.999%) reliability from their traditional PSTN telephone services. They take it for granted and won't accept poor quality telephone calls while doing business with their customers. In 2009, a significant number of business customers cut their traditional PSTN TDM landlines to reduce costs and adopted new carrier provided business VoIP and IP Centrex phone services. This recent uptake, driven in large part by the economy, requires service providers to pay close attention to VoIP service quality, performance and reliability to retain enterprise customers that have migrated in larger numbers recently to the Next Generation Network (NGN).

The underling IP network and the equipment of NGN are drastically different from the legacy network. The legacy PSTN utilized copper wire, class 5 central office switches, class 4 tandem switches for long distance service, and SS7 for intelligent network services. In contrast, the IP network that underlies business VoIP services is made up of new softswitches, session border controllers, feature servers, gateways, PBXs, and IADs, Routers, and software components that introduce new IP enabled communications features and applications blended with traditional calling features over wired and wireless networks.

NGN Reliability Challenges

A key challenge that service providers face today is delivering the same reliable phone services across IP networks that customers became accustomed to on a more expensive, but very reliable PSTN network.

The "Can You Hear Me Now?" form of NGN testing is a useful method of determining if common business voice services are functioning as needed but only on a small scale

Making a two-way or 3-way conference call, placing a call on hold, call transfer, call forwarding, and listening for good two way voice quality can easily be tested by dialing phone numbers and pressing keys (flashook, or star codes, i.e. *3) to actuate typical business VoIP services or features.

The "Can You Hear Me Now?" form of NGN testing does not scale to effectively maintain continuous high quality, business class reliability, and assure service agreements to today's corporations and enterprise customers doing business across NGN networks.

Test Equipment must be deployed to perform bulk call generation and voice quality measurements - Empirix is a leader in this arena

Effective operation of this test equipment requires a data driven solution to effectively generate the correct instances for all network city pairs, call types, and service parameters

Ensuring that calls signal properly and provide a business class voice path must be performed during installation and continuously thereafter to detect and correct issues that can negatively affect the customer's quality of experience and satisfaction

Network congestion, equipment problems, or software errors are just a few of the defects that can contribute to poor sound quality over NGN VoIP networks
- One way speech path, delay, echo, choppy voice (you're breaking up!) can render business conversations difficult and frustrating to users

Failure to signal calls properly may indicate a network component is failing or service provisioning may not be correct for the feature in use

Test equipment can detect many of these intermittent problems as well as and hard outages if executed continuously and automatically across the networks

Managing all the test equipment and scripts, endpoint data, and large volume of tests results is challenging. Making this information accessible across the organization and keeping in sync with customer data requires an innovative solution

The number of tests that must be executed routinely may number in the thousands requiring a more automated and repeatable solution -- one that is easily available across the entire service provider's organization

The "Can You Hear Me Now?" form of NGN testing must be augmented with systems and the service providers recognize this and are working towards solutions. The following explains some of the major enhancements that today's service providers are now deploying.

Service Provider Wish-List

Service providers must ensure that:

1) Each component is thoroughly tested by their vendors

2) Verify that all components operate reliably as a system and set of billable services

3) Assure that the systems continue to provide the required level of service reliability once deployed

Operations mangers want to simply "push a button and test the entire business VoIP network"
Lab managers need to identify problems with equipment, configuration, design boundaries, stress conditions, and much more to ensure that problems don't make their way into the network

They want tests run automatically and auto-generated reports delivered to users on completion in formats they can easily analyze and share with the proper level of detail needed
- Auto generated reports emailed to selected users -- summary, detailed
- SMS alerts sent to selected users if failures occur and human intervention is needed
- SNMP traps sent to operations NMS systems when selected failures occur
- SOAP Web Services interfaces to generate trouble tickets in the operations trouble management systems and quality management systems

They want to test across disparate communications networks and measure quality end to end
- Data driven, end-to-end tests that cover all service locations and service permutations
- Tests Run on demand after a network change to verify they didn't introduce problems and then run tests continuously as a network diagnostic and measurement probe
- Scheduled test suites and recurring suites of tests run as needed to monitor network performance

They need to not only test NGN networks but also the inter-working across disparate networks
- Public and private IP, NGN platforms, VoIP gateways, Wireless networks, PSTN
- SIP, MGCP, Megaco H.248, Analog, Digital PSTN protocols, Diameter, protocols

The national annual cost estimates of an inadequate infrastructure for software testing are estimated to be $59.5 billion. The potential cost reduction from feasible infrastructure improvements is estimated to be around $20 billion." Source - The Economic Impacts of Inadequate Infrastructure for Software Testing - Study by NIST. Rapid return of investment (ROI) is achieved through a combination of costs savings, cost avoidance, and productivity improvements. Cost savings are achieved though improved efficiencies and cost avoidance is achieved by ensuring that defects are caught and corrected early in the testing process reducing costly deployment problems.

Today's difficult economy requires businesses to do more with fewer resources. This is a challenge that demands flexibility, focus and an innovative approach to business processes. Testing of VoIP, IMS, IPTV, PSTN, and Wireless services is tedious, repetitive, and resource intensive. CallMation's WerkMate enables equipment makers and service providers of these complex new services, to improve testing productivity, implement test information consistency, speed test cycles, and significantly decrease the costs of their development and maintenance cycles. Even more importantly, WerkMate helps to avoid the costs of correcting defects after deployment by ensuring all necessary quality tests are run successfully during the testing cycles prior to deployment. WerkMate enables services providers to more easily honor their service level agreements periodically executing suites of network-wide tests and measurements to assure service quality and availability.

My next blog post will focus another challenge facing service providers today, fully automated Gigabit Ethernet services testing. I'd like to hear your thoughts and comment on these topics.