decision analysis partners (dap) was selected by GSA to lead the testing of Networx contractors’ ordering and billing systems to ensure and certify that each contractor’s operational support systems (OSSs) would work correctly for their Networx service offerings before a contractor could accept and process Networx orders.
When FTS 2000 and FTS 2001 were first implemented, GSA and its agency customers encountered serious problems with the ordering and billing systems of the various contractors. In an effort to avoid similar difficulties, the Networx RFPs and resulting contracts included requirements for OSS Verification Testing to be conducted. While the basic requirements of this process were specified, the details had yet to be developed.
Drawing on its experience with communications engineering, systems development, testing, service enablement and revenue assurance, dap developed a repeatable testing process that ensured equal treatment of all Networx contractors. The first step in this process required each company to submit a comprehensive OSS Test plan. dap used a structured review process to evaluate each plan, and provided feedback to Networx contractors who made revisions necessary to make their plans compliant. Based on our experience with all Networx Service Types and Services including switched voice, Internet, cellular, land mobile radio, VPN, Security, Access, Satellite and others, dap developed over 250 test orders that exercised all Networx services and representative CLINs for each service offered by each contractor for CONUS, OCONUS and international locations. Access and provider options, NSEP restoration priorities and coordinated orders were also exercised within appropriate order scenarios.
Test scenarios included transition of legacy services from previous contracts, transition of legacy services to new technologies, orders for new services, and modifications to existing orders. For usage-sensitive services, dap generated over 20,000 test records to exercise contractor rating and billing systems effectively. Orders covered multiple billing cycles to ensure changes and adjustments were made accurately. Complex databases were created to generate, track and manage test data, test cases, and
As part of this process, dap worked closely with several agencies including GSA, DISA, the Department of the Interior, the Department of Energy, and the Administrative Office of the U.S. Courts to ensure that test orders, locations and conditions were realistic and addressed agency ordering and billing requirements and concerns. In addition to test orders prepared by dap, dap conducted and evaluated “Live Testing” using various agency volunteers who used actual orders for complex situations.
RESULTS
The dap-led OSS Verification Testing revealed 499 serious defects in the OSS systems of the five Networx contractors. The contractors were required to correct all such errors before accepting Networx orders. A large number of cosmetic errors were also identified and corrected.
This process eliminated the errors that would have otherwise occurred in production Networx orders as a result of these defects. Eliminating such defects saved government personnel and Networx contractors thousands of hours analyzing and correcting ordering and billing errors. The result: Networx orders can be processed quickly and billed accurately, saving millions of taxpayer dollars.
