Centralized test information repository

In my earlier blog I said I want to put the spotlight on the future of testing.  As next gen products and services evolve, the lessons from legacy testing need to be incorporated. 

The next critical issue that I have run up against, in my years in the field of testing, is the data....not the just the high-level test status themselves (Passed/Failed), but the easy access and availability of those detailed test cases, test procedures and test results. 

Many test labs rely primarily on skilled test engineers to prepare and repeatedly run hundreds or even thousands of test cases for each major test cycle, manually gathering and interpreting the results, and consistently reporting problems and progress.  So, the

question arises, how does one take the results of many tests, from multiple vendors' test equipment, which are often in separate silos and locations, and consolidate them into a meaningful report combined with logs and other resulting details from all the devices in the test?

Test organizations must collect and manage important test information manually, typically using Microsoft Office tools or some form of internally developed and maintained test management tool. Manual methods can drastically limit collaboration and reuse, and certainly restrict knowledge base sharing. Unless a concerted, manual effort is made to coordinate and distribute updated and timely test information to everyone on the team, it can be difficult to track and manage valuable test information. Test information can be lost, requiring testers to recreate the test setup from scratch. This can be extremely frustrating and time consuming if accurate and detailed test procedures and associated scripts aren't easily retrievable and usable.

Automation of these consolidation techniques is one part of the equation.  The other is to have a central repository that everyone can access, draw information from, and post reports and information into, thus building a knowledge base over time.  This database can centralize valuable test suites, test cases and test results in an online repository, allowing the data to easily be organized, searched, shared, reused, copied, modified, and compared to previous tests and results.  The single data repository allows testers to catch inconsistencies and bugs that may crop up in new or updated software, no matter when or where the product was tested. This is particularly important when members of test groups are geographically dispersed.

Reports should be able to be easily exported in standard formats, such as PDF, XLS, CSV, XML, and HTML, particularly for managers who are not as close to a test process as an engineer.  This allows them to be able to evaluate and oversee test results that may be taking place in labs down the hall, or overseas! 

A common, centralized database and file system is an idea whose time has come.  Any thoughts or experience with this? Do you have a central repository for this valuable test information in your lab? I'd love to hear from you!

Terry