The development of test data management is an important component to software testing because it allows software developers and test engineers to reproduce and test real data sets representing the underpinnings of a software system or program. Test data management is simply the creation of non-developmentally-performant non-production user data sets which reliably mirror an organization's actual production data so that technical developers can do valid and robust system tests on these test data sets. It is a valuable tool for both quality assurance and maintenance activities. However, despite its value, test data management has several shortcomings that merit some attention.
While test data management has helped decrease the time and cost of software development by providing test data that are consistent and repeatable, it has also created problems with maintaining consistency and continuity within the test data. This is because many of the test data files to become locked in a set of repeating patterns that make it difficult to make any changes to a specific test data file without restarting the entire test. Because automated test suites rely primarily on manual execution for executions, manual execution tends to override the efforts of automated test suites and result in test data discrepancies and failures. While organizations have been able to solve some of these problems by employing manual control methods such as the use of role-based checking, the reality is that manual checking is still insufficient to overcome some of the test data management difficulties.
Another problem with test data management is that it tends to focus too much on concreteness. Test automation software suites are often targeted toward producing simple, executable test scripts and may not take into account concreteness issues. Organizations should instead be focusing their attention on concreteness and the tradeoff between concreteness and execution speed. Concreteness is often a better indicator of performance quality than the quality of execution, and therefore it can provide a more accurate picture of what software products can actually deliver. In this regard, it is important to realize that it is impossible to have one perfect test data management tool and to expect the tools to produce the ideal test data management results.
One way to overcome the test data management difficulties is to maintain test data management manually. This allows test automation developers to focus more attention on making the test scripts easier to execute and less time consuming to read. However, this approach does not solve the challenges that can arise when automated test suites need to execute in environments where concreteness is an issue. Manual test data management inevitably relies on programmers to check the generated test data files for concreteness. If programmers find the test data management test scripts are poorly written or lack sufficient concreteness, they may fail to generate the code that will generate the required data and could potentially result in test data management failures.
There are solutions to the test data management problems that can be applied both at the application level and at the system level. The application-level strategies involve taking the right test data management approaches to help test automation developers write the test scripts faster and generate more test data management results. For example, there are application level data management strategies that rely on event-driven construction of test data management data and test data management strategies that rely on modeling and leveraging database modeling techniques. The system level strategies include automation strategies for integration testing, fault injection, and application performance monitoring and control. Automation strategies for concreteness of test data management results and fault injection also rely on application-level data storage and processing techniques.
Automating the test data management lifecycle involves taking the appropriate decisions at the application, system, and logical levels. The first step in lifecycle management is taking the decision to automate the test data management lifecycle. There are two main decisions that must be made at this point. First, whether to use source codes or VBScript or both. Second, what type of test data management tools to use. Once these decisions are made, the lifecycle of test data management can move forward with the automation of source codes and testing of test data.
The second step in lifecycle management is the implementation of the test data management tools that will be used. There are a number of popular choices, including Java, .Net, Windows, and Unix/ Linux/Mac OS X. Each of these options has their own strengths and weaknesses. Each one also has their own cost and time restraints.
When an organization adopts a well-designed test data management strategy, the benefits are extensive. Increased productivity due to the reduction in manual tasks and a focus on reducing mistakes will lead to significant increases in overall business profits. The reduced costs associated with automated testing needs will result in lower expenditures. In addition, the operational process will be streamlined and the time required for testing and maintenance will be eliminated.