Purpose

Define all testing components needed to assure that the delivered system functions in accordance with specifications and will meet customer needs.

Focus

Testing is often sacrificed in order to “meet the deadline”. The deliverables of this session enable:

• end-users to validate that the delivered solution is the right system

• developers to verify that the system works right (according to specifications)

You will need

Inputs

Requirements definition document, System design diagrams, Program specifications

Contributors

End-users, Project leader, Systems developers

Experts

System testers; Quality Assurance, Independent Verification and Validation (IV&V) team

Advisors

Auditors, Legal department

Techniques

Requirements validation, Equivalency grouping, Boundary value analysis, Error guessing, Black and White box testing, and Test data engineering Techniques

Tools

Word processors, spreadsheets, Process and Data Modeling, appropriate configuration-specific testing tools

Agenda

The following example depicts the sequence of activities during a typical 5 day session.

Time

Participant activities

Day 1, a.m.

• Introductions and agenda presentation

• Adapt test script template for project specifics

Day 1, p.m.

• Evaluate impact of failure to meet requirements

• Identify and specify needed test cases

Day 2, a.m.

• Create requirements test case matrix

• Determine acceptable module coverage levels

Day 2, p.m.

• Create program specification test case matrix

• Define additional test cases

Day 3, a.m.

• Define equivalencies and probable errors in break-out sessions

• Define criteria for test tools

Day 3, p.m.

• Document expected results in break-out sessions

• Evaluate in-house expertise and availability of test tools

Day 4, a.m.

• Create preliminary project test plan

• Optimize test case groupings for test runs

Day 4, p.m.

• Walk-through test scripts during break-out sessions

• Determine resources needed for test execution

Day 5, a.m.

• Define test execution roles and responsibilities

• Finalize working project test plan

Day 5, p.m.

• Revisit open issues and questions lists

• Assign post-session tasks and schedule post-session meeting

Tips and Tricks

Once the new system is in production, the deliverables of this session expedite error finding and enable regression testing.

Keep the test environment in synch with any changes made in the production environment.

The cost of creating and maintaining a reliable test environment may be prohibitive for some systems. During the pre-session, compare the cost of system failure with the cost of sufficient testing and act accordingly.

Output description

The deliverables define all testing activities required to validate that the system works according to specifications and that the business requirements are met.

A testing strategy identifies testing goals, tools, and environment.

Risk analysis evaluates system components with categorization into components whose failure results in:

• external system influence

• system data corruption

• system failure without external impact

• and functional failure within the system

This categorization enables the establishment of required levels of statement and path coverage at the decision or condition level per component.

A requirements validation matrix cross-references identified test cases to functional, performance, constraints and informational requirements. Subjective requirements must be clarified to enable test case development.

The system component impact matrix identifies the modules exercised in each defined test case and helps to reduce the number of test cases required to achieve desired coverage levels

Unit, integration and system test plans identify milestones, test schedules, resource usage, dependencies and contingency plans.

Test runs optimize the sequence of test jobs. Typical groups are valid data, exception data, invalid data and missing data categories to isolate specific problem domains.

Test cases depict prerequisite test bed conditions, input data, expected results and failure actions. Specific input data is based on equivalency groupings and boundary values to minimize the number of test cases required. Test cases are defined for unit, integration, system and/or customer acceptance testing.