This document details the criteria used by TPTP to determine if the project can enter or exit a test pass.

• 1.0 Entry Criteria
• 2.0 Test Buckets
• 3.0 Exit Criteria

 

1.0 Entry Criteria

The following criteria must be met before entering a test pass:

• Each Project Lead updates their project's full and smoke test buckets for the test pass.
• The Release Engineering Team publishes a candidate driver on the TPTP Download site.
• All of the automated test suites have passed the Build Verification Test (BVT). See the Test Automation Initiative for more information.
• No new high severity (for example, critical and blocker) defects expected during the test pass.

It is the responsibility of the Project Leads to ensure their project's full and smoke test buckets are maintained.

 

2.0 Test Buckets

The following tests buckets must be executed for each test pass:

• For TP1, the full test bucket (for example All<test type>Tests.testsuite) is executed.
• For TP1, the smoke test bucket (for example All<test type>SmokeTests.testsuite) is executed.

It is the responsibility of the Project Leads to determine the acceptable level of testing for the full and smoke test bucket.

For more infomation on the test buckets, see the TPTP Testing Strategy.

 

3.0 Exit Criteria

The following criteria must be met before exiting a test pass:

• 100% of all test cases have been attempted.
• 95% of all attempted test cases have passed.
• For GA deliverables, 100% of all attempted test cases have passed.