11.4. Test Runs

This chapter explains how to create, search, edit, execute, and generate reports for Test Runs in TCMS.

  • To view Test Runs you have created, click TESTING, then My Runs.
  • To view Test Runs assigned to you, click View My Assigned Runs.

11.4.1. Creating a Test Run

Test Runs are created for a specific Test Plan. Only Test Cases with a status of CONFIRMED will be added to the Test Run. A Test Run can be assigned to any user in the TCMS.

11.4.1.1. Procedure: Creating a Test Run

To create a Test Run:

  1. Select a Test Plan, on the Cases tab hover over Run then click on Write new run from the drop-down menu.

    The New Run button

  2. In the Create New Test Run screen, perform the following actions:

    • Edit the Summary.
    • Select the Product.
    • Select the Product Version.
    • Select the Build.
    • Edit the Run Manager.
    • Edit the Default Tester.
    • If applicable, select the Set Status Automatically checkbox.
    • Enter the Estimated time.
    • Enter any Notes.
    • Select Environment property values.

    The Create New Test Run screen

  3. Click Remove on any Test Cases that are not required for this Test Run.

  4. Click Save.

11.4.1.2. Procedure: Add Test Cases to an existing Test Run.

To add a Test Case to a Test Run:

  1. Select a Test Plan, use filter if required.

  2. Click Run then Add into Run from the drop-down list.

    The Add cases to run button

  3. Select the Test Run.

  4. Click Update.

    The Update button

Note

Test Cases can only be removed from a Test Run when it is created.

11.4.1.3. TCMS Notifications

TCMS notifies the default tester by email that they have been assigned a Test Run.

11.4.1.3.1. Example: Test Run Notification

 new test run has been created for you.

 ### Links ###
 Test run: http://example.com/run/23228/


 ### Basic run information ###
 Summary: Test run for dummy testing plan on Unknown environment

 Managed: username.
 Default tester: username.

 Product: TCMS
 Product version: 3.8.1
 Build: TCMS-3.0.3-1.svn2841

 Estimated time: 0:00:00

 Notes:


 ### Test plan information ###
 Test Plan: dummy testing plan

11.4.2. Searching for Test Runs

To search Test Runs created by other authors, use the following fields:

  • Run Summary
  • Product
  • Environment group
  • Owner: Manager|Tester, Manager, Default Tester
  • Case Run Assignee
  • Plan
  • Product version
  • Build
  • Status
  • Tag

11.4.2.1. Procedure: Searching for Test Runs

To search for Test Runs:

  1. From the TESTING menu, click Search Runs.

    The Testing menu 2

  2. In the Search Test Run screen, enter the required search details.

    The Search Test Run screen

  3. Click Search. The search results appear.

    Test Run search results

11.4.3. Editing a Test Run

The Edit function modifies fields in a Test Run.

11.4.3.1. Procedure: Editing a Test Run

To edit a Test Run:

  1. Select the Test Run to be edited, and then click Edit.

    The Edit button

  2. Edit the fields as required:

    • Summary
    • Product
    • Product version
    • Manager
    • Default Tester
    • Estimated Time
    • Environment Property value
    • Notes
    • Finished
  3. Click Save.

11.4.4. Deleting a Test Run

A Test Run can be deleted (removed).

11.4.4.1. Procedure: Deleting a Test Run

To delete a Test Run:

  1. Browse to the Test Run.

  2. Click Delete.

    The Delete button

  3. Click Ok to delete or Cancel to return.

    The Delete confirmation screen.

11.4.5. Cloning a Test Run

A Test Run can be cloned.

11.4.5.1. Procedure: Cloning a Test Run

To clone a Test Run:

  1. Browse to the Test Run.

  2. Select test case-run(s) in the test run. Use a filter, if required, to help restrict the number of visible runs.

  3. Click Clone.

    The Clone button 2

  4. Enter the details for the cloned run. Details are auto-populated from the original run.

  5. Click Save.

11.4.6. Executing a Test Run

Test Runs can be executed at any time. The user can execute any of the Test Cases within a run, regardless of the order they appear. Use the Comment field to make notes about a Test Case. All comments will be displayed when a report is generated for a Test Run. Upon completion of a Test Run, the TCMS will annotate failed Test Cases with the message “File Bug”.

11.4.6.1. Procedure: Executing a Test Run

To execute a Test Run:

  1. From the TESTING tab, click My Runs.

  2. From the Test Runs list, click the Test Run to execute. The Test Run summary is displayed.

    The Test Run summary

    The user is able to change the Test Case status on this page.

  3. Execute each Test Case. Enter a Comment if required. Comments will be displayed when a report is generated for the Test Run.

    A Test Case

  4. Select the appropriate Status icon.

    Icon Meaning
    image78 Idle - Default value. The Test Case has not been examined.
    image79 Running - Test Case is in progress.
    image80 Paused - This status is used to denote a problem with the test case itself that prevents the test from being completed.
    image81 Passed - Test Case met all the expected results.
    image82 Failed - Test Case did not meet all the expected results, or produced an unhandled exception.
    image83 Blocked - Test Case has a dependency that has failed.
    image84 Error - Test environment has problems that prevent Test Case     execution.
    image85 Waived - Test Case is not suitable for this run or blocked by other cases.

11.4.6.2. Procedure 4.9. Bulk update of Test Cases

Bulk operations include change case-runs status, add/remove bug by input bug ID from case-runs, add comment to case-run.

  1. Select the Test Cases to be updated.
  2. Click on the menu item for the required operation:
    • Status - select the new status.
    • Bugs - enter the Bug ID.
    • Comment - input the comment.

Note

It is important that you file bugs against Test Cases that fail. TCMS reminds users of this by annotating failed Test Cases with “File Bug”.

11.4.7. Changing the status of a Test Run

A Test Run’s status can be changed from Running to Finished even if all Test Cases have not been completed.

If the check box Set Status Automatically is selected in the test run, when all the test cases in the run have a passed, failed or blocked result; the test run’s status will be changed to Finished.

11.4.7.1. Procedure: Changing Test Run status

To change the status of a Test Run:

  1. Browse to the Test Run.

  2. Click Set to Finished.

    The Set to finished button

  3. To re-activate a Test Run, click Set to Running.

    The Set to running button

Note

Change status in Edit Test Run

It is also possible to change the status of a Test Run from the Edit Test Run menu.

11.4.8. Generating a Test Run report

TCMS generates reports for Test Runs, regardless of their state. A report provides the following information:

  • Plan details:
    • Product
    • Product version
    • Plan
    • Plan version
    • Platform
    • Operating system
    • Run summary
    • Run notes
    • Start date
    • Stop date.
  • Test Case details:
    • Closed at
    • ID
    • Summary
    • Case ID
    • Tested by
    • Group
    • Status
  • Summary statistics:
    • Total number of Test Cases Run.
    • Total number of Pending Test Cases.
    • Test Run completed (%).
  • Bug List:
    • Individual bugs
    • View all bugs

11.4.8.1. Procedure: Generating a report for a Test Run.

To generate a report for a Test Run:

  1. Select the Test Run.

  2. From the Case Status box, click Report.

    The Report button

    A printable version displays.

  3. From the File menu in your Browser, click Print.