aUTO-VERIFICATION RULE BOUNDARY TESTING

How do you test your Auto-Verification rules?

Many of us think that if you just test the value in your rule – that you are good to go!

However, to ensure that your rule ranges are configured correctly for your auto-verification rules, Rule boundary testing will not only test the behavior of your software but will give you more confidence that your ranges or input values represent your desired outcome. Boundary testing is a technique used to identify input issues at the extreme ends of a range. The primary goal is to ensure that the system works as you intend with no data entry or nomenclature issues when using these mathematical operations., e.g. <, > or = signs.

What is boundary testing?

The idea behind boundary or edge testing is that it exercises a range of values within the partitions of the software – meaning it is a check on how the software behaves at the boundary of selected upper or lower values.  For example, reference ranges or critical ranges for auto-verification rules most commonly have an upper, lower range or both. These values are at the beginning or end of a specific partition or range within the software.  Therefore, testing these partitions is important to challenge the program functionality or rule configuration at the boundaries with values just within or just outside these boundaries. The idea is to test the ‘edges’ of the boundary partition based on the endpoint value:

• Endpoint value

• Just above the endpoint value

• Just below the endpoint value

• Inside range values

Pro-tip: If the values within the rule boundary of the rule range trigger than this is a positive test, whereas if the values are beyond the boundary values, then this is considered part of negative testing

What is positive rule testing?

The concept of positive rule testing is what you would expect – confirming that the rule triggers at the ‘exact’ end-point value configured for the rule and then above this range. This means that you are conducting a test that contains only valid data – sometimes called the ‘happy path testing.’  The positive rule test case scenario will run the test exactly how it should be run with values that will trigger the rule.

What is negative rule testing?

The idea of negative rule testing is to use result values that will not trigger a specific rule or condition. Negative testing is verifying that a rule will not trigger when it is not supposed to in a given situation such as reference range, end-point range or critical range rule testing. The design of the negative test case should exercise the rule using values that should not trigger the rule. The purpose of a negative test case is to check if the rule does not trigger, but more importantly to identify if the software handles an invalid value gracefully.

Boundary Rule Examples

The LIS or middleware software (software hosting the auto-verification rules) has a reference range of 80-130 mg/dl. For boundary testing, there will be three (3) sets of test scenarios that will confirm that the software and values are tested appropriately.

·         Low range challenge: 79, 80 and 81

 ·         High range challenge: 129, 130 and 131

The value below the specific reference range (either low or high) is often called the ‘negative’ test case. This test case scenario confirms that the rule will not trigger when not expected, whereas the value above the target range will trigger as expected. The test case challenges for the exact trigger value and above the specific range are called ‘positive’ test case scenarios.

The combination of both positive and negative test case challenges are powerful testing techniques that build quality within your rule testing program.  Your auto-verification rules will be properly exercised to ensure your patient results are qualified against the proper boundary of input values.

What issues will boundary testing uncover?

1. Decimal place issues – The test cases must be set up to accommodate the appropriate decimal places for the reporting test result. If the result is reported in one to two decimal places, the testing challenge should be set up to accommodate the test case for the negative and positive test cases.

Example:

Creatinine reference range: 0.5 to 1.0 mg/dl

·         Low range challenge: 0.49, 0.5 and 0.51

 ·         High range challenge: .99, 1.0 and 1.01

Pro-tip:  The set-up of the test case can identify any decimal place configuration issues for your input result values.

2. Mathematical operator signs that are pointed in the wrong direction – Issues with the set up of the rule can be identified by setting up the test cases to challenge the endpoint value based on the mathematical operator.

Example:

Rule End Point: > 100

The test case should be set up to represent the > sign.

·         Positive Test Case = 101

·         Negative Test Case = 100

 Pro-tip: Test cases scenarios should be designed to challenge that the mathematical operator has been appropriately configured to trigger the rule at the appropriate partition.

3. Replacing an end-point value with a coded comment – A test case should be designed to identify if a numerical value is replaced by a coded comment. There are specific linearity restrictions that do not allow a laboratory to report out a numerical value. In these specific cases, a test case should be designed to challenge the limits or boundary to ensure that the appropriate workflow is triggered for reporting of the result.

Example:

Test Result = 0.4: Report values as positive

Test Result < 0.4: Report as negative

·         Positive Test Case = 0.4 and confirmation of Positive as replacement for 0.4 value

·         High Boundary Test Case = 0.41 and report as positive

·         Low boundary Test Case  = 0.39 and report as negative

Common Boundary Testing Mistakes

Test case designers and testers must understand the potential pitfalls of boundary testing to ensure comprehensive testing that verifies your configuration effectively.

• Initial Emphasis on Boundary Values: Initially, every boundary value holds significance. It's crucial to verify each boundary during range implementation. However, prioritizing boundary testing should be based on the risk, considering follow-up or periodic testing needs.

• Understanding Mathematical Operations: A deep comprehension of your software and its internal and external range operations is essential. Understanding how the software utilizes mathematical operations such as (<, >, =) aids in identifying both valid and invalid inputs for boundary value analysis.

• Consider Cross Boundary Values: In scenarios like clinical tests, there may exist linearity ranges, reference ranges, and critical values. It's imperative to be mindful of these ranges and any potential overlap when crafting boundary test cases.

• Utilize Realistic Data: Boundary testing should incorporate realistic data reflective of patient acuity and demographics. While it's important to challenge the system with edge values, it's equally vital to test with absurd values to assess the robustness of the system's design.

Summary

Boundary testing serves as a crucial tool for identifying rule configuration issues. Employing boundary testing enhances the likelihood of detecting logic and syntax errors that might remain unnoticed during rule inspection. By establishing rule test case scenarios at the extremities of endpoint values, you can ensure that your rule is configured accurately to meet your clinical requirements.