CS 4390/5387 SOFTWARE V&V LECTURE 5 BLACK-BOX TESTING - 2

Transkript

1 1 CS 4390/5387 SOFTWARE V&V LECTURE 5 BLACK-BOX TESTING - 2 Outline 2 HW Solution Exercise (Equivalence Class Testing) Exercise (Decision Table Testing) Pairwise Testing Exercise (Pairwise Testing) 1

2 Homework 3 Valid Invalid X<= <X<1801 X>1800 Too Little Too Many T#1 T#2 T#3 L S B L S B Test # Locks Stocks Barrels Total sales in $ Expected Output $ $ $ Error Error Error Error Error Error T#4 T#5 T#6 T#7 T#8 T#9 Homework (Boundary Testing) 4 Test # Locks Stocks Barrels Total sales in $ Expected Output Error Error Error Error Error Error

3 Exercise (Equivalence Class Testing) 5 HW Solution: Part 1 6 Call spans time change Call doesn t span time change March time change November time change Duration > 20 Duration <= 20 Duration > 20 Duration <= 20 Duration > 20 Duration <= 20 No No No No 12 total test cases No No 3

4 HW Solution: Part 2 7 Valid year invalid year Leap Year Non-Leap Before 1812 After 2016 Valid month Invalid month Valid month Invalid month 0 > 12 0 > 12 1,3,5, 7,8,10 4,6,9, 11 Dec Feb 1,3,5, 7,8,10 4,6,9, 11 Dec Feb 30,31, 32 29,30, 31 30,31, 32 28,29, 30 30,31, 32 29,30, 31 30,31, 32 27,28, 29 DT Testing: One More Example 8 A marketing company wishes to construct a decision table to decide how to treat clients according to three characteristics: Gender, City Dweller, and age group: Young (under 30), Middle-age (between 30 and 60), and Old (over 60). The company has four products (W, X, Y and Z) to test market. Product W will appeal to female city dwellers. Product X will appeal to young females. Product Y will appeal to Male middle aged shoppers who do not live in cities. Product Z will appeal to all but older females. 4

5 Solution Gender F M F M F M F M F M F M City Y Y N N Y Y N N Y Y N N Age A A A A B B B B C C C C W X X X X X X Y X Z X X X X X X X X X X Solution (Simplified) Gender F M F M F M F M F M F M City Y Y N N Y Y N N Y Y N N Age A A A A B B B B C C C C W X X X X X X Y X Z X X X X X X X X X X 5

6 Solution (Simplified) Gender F M F M F F M F F M City Y Y N N Y N N Y N N Age A DC A A B B B C C C W X X X X X X Y X Z X X X X X X X X Something Totally Different 12 Example Handout How would you test this? 6

7 Possible Strategies 13 Don t test at all Test all combinations Chose one or two tests and hope for the best Chose tests you have already run Chose the easy tests Make a list of all combinations and chose first few Make a list of all combinations and choose a random subset By magic, chose a specially selected and fairly small subset that can find great many defects Pairwise Testing 14 Test all pairs of variables when combinations to test is very large Most defects are either single-mode or double-mode defects # of tests needed to trigger a defect 11% 4% 4% 1% 84% 7

8 Pairwise Testing 15 Test all pairs of variables when combinations to test is very large Most defects are either single-mode or double-mode defects Pairwise testing defines a minimal subset that guides us to test for all single-mode and double-mode defects Four parameters, with three possible values each 3 4 = 81 possibilities Can be done in 9 tests 13 parameters, with three possible values each 3 13 = 1,594,323 possibilities Can be done in 15 tests 20 parameters, with 10 possible values each = MANY possibilities Can be done in 180 tests Orthogonal Arrays 16 Two-dimensional array of numbers that has this property: Choose any two columns in the array, all the pairwise combinations of values will occur in within the two columns. Three variables, with two possible values each: 2 3 = 8 Possibilities A B C

9 Orthogonal Arrays 17 Two-dimensional array of numbers that has this property: Choose any two columns in the array, all the pairwise combinations of its values will occur in every pair of columns. If a pair occur in the array multiple times then all other pairs will occur the same number of times Three variables, with two possible values each: 2 3 = 8 Possibilities, but we could do this in half the test cases A B C Note that we are not looking for triplets such as 121. Only pairs Orthogonal Array Example 18 An orthogonal array with 4 columns (attributes), maximum value 3 (1-3) L 9 (3 4 ) # of columns # of rows Maximum value 9

10 Orthogonal Arrays 19 Not all columns must have the same range of values L 18 ( ): 18 rows, 1 column of 1s and 2s, 7 columns of 1s, 2s, and 3s Good news about pairwise testing is that we do not actually have to come up with the OA. Tools will assist in doing this task Using Orthogonal Arrays 20 a) Identify the variables b) Determine the number of choices for each variable c) Locate an orthogonal array which has a column for each variable and values within the columns that correspond to the choices for each variable d) Map the test problem onto the orthogonal array e) Construct the test cases A library of over 200 orthogonal arrays: Maintained by N.J. A. Sloane. 10

11 Example: Mortgage Application 21 Loan Details Term of loan (30yrs, 20yrs, 15yrs) Loan Amount (Large, Medium, Small) LTV Ratio (90%, 80%, 70%) 27 possibilities Customer Details Income (High, Medium, Low) Credit Score (High, Medium, Low) Customer Status (VIP, Regular, Employee) 27 possibilities Variables and Possible values Property Details Type (House, Apartment, Condo) Location (Washington, California, NY) Residence Status (Primary, Rent, Investment) 27 possibilities All Possibilities? ? 27*27*27? In your teams 22 Create an account in Create an orthogonal array for the previous problem How many test cases do you need? 11

12 Example: Mortgage Application 23 Example: Mortgage Application 24 e) Construct the test cases Construct a test case for each row in the orthogonal array 19 test cases Any single-mode or double mode defect will be discovered 12

13 Applicability and Limitations 25 It is equally applicable at the unit, integration, system and acceptance test levels Reduces the number of test combination significantly Issues with OA use Plan Forward 26 Wednesday 03/16/2016 Presentations of HW assignment for NextDate Homework Team work on semester project Come to class with your laptop Have a pairwiser account ready Exam 1 review Wednesday 03/23/2016 Exam 1 13