Test Measurements & Metrics
What are testing metrics and measurements?
The evaluation of important parameters e.g. (what dose the input and output data actually mean?)
What dose some chart really say?
What is the norm? Performance is relative.
Matrices are protocols of measurement to interpret raw data.
The right parameters must be measured (watch out for data noise i.e features of the data that are not relevant).
It is important to select the right type of analytical methods!
Basic measurement concepts in testing:
- Effort: the actual time (often measured as man hours or person days) spent on a particular activity.
- Elapsed days is the difference between the start of an activity and the conclusion there of. For instance, it might only take 5 minutes of effort to order a book from Amazon.com but 3 elapsed days to get it.
The steps in a Metrics Program Cycle:
- Identify what to measure.
- Transform measurements to metrics.
- Decide operational requirements.
- Perform metrics analysis.
- Take actions and follow up.
- Refine measurements and metrics.
Important to know:
- How many activities will be involved?
- How many elapsed days it will take to complete the activities.
- How many person hours.
Its all knows as the Baseline effort estimation.
Real Life Examples Demonstrations
A software company is working on a large project; the company’s software testing team has been given the alpha version of the application to evaluate. It is estimated that each of the application’s 10 components will take 100 person days to test properly.
- If there is only one month (30 days) for testing the product, what is the minimum number of testers to be recruited so that the project should finish on time?
10 components * 100 person days = 1000 person days needed
1000 person days / 30 testers = 33.3 at least 34 testers needed
- The company manager wishes to evaluate the efficiency of the members of his software testing team. For this particular project, a software tester’s job is to perform as many test cases as possible within a standard 8-hour working day. Given this information, formulate a metric to evaluate a software tester’s efficiency.
Possible answer (metric expressed as a formula) Daily Testing Efficiency = No. of tests performed per working day
- The company manager also wishes to evaluate the efficiency of the software programmers; for any given working day, a programmer is expected to (a) write as many lines of code as possible and (b) make as few mistakes as possible , formulate a metric to evaluate a programmer’s efficiency.
Possible answer (metric expressed as a formula)
Programming Efficiency = No. of Lines of code per day –No. of Errors per day