QA METRICS

A case for metrics :

As the adage goes, ” If something cannot be measured, it cannot be managed “, If an organization wishes to improve its skills/processes/capability of software development , it must have a scale by which to measure it. Metrics is one such scale which organizations adopt in their improvement journey.

Though we have made considerable progress in usage of metrics for project sizing and cost estimation, its usage for process measurement for in flight projects or as an input for weekly project meetings continues to be low.

Test Metrics:

  • Simply put are a standard of measurement.
  • They help us objectively measure several aspects/attribute of software development activities.
  • Are collected and interpreted throughout the testing effort.

Metrics are also referred to as the extent or degree to which a project/product/process possesses and exhibits a quality, a property, or an attribute.

Why QA Metrics:

  • One of the characteristics of a maturing discipline is the replacement of art by science.
  • Metrics are rational. They promote fact/evidence based conversations and decision making.
  • Finally you don’t have an alternative to metrics usage. The other option is to base your understanding, decisions, and actions on subjective, uninformed opinions & gut feel.

Also wee need to appreciate that software is often measured to serve following purposes.

Characterization, i.e., the gathering of information about some characteristic of software processes and products, with the goal of acquiring a better idea of “what’s going on.”

Evaluation, i.e., judging some characteristic of a software process or product, for instance based on historical data in the same development environment or data available from external sources.

Improvement, i.e., using a cause-effect relationship to identify parts of the process or product that can be changed to obtain positive effects on some characteristic of interest, and collecting data after the changes have been made to confirm or disconfirm whether the effect was positive and assess its extent.

Prediction, i.e., identifying a cause-effect relationship among product and process characteristics.

Tracking, i.e., the (possibly constant and regular) acquisition of information on some characteristic of software processes and products over time, to understand if those characteristics are under control in on-going projects.

Validation, i.e., validating the identified best practices experimentally.

 

Characteristics of Good Metrics

Like any good measurement device or indicator, a metric needs to have the below traits to improve acceptance.

Simplicity: Metrics should be simple to gather, compute, interpret and maintain.

Utility:       Metrics should be meaningful, objective, unambiguous.

Precision:   Metrics should bee sensitive to changes and reflect them accordingly.

ROI :         The returns from the metric is far greater than the cost incurred in data collection, computing and interpretation

 

Metrics   Aphorisms :

  • “Not everything that you can measure matters, and not everything that matters can be measured”.
  • “You get what you measure”.
  • “What gets measured gets fixed”.
  • “Quality is more than metrics.”.

 

Industry benchmarks:

Watts Humphrey in his article   The Quality Attitude , states that on An analysis of data on more than 8,000 programs written by 810 industrial software developers

  • The average injection rate for these developers is 120 defects per KLOC, or one defect in every eight lines of code.
  • The top 10% of the developers injected 29 defects / KLOC.
  • The top 1% injected 11 defects / KLOC.
  • Even at the injection rate for the top 1% of software developers, a 1,000,000 LOC system would enter compiling and testing with 11,000 defects.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s