Build NQuality Score to Measure the Build Health!!

Build NQuality Score to Measure the Build Health

Managers always end up asking the health of the build to the test engineers but most of the time it happens that engineer can’t give any absolute answer for this question. Many organizations create large number of graphs to convey the health of the build but it happens that no one will be patient enough to go through all the graphs so I felt the necessity of having just one value which gives clear picture of the build health.
Build NQuality Score (BNQS) is parameter which gives the value which indicates the health of the build. Zero BNQS would mean a very high quality build and large values indicate the poor quality. Ideally BNQS not just indicates the build health but also how close are we to release/ sign-off.

How to Calculate BNQS:
Consider the following parameters:

1. Open/Resolved bugs for the build at the start of build testing O={o1,o2,o3…} with each bug of different severity {critical, high, medium, low}.
2. Requests logged for the build X L={l1,l2,l3…} with each bug of different severity {critical, high, medium, low}.
3. Requests verified for the build X V={v1,v2,v3…} with each bug of different severity {critical, high, medium, low}. (These should include only requests which are verified and not reopened)

Associate weight with each of severities {critical, high, medium, low} as {w1,w2,w3,w4} where w1=1.0, w2=0.75, w3=0.50, w4=0.25. (These values can be different based on how exactly the organization is setting the severity of the request). Now the BNQS is calculated as :
BNQS= (Weighted sum of all open requests)+(Weighted sum of all logged requests)-(Weighted sum of all verified requests)
Where the weights mentioned in weighted sum are those associated with criticality.

An Example:
Initailly say there are 2 bugs (1 critical and 1 medium)
In Build 1, say 2 bugs are logged (both critical) and 1 is verified(medium) then
BNQS = (1*1+1*0.75)+(2*1)-(1*0.75) = 3 (for Build1)
In Build 2, say 1 bugs is logged (medium) and 1 is verified(critical) then
BNQS = (3*1) +(1*0.75)-(1*1) = 2.75(for Build2)
In Build 3, say 0 bugs are logged and 3 are verified(2 critical, 1 medium) then
BNQS = (2*1+1*0.75) +(0)-(2*1+1*0.75) = 0(for Build3)

Build Health Graph:

This is ideal case wherein the BNQS was zero for last build, but many times some of bugs gets deferred to future versions in which case BNQS will not be exactly zero. Ideally the graph should keep going down with each build.