A tester's itch to start breaking the program should be as strong as a programmer's itch to start writing code. Tester’s always need to start off being optimistic that the bugs definitely exist in the module which he/she is testing.
One piece I'd like to add about the advantage of being a Tester (whatever the job title : ) is that you'll learn your underlying product literally *better than anyone*. It's true that Project Managers have put their sweat and blood into creating detailed Specifications, and Developers have literally built the application from the ground up and have an extensive knowledge of underlying algorithms- it's their own code, after all! Ultimately, I would have to argue that the Tester of a feature is going to have the most knowledge of the in's, out's, upsides, and downsides of the features.
Hi friends,
I found this interview of James Bach on software testing in one blog and felt that it will be useful for all of us and posted in this blog.
James Bach, Co-author of Lessons Learned in Software Testing, is a popular name in the world of testing because of his new and unconventional ideas. Hope that this Interview will be useful for all the testers.
Mallik: Did you have any formal training in software testing ? Were you a developer before moving to testing ? Tell us something about your background and experience.
James Bach: I'm not sure what formal training in software testing would mean. If you mean have I taken testing classes, then the answer is yes, but I didn't consider that to be training, because I already felt that I knew how to test. I've attended bits and pieces of testing training over the years, but mainly I stay away from a classroom setting, because I'm so disruptive. I started as a video game developer after dropping out of school. I moved to testing in 1987 because Apple Computer was hiring testers and I needed a job. Plus, I was intrigued by the idea of testing as a dedicated activity.
Mallik: What is the most interesting bug you have found?
James Bach: I can't say what is the most interesting, but one interesting bug I found was a piece of code X that overwrote part of a second piece of code Y, so that invoking Y sometimes mysteriously invoked the code Z that came right after Y in memory. It took me days to figure that one out, because all the evidence pointed to Z as somehow th culprit, even though it was totally innocent. It taught me how devious and non-linear software can be.
Mallik: Should testers have access to code (under test) or not ? Why ?
James Bach: Testers should have access to any resource that helps them do their job For some testers, in some situations, code can be a valuable resource. My experience is that code often does not help testers much, however, because it's a lot of work to understand.
Mallik: If an organization wants you to interview a candidate for testing position, How would you go about it ?
James Bach: My short answer is that I give them something to test and watch how they test it. Some people who can talk about testing can't do it. Some people who can't talk about testing CAN do it. So I just want to see them do it.
Mallik: Do you think that development/coding skills are essential for a testers ? If its a black box testing then where will they be using it.
James Bach: No I don't think those skills are necessary. But, they are useful. In any high performing test team, I would expect to find at least one programmer. There are three major ways that coding skills help:
Mallik: Many organizations associate QA with Process While process is important for everything why is it associated mostly with QA . And do you think that testers are missing out on the primary job of testing and spending too much time on documentation because of process ?
James Bach: First you have to be clear on what you mean by process. Process is a danger word, because it can mean many different things. If you mean, "the way thing happen", then we all are involved in process, even if we don't understand most of the processes by which we work. The aspects of process QA is associated with are process instructions (methods and procedures) or process descriptions (documentation). They are associated with that partly because our craft suffers from terrible ignorance about how highly cognitive processes work. In fact, process instructions and descriptions play a relatively small role, and sometimes no role at
all, in the actual process by which software is created and tested. QA can manipulate the paperwork all they want, but typically they have little impact on the true capability and reliability of the system to produce a good product. QA can however interfere with the process, to make it harder to produce good work. I've seen a lot of that. True QA is project management, project coordination, company management, maintaining close communication with stakeholders, craft-by-craft mentoring, and personal commitment and skill. That true QA because those are the activities that genuinely result in a quality product. But that's not what most QA groups think they are doing, it seems.
Mallik: A tester's main role usually comes into picture too late in the SDLC, and thus, he is expected to compensate for all the slippage that has occured in design as well as development(since the product release deadlines are fixed). Under such constraints, should a tester signoff a product which he himself might not be satisfied with the testing of ?
James Bach: It is not the role of testing to signoff on a product release. The most I would be willing to sign for is that I believe I have provided the necessary information for my clients to make good decisions about the product. If that didn't seem true, then I would not sign.
Mallik: What are your suggestions to the people who wants to come into testing field/budding testers ?
James Bach: I suggest that you test something and try to describe the problems you found and the process you followed. Also, I suggest reading the articles on my site, and viewing Cem Kaner's videos on Google Video.
Mallik: Should a tester restrict to one particular technological domain, and try to achieve expertise or its better to be diverse in testing ?
James Bach: I prefer variety.
Mallik: What must be a tester's approach over the years in order to improve his testing skills ?
James Bach: Pay attention to what happens when you test. Don't just believe what you are told about testing, but rather reinvent and construct the skill for
yourself.
Mallik: How can a fresh college graduate identify, if "testing" is the right profession for him ? I mean, what kind of attitude suits to this profession ?
James Bach:The kind of testing I do rewards people who like to solve puzzles and figure out how machines work. Fast learners. It's the same thing that draws people to programming, but programming requires a longer attention span. I'd rather have a more social activity that involves a lot of short assignments. Testing is like that, for me.
Mallik: How will you differentiate the skills of a tester from that of a programmer ?
James Bach: A tester is a skeptic, a programmer must have faith. Skepticism requires a bigger imagination for what is possible, whereas the programmer must be more able to figure out how to go from what is plausible to what is actual. The programmer must think algorithmically, but for the tester, lateral and logical thinking dominate. Programmers have to focus, but testers need the ability to de-focus. Programmers can be laser-like, whereas testers must shed light in all directions. Programmers and testers both create and use models. Both benefit from technical knowledge, although it is more important for programmers to have that than testers. A tester could be useful even without much technical knowledge, if he has domain knowledge, or quick learning ability. A tester is an applied epistemologist. Epistemology is optional knowledge for programmers.
Mallik: Should a tester also be involved in suggesting solutions for the bugs he has discovered ?
James Bach: Be careful with that. If the tester presumes to act like a designer, that can bruise and confuse the programmer. Having said that, offering ideas is okay, as long as you do so with respect. The testers job is to reveal important information about the product, not to design the product.
Mallik:I believe you have delivered some courses in India too, what is your experience of indian testers ?
James Bach: The 60 testers I had the pleasure of teaching were mostly younger than the ones I teach in the U.S. Once I convinced them that I wanted them to speak up and show me their thinking, I got a very positive and impressive response. I had come to India thinking that your culture is possibly not amenable to independent thinking, but I believe I was wrong about that. I came away with the sense that the testers I was teaching shared a culture of trying to please their clients. You can think independently, even though you wish to please your customers. Pradeep Soundararajan and Shrini Kulkarni are two Indian testers who are becoming well known in America with their original and incisive ideas. They are making a good showing on behalf of the Indian testing community, but Ihope that others also share their thoughts.
Mallik: Some books you would like to recommend to sharpen thinking in general, in order to be a better tester ?
James Bach:
· Introduction to General Systems Thinking, by Weinberg
· Lateral Thinking, by de Bono
· Tools for Critical Thinking, by Levy
· Godel, Escher, Bach, by Hofstaedter
Mallik: Amidst the sheer lack of good books of testing, can we expect some more good books from you ?
James Bach:I'm working on a book on Exploratory Testing.
Is Code Review Developers Job or Testers?
Did you ever ask a question, why separate Testing/QA teams exist, and why not developers themselves test their code? The answer many people give is developers cannot test, which is not true. Developers can always test, but testing will be impartial only if the underlying code is not written by him. So this led to model where in code written by A is tested by B and code written by B is tested by A. This model has one problem, primary job of the developer is writing code, and not testing so developers ignored the testing, this led to having separate Testing/QA teams. But ever wondered doesn’t the same hold good even for code inspections/reviews. Currently many organizations have peers doing the code review, which again is the same model wherein code written by A is reviewed by B and code written by B is reviewed by A. Once again developer is not very much keen in reviewing the code written by others. Many times I found the bugs as a tester in the code even after it’s reviewed by couple of developers. So, what is solution? Should we have three teams now? Of course not, but the code review can be effectively moved from development team to Testing/QA team. But, in this model we need to ensure that same tester is not doing both code review and black box testing for a module. Tester A should be doing code review for Module X and black box testing for Module Y and Tester B should be doing code review for Module Y and black box testing for Module X. Since its the testers who will answerable for any production issues they will do the code reviews with more responsibility unlike the developers.
Managers always end up asking the health of the build to the test engineers but most of the time it happens that engineer can’t give any absolute answer for this question. Many organizations create large number of graphs to convey the health of the build but it happens that no one will be patient enough to go through all the graphs so I felt the necessity of having just one value which gives clear picture of the build health.
Build NQuality Score (BNQS) is parameter which gives the value which indicates the health of the build. Zero BNQS would mean a very high quality build and large values indicate the poor quality. Ideally BNQS not just indicates the build health but also how close are we to release/ sign-off.
How to Calculate BNQS:
Consider the following parameters:
Associate weight with each of severities {critical, high, medium, low} as {w1,w2,w3,w4} where w1=1.0, w2=0.75, w3=0.50, w4=0.25. (These values can be different based on how exactly the organization is setting the severity of the request). Now the BNQS is calculated as :
BNQS= (Weighted sum of all open requests)+(Weighted sum of all logged requests)-(Weighted sum of all verified requests)
Where the weights mentioned in weighted sum are those associated with criticality.
An Example:
Initailly say there are 2 bugs (1 critical and 1 medium)
In Build 1, say 2 bugs are logged (both critical) and 1 is verified(medium) then
BNQS = (1*1+1*0.75)+(2*1)-(1*0.75) = 3 (for Build1)
In Build 2, say 1 bugs is logged (medium) and 1 is verified(critical) then
BNQS = (3*1) +(1*0.75)-(1*1) = 2.75(for Build2)
In Build 3, say 0 bugs are logged and 3 are verified(2 critical, 1 medium) then
BNQS = (2*1+1*0.75) +(0)-(2*1+1*0.75) = 0(for Build3)
This is ideal case wherein the BNQS was zero for last build, but many times some of bugs gets deferred to future versions in which case BNQS will not be exactly zero. Ideally the graph should keep going down with each build.
Thousands of test cases well documented and being executed with each build manually, if thats called being organized then I prefer being adhoc. Whenever a module specs/code arrives to tester he/she creates hundreds of test cases and executes them manually for the first time and finds lots of bugs (if either tester is good or developer is bad or both :)). All these test cases are well documented and executed with each of the builds. With the time the bugs and regressions will be reduced on this module and the module gets stabilized. After reaching this stage executing the regression test cases may not be worth the efforts as it rarely find the bugs. In fact once module gets stabilized adhoc testing helps in finding more bugs. Have you ever observed a new guy joining a team when asked to test a stable product he still will be able to find few bugs which we couldn't, just because our mind becomes dumb as we are just executing the documented test cases manually. whereas new guy does adhoc testing and comes with some great cases where product might fail.
Actually this is what has made people think of testing profession as dumb/boring job. Testers spend most of time on repetitive tasks like test case execution, build deployment etc. So I am not saying that we should stop ourselves from executing regressions as that may cause serious issue/bug to be missed out, but take up the automation more aggressively. Also software testers always ends up using traditional tools which have many limitations while automating. Software testers should be able to use regular languages like C,C++,C#,Java as well for automation and not limit themselves to available tools as this will help in increasing automation coverage. With more stuff automated, testers can spend more time on creative testing.
