How would you test a pencil?
My favorite kind of test interview questions are the scenario based, open ended, vague, yet very familiar to average Joe on the street. A prime example of what I mean is right there in the title of this post.
I asked this question of one of the candidates for a manual QA tester position at my company. The position is still open, so you, your friend, friend of friend, are welcome to apply.
The question is meant to probe the testing skills, abilities, ideas, thought processes, etc., and not merely to get an answer. Much like my integration tests cutting across features, the question cuts across a wide gamut of test qualities I consider necessary in a software tester.
So, let's break this question down and see how it turns out to be the canary in the software testing recruitment mine. Note that the candidates considered have some experience in software testing and are not graduates fresh out of college.
Q How would you test a pencil? A ...pause for 5 seconds. This is good. I like the candidates who take some time to think and organize their thoughts.
Well, I will sharpen it and write on a piece of paper.
Interesting. A minute ago, the candidate indicated that he would talk to the developer when testing an undocumented feature. And here, I find a total lack of application of said process. I also find that an assumption is being made - the pencil can be sharpened. What if this is a mechanical pencil that needs no sharpening? Since I didn't provide any requirements, I expected the requirements to be hashed out before any kind of test scenarios would be listed. The proverbial cart is definitely before the proverbial horse here.
Note that before asking the question I clearly stated that the requirements aren't written down in a document, and I, as a developer, am available to answer any questions and clarify scenarios that would help the candidate test the pencil to his satisfaction.
Q Any additional test scenario that comes to your mind? A I will test if this is a color pencil or not.
This is good. Now the candidate is thinking about different kinds of pencils and suddenly the test surface increases. My hopes rise.
Q What all colors would you test for? A The candidate catches on. What are the different types of colors the pencil can color in? I respond by citing the rainbow colors and one usual black pencil for writing.
Q How would you test that the red color pencil is actually red? A Well, Red is Red.
So true. My wife can differentiate among a zillion shades of green. I can see only dark and light green. A partially color blind person like me would be a terrible tester for testing anything green in color. I imagine my family malady extends to other colors in the visible spectrum as well.
I believe the candidate meant that he would write with the red pencil on a piece of paper and compare the color with The Red color. If it matches, then the pencil would pass the color test. However, lack of any explanation makes me doubt the analytical capabilities of the candidate. I was hoping for some RBG (or some qualitative) color matching techniques that go beyond the limited capabilities of the human eye.
Q Any additional test scenario that comes to your mind? A I will test how soon the pencil breaks while sharpening.
Nice. We are stepping into the stress testing zone. In spite of my probing the candidate fails to ascertain the test metric involved here. Merely repeatedly sharpening the pencil and counting the times it breaks is neither a repeatable test nor does it surface anything about the quality of the pencil. If the pencil breaks all too often, perhaps the sharpener is to blame or perhaps too much or too little force is being used to sharpen the pencil. The poor pencil begs the same level of detail here as a crash test dummy. Sadly, it doesn't get it.
Q Any additional test scenario that comes to your mind? A That's all for now, although I am sure I have missed some.
Here is how my ideal candidate would respond.
Q: How would you test a pencil? A: The candidate doesn't make any assumption whatsoever and responds with a question instead. What are the requirements surrounding the pencil?
Q: Well, we work in an agile environment. We typically don't have much formal requirements. Plenty often the stakeholders simply place their requests and the developer starts coding them up. Then we test the features.
A: Do you have any use cases to share? or stories around the pencil, to tell?
Use case 1: The pencil is a special order from NASA and would be used aboard the international space station for writing purposes, for up to a year.
Woah! The writing functionality of the pencil suddenly becomes subordinate to a whole bunch of exit criteria that the poor pencil must pass before it can even write. The mundane run of the mill pencil must now exhibit some rather awesome super powers. It must be able to withstand the blast off G force, should ideally need no sharpening, nor break off, nor fall apart easily, weigh as little as possible, ...
Use case 2: The pencil is a special monument, shaped like a pencil, to be installed at the architectural hub in down town.
This pencil would never need to write anything, on anything. Ever. Its testing must be performed during the design phase and at every step of its construction, much like a bridge over a river. If the testing commences after this pencil has been built, testing is likely too late.
Understanding the product, the requirements, the use cases, the exit criteria comes first. Test scenarios come later. Then come the test cases, the test infrastructure and then the test execution; more or less in this order.
No requirements, no use cases, no testing.
What are you testing today? I would love to hear your story.