© 2005 Robert (Mike) Melendez
Software testing provides a wealth of tall tales, some of them true. If we bury ourselves in our work, we can tell the tales, but not see their significance. Sometimes we need a break. Sometimes we need to get far away and do something very different. Then we can learn to see what is just in front of our eyes everyday at work.
I don’t fish. But it had been an impossible week. Developers kept pushing back the agreed upon check in time. We testers stayed later and later each night. We were well beyond the originally advertised ship date. The developers had not yet gotten to debugging the problems we found. Then came the night before the decision to ship. The developers pushed the build time back to 11pm. We wearily started testing. At 1am we were told that the product had shipped. No completion of a testing cycle. No QA involvement in the decision. No concern that we might find a showstopper. I don’t fish, but I have a friend who does so as a religion. So I mentally checked out of work and we headed into the New Hampshire back country looking for a good fishing pond.
We drove straight to Petty Pond, my friend’s favorite spot: small enough to feel isolated and big enough not be fished out. We were going after bass. We parked along the side of the road and headed for the small pier alongside the gravel boat launching ramp. Whoa! Every 10 feet all around the pond as well as on the single small island were, no more than, teenage boys with cane poles and tin cans full of worms.
"What’s going on?" my friend asked the first boy by pier.
"We’re working for the government!" said the boy.
"Doing what?"
The boy pulled an ugly black fish out of a plastic bag and said, "Catching these. Our boss says they’re trash fish and we got to get them all."
That did it. We packed up and headed for Modo Lake, bigger; too big for an entire secondary school of boys to crowd. No! This time the lake was full of dingys with trolling motors. The drivers were older but appeared somewhat unsure of what they were doing.
Fortunately, my friend spotted a Fish & Wildlife SUV with a long-time acquaintance from that institution standing by the hood checking a map.
"Hey, Jane, what’s going on?"
"Hi Bob. We’re trying to clear the lake of malis."
"What?"
"Oh, you probably haven’t heard. We’ve had an infestation of fast breeding trash fish from the Far East in most of the southern New Hampshire fishing spots. They’re forcing out the game fish. They’re called Ichthys Anomalus, mal for short. We’ve hired a lot of companies to help clear them and are trying a few things ourselves. Hey, you’re an avid fisherman, maybe you have ideas we haven’t tried."
"We saw the boys being paid for playing hooky at Petty Pond. Is that your crew?"
"No, that’s a contractor. Do you think that will work?"
"I don’t know malis, but I doubt it. There are several heavily weeded shallow spots in the pond away from the shore where fish congregate except at feeding time. That’s at dawn and dusk. Right now, the boys are probably in the wrong place to catch many."
"What do you recommend?"
"I’d say get a few real fishermen in there with boats, who know what they’re doing. They should be able to plan a strategy, especially after some exploratory fishing to learn the malis’ habits."
"Do you think that will work here?"
"Maybe, but Modo Lake is probably too big. Looks like you have a good idea for coverage with the trolling boats. But keep track of where the most catches are made. You’re bound to find more malis at those spots. In fact, you might want to sweep those areas with nets."
"Good idea. We’re trying to do something similar at the big lakes, covering the area with small net trawlers, but it’s darn expensive."
"Learn the malis’ habits first. Then you can be more selective on what you cover and how well. For example, I doubt there’s need to trawl the deep water."
"That’s a shame, the trawlers work best in the deep water."
"Hey, I’m a taxpayer too. If you’re going to catch malis, you need to go where they are, rather than fishing for them where it’s easiest. By the way, how will you know when you’re done?
"The females are the indicator. They can lay 1000 eggs at a go. We figure if we go two days without catching a female, we’ll be done."
"Just bear in mind you’ll never get them all."
"I hear you talking."
"I also suggest you get the fishermen to work in twos. No fisherman knows it all. They can help each other out. And think about having a number of fishermen out to try out your cleaned ponds. You’ll get a better idea how the general public will react when you let them in."
"Thanks, Bob."
"Goodbye, Jane."
We visited a few more lakes to see how they were doing against the malis. We even tried our own luck catching a few malis. They don’t fight back and are ugly to boot, but they knew how to hide, so it was fun flushing them out. We headed home just past sunset, both content, if without bass to dine on.
I was so relaxed the next day, I was glad to get back to work, but somehow I felt like I never left.
The software test story is true. I lived it several years ago. The rest is a fish story, though one I wished I had lived.
Ask yourself, “What do testers do?” I’ll bet the answer includes “Run tests” rather prominently. Running tests is a pointless exercise, unless you give it a point. It’s taken me years to get that clear in my head, though I continue to have difficulty when I need to apply it. I’ve known a fair number of very good managers, developers, testers, and customers who take the “Run tests” answer literally. What mattered to them, as it did to me, was how many tests had been run, what coverage were you getting, and, most important of all, how fast can you run those tests. Coverage frequently meant the percentage of tests available that had been run. If you caught a few malis in the process, that might be good, but that wasn’t what really mattered.
Today, as a tester, I ask about the point upfront. “If I find bugs, who’s going to fix them?” being a prominent question. I need my own Bob and Jane so I can judge whether I’m doing any good or just trawling the deep water. I also need to draw attention away from testing as an end in itself. Every now and then, I break through and a manager or developer asks me, “What do you think should be the point?” That can lead to some very useful conversations, as we figure out what the point should be — this time. Next time, at the next pond, it might be different.