Software Testing: A Hunt for Fragility
Software testers are hunters. At least, they should be. Their prey: fragility. Like some organizations and people, software too can suffer from fragility, and it is the direct responsibility of the software tester to sniff out fragility, call it by its name, and work to squeeze the life out of it.
How do you know if software is fragile? Things that are fragile break or suffer from chaos and randomness. Think the Titanic—too massive to change course in time to avoid an iceberg. Volatility brings fragile systems and people to their knees, and they seek out tranquility to avoid volatility. Think RBS Bank, which neglected its technology for decades, presumably to avoid disrupting the balance sheets.
Nassim Nicholas Taleb, in his book Antifragile:Things That Gain from Disorder, likens the fragile to the legend of Damocles. Damocles was a courtier of Dionysius II, a king of great power in 4th century BC Syracuse, Sicily. Damocles admired the king’s authority and luxurious surroundings, and Dionysius offered to switch places with Damocles so that the courtier could feel what it was like to rule with such opulence.
But there was a catch: Dionysius had a huge sword rigged up above the throne, held there by a single hair of a horse’s tail. The moral of the anecdote is that with greatness comes the constant threat of destruction.
Likewise, software that is “great” in size can suffer from fragility because one crisis can send the house of cards toppling down. To squelch this fear, anxious project heads often attempt to hamstring the software testing effort by relegating it to a position of no power or voice. This effort, of course, is counterproductive: Falsely eliminating the stress brought on by problems revealed doesn’t mean the problems go away. The giant sword still glistens precariously above.
Fragile software is also identifiable by its inability to deal with stress. When the application is stressed, it’s “all hands on deck” because the software has no built-in mechanisms for error recovery or redundancy. Outside help is required to deal with problems in real-time.
On the other hand, overoptimization also makes for fragile software. The world of software testing is obsessed with automated checking, but those who point to a machine and say “there’s our test department” are probably sitting ducks, cruising full-speed ahead toward that invisible, but very real, iceberg. Relying on binary evaluation to shoo out bugs—or worse, relying solely on “green” results to make us feel good about our software—ignores the fact that randomness is the rule, not the exception.
Change introduces new pathways of failure, such that automation designed to find old forms of failure is ineffective as soon as one line of code is changed. Introduce a bug here, but the automation shows green. More change piles on top of that missed bug, and soon the software is crawling with them. It is fragile. Software testers should be in relentless pursuit of this fragility with the goal of extermination.
