Software Defects: Why They Won't Go Away
Lou Marcoccio—a CIO, IT adviser, research analyst, and big-four management consulting partner—recently wrote up a post on the Linkedin CIO network on his research on CIOs. According to Marcoccio, he spent nine months conducting research obtained through a survey of 188 large companies and ninety-two midsized companies. Although he states that some of the results are confidential, he does offer a conclusion:
“The survey showed significant increases in bugs making it into production and needing fixes for all types of software listed. The nine-month study research showed significant increases in bugs and necessary updates and fixes. The types of bugs and level of priority included a wide assortment. I don't feel comfortable sharing more detail, but the issue that popped out all over the place is that the number of defects/bugs are increasing considerably.”
Lou states that software defects are up more than 25 percent from 2009 and 2010. He then asks a series of questions about the state of development projects to determine why the defect rate is increasing. In this post, I will answer Lou’s questions in an attempt to explain why I think IT development defects are rising:
Lou asks: Is the problem of increasing defects attributed to the fact that development projects are not getting the time needed to be properly developed and tested?
Answer: Yes, I think this is definitely the case. Defects will increase as more code is churned out and testing windows get shorter. While there are no good measures for developer efficiency and output—i.e. Lines of Code (LOC) measurement—developers are under more pressure to produce faster. I think agile has a lot to do with this or at least the misapplication of agile has a lot to do with this; shorter sprints and development cycles could be pushing developers to the brink.
Lou asks: Are QA teams being cut?
Answer: Everything was cut during the “Great Recession.” Unfortunately QA and Quality Control is one of the first things to go, followed by CM people and other groups seen by management as being overhead and not providing value. Personally, as a result of these cuts, I think businesses went too far and cut too deep, resulting in reduced quality. The minute employees are cut, defects will go up as a result of shortsightedness of management teams and their view that QA/QC/CM is expendable.
Lou asks: Are developers less educated and experienced?
Answer: I am going to say “no” on this one for the simple fact that education hasn’t changed that dramatically since 2009 and 2010. As far as experience is concerned, a lot of graduates during that time could not find jobs. Additionally, as many people know, if you aren’t using your skills, you’re losing them, which might be a reason for any perceived increase of inexperienced developers.
Lou asks: Are companies letting experienced developers go in favor of junior people?
Answer: Yes and no. Some companies are always looking for ways to cut costs, which could be a reason for more defects, but this has always been the case, so a person could say this is a non-issue.
Lou asks: Are the tools and processes being passed over and ignored?
Answer: Yes, and this will always be the case. The Agile Manifesto states that it values “individuals and interactions over processes and tools,” and I believe this is a major cause of the increase in defects. People have twisted agile’s core beliefs to justify sloppy coding.
Lou asks: Have time pressures and release dates gotten cut too short?
Answer: Yes. As business is expecting more return on investment (ROI) from IT, it’s inevitable that IT professionals must produce results or risk being outsourced—or worse, terminated.
Lou asks: Has agile blown up in our faces and shortened the time for code development iterations and needed effort too much?
Answer: Agile has indeed been misapplied to many organizations, as some of the respondents have indicated. A lot of people use the agile moniker when in fact they are simply “cowboy” and “cowgirl” coders using chaos and calling it agile. For a good read on the decline and fall of agile, check out this post at James Shore’s blog.
Lou asks: Has it become too costly to fix bugs before release and less costly later?
Answer: No, it’s always more costly more to fix issues down the road than at the beginning as a 2004 study from NASA illustrated. The study by J.M. Stecklein, which I believe is still valid today, shows three different methodologies—bottom-up cost, the total cost breakdown, and the top-down hypothetical project—for calculating the impact of changes later in the lifecycle of a project.
Lou asks: Has most software gotten so complex because it needs to operate on too many types of devices?
Answer: Lou is definitely on to something here. I think there is some validity to this point, and as more tablets and other peripherals become available, complexity will only increase.
Lou asks: Have iPhone and iPad apps become too functional and too difficult for IT and business app vendors to compete with? Has Apple's major increase in market share of PCs and laptops made development too difficult to address Microsoft and Apple?
Answer: No, the iPad and iPhone apps haven’t become too functional or too difficult. However, the success of Apple products has resulted in more tablets from competitors that feature different platforms. This will drive development to create apps that work on a multitude of platforms, leading to a possible increase in issues and defects.
Lou asks: Has the continued weakening of the economy stripped away too many necessary steps in development processes?
Answer: Yes it has, resulting in a suffering process. The “we can’t afford to waste time on process” approach has cropped up and because of this, if Lou’s research is correct, defects are substantially up.
After reading the responses to Lou’s post, it is obvious that there are issues in the development and QA/QC arenas. The “Great Recession” had its casualties. The result has been what many face today in their development shops. Benjamin Dell commented on Lou’s article and stated that he lacks “skilled personnel, not in specific tools, but in general—how to reason and problem solve without external input.“ Maybe this is a reality we have to deal with, but as the business side puts more pressure for faster and bigger results from IT, mistakes and defects will continue to go up.
