The importance of software testing for the success of a project

I specifically remember a project I was the project manager of, whose goal was the introduction of an off-the-shelf product (software + dedicated servers) into a quite complex and customized environment. We followed a waterfall SDLC and during the development phase we were confronted with the following challenges:

• We discovered that the assigned test manager/tester was overbooked by multiple projects, thus having not enough time to properly work on each of them.
• The product vendor revealed that, given the complexity and degree of customization of the existing environment, integration testing could only be performed by an expert.
• A last-minute-requirement was added, demanding for additional multi-language features.
• The sponsoring department was undergoing a reorganization, which put additional “political” pressure on the project.

When entering the Testing phase, the project’s status was amber, tending to red. We decided to invest more in testing, building a Testing Team:

• We hired an expert from the vendor to properly test the integration of the new product and to transfer knowledge to our tester.
• The project manager (myself), system administrators of the involved systems and specialists of the sponsoring department were also involved in testing.
• To reduce the tester workload, I additionally took over the Test Manager and Defect Manager roles.

Organization of Testing

Testing was organized in an iterative way, as follows:

• Functional Testing (manual) was performed by the assigned tester.
• SW Integration Testing was performed by the vendor’s expert.
• HW Integration Testing was performed by the involved system administrators.
• Translations for the last-minute multi-language feature were double-checked by native speakers, surprisingly finding several errors.
• Once a subset of functionalities successfully passed functional and integration testing, the system administrator and I performed Exploratory Testing on it.
• After exploratory testing was completed and detected bugs fixed, the subset of functionalities was submitted to Usability and Acceptance Testing, which were performed by selected specialists from the sponsoring department. Thanks to their user’s perspective, bugs could be found which would have not been detected otherwise.

The project went live on time, with very good perceived quality and with only a small budget overrun despite the expenses needed to hire the vendor’s expert. That was mainly due to the efficient way we worked during testing, sometimes sitting all together in the same room, thus saving time for coordination, meetings, bugs turnaround and users training.

Main lessons learned from that project

• Networking and clear, factual communication are fundamental.
• Building a team spirit is the key to success.
• Development skills in the testing team can make things much easier.
• Always question resource planning and skill sets available in the assigned testing team.
• Plan resources for testing translations.
• Plan enough time and resources for integration testing, always. Even if the software to integrate is off-the-shelf and used by millions of users, it could not work as expected when you integrate it in a specific, highly customized environment. Moreover, integration of new hardware is never trivial (due to security policies, firewall rules etc.).
• Encourage exploratory testing during the whole testing phase, time-boxing it carefully.
• Involve direct customers during testing, but let them test when you are sure there are neither critical nor high severity bugs. They will perceive the good quality of the product and will turn into supporters of the project.

But the most important lessons I learned and keep learning in every project is that testing is definitely not dead: software testing is of utmost importance to ensure the quality of the end product and the project’s success.