Why Configurations Must be Tested

Updated on August 2, 2016 by Mike Baukes

Configuration testing should not only be an essential step in the overall development process, but also important in the process of installation of new apps for use on web and application servers. Without proper testing, apps can often fail or be open to vulnerabilities. Exposure to attack by hackers or viruses can lead to needless expenses and excessive time correcting these problems. It is not unusual for app developers to overlook the need for configuration testing. This is because they believe that using automated methods, like Chef and Puppet (or other systems that test the deployment of their products), will work just fine. They feel that by using these fully automated processes they can test consistency, reproduce outputs adequately, and determine if things are working as predicted or not. This kind of thinking can delay a timely product delivery, produce unnecessary costs, and create additional workloads to address vulnerabilities that can occur later in production.

Why Automated Testing Isn't Enough

Automated test suites are just that - automated. These tools are generic, and are not designed to really know your product, or to actually be able to detect flaws based on any problems that might occur. These automated test methods lack the ability to emulate the different changes that might show up, which are pertinent to your particular app and configurations. Problems can occur due to firewall changes, security configuration changes, or from patch code changes that are implemented as a quick fix to a problem.

The whole process of developing a software app requires complex and time-consuming processes, such as planning, building, testing, and deploying your product. One of the most important aspects of development is security testing during the deployment stage. Traditional means do exist to test authentication and authorization techniques, along with password protection, on the front end. Developers may have the knowledge of what security needs to be implemented and tested on the initial source code for the application, but they may not have the necessary knowledge for testing the application under a complete integrated system, or the operational environment. For example, they may not have knowledge about a server that hosts the web application, whether a valid SSL certificate is required for a secure configuration, or how a change to the firewall might effect security concerns. Testing configuration issues are a major characteristic of security testing and should be implemented to prevent potential attacks.

Case for Everyone Being Part of the Process

Even the most sophisticated security tools cannot compete against an experienced security tester, someone who knows the security issues for the system, including the root cause of the security breach, testing technique to find the cause, remedies, or countermeasures necessary to fix it. Using someone, or a method, that doesn't test or know your security issues will only result in giving you a false sense of security.

For example, when a developer issues a patch to solve a coding problem, how do you know that it is clear how to use the patch, whether the patch is easily accessible to be implemented, or how the patch effects other components? Often, these patches or changes are made, and then fail to be readily communicated to others or never implemented successfully. Configuration testing ensures that all changes are readily accessible, easily integrated with current systems and requirements, and that everyone involved in the process is kept up to date at all times.

Each Team Has Its Place in the Process

Your teams should work independently using their skills in the various stages by offering inputs about what they know best. The development team should concentrate on the build for the application, the security team should focus on security testing, and your operations teams should be responsible for the compliance and validation processes. By implementing testing methods, where everyone has a say so in the process, which they have full access to, bugs can be eliminated and changes can be implemented smoothly for their particular area of responsibility. When configuration and security issues arise, the development team should not be required to learn new code or test new frameworks. The security team can play an important role in protecting the contents of the site, and can be responsible for requirements of the web server or application server configuration. The system administrator has the necessary knowledge to know how a server should be configured and the common guidelines which should be taken into account. When this knowledge is applied early in the process, problems and vulnerabilities can be addressed early on and can often cost less to implement.

Last But Not Least: Need for Sharing

Teams will be able to collaborate on their configuration testing ideas and share in the individual and overall configuration testing responsibilities. Everyone will benefit; duplicate efforts are eliminated, systems configuration is defined and information readily documented, communication and collaboration is readily improved, and time and money spent are reduced to achieve efficient results with each change implemented. Using configuration testing ensures that adequate communication, monitoring and documentation of the app development is readily available to all team members. Communication between teams will foster thinking "outside of the box". Normally, use cases for good security would test only what one might expect would happen. Seldom would an automated method really test unusual cases, which could break the application or cause an app to fail in an insecure manner. Since automated methods often fail to catch these out of the ordinary cases, it is imperative that organizations consider other cases that arise by using creative thinking techniques. Creative thinking can often help determine what may cause an application to fail and how to help avoid or solve any problems in advance.

Using UpGuard will be ensure that your teams will be able to define, share, and run the correct configuration tests to ensure the quality required to meet your company's goals. Using the correct configuration testing can virtually close the gap on future security risk costs by addressing them before there is a problem. You can shorten the time needed before releasing your products without sacrificing quality or performance by using configuration testing and implementing continuous integration as part of your teams release management strategy.

Follow UpGuard on Twitter