Operational Acceptance Testing

Posted by Mike Baukes

How Can Operational Acceptance Testing Be Automated?

Operational Acceptance Testing (OAT) is a crucial part of the Software Development Life-Cycle (SDLC). OAT is where the underlying software configurations and operational support components come together. This process tests the implementation of structural or functional changes to a software or service within a functional or non-functional environment. In other words, Operational Acceptance Testing evaluates whether or not an application can be deployed to a network according to IT Infrastructure Library (ITIL) standards. OAT determines if a software will operate the way it is designed to without disrupting the whole installation, network or business that uses it.

Ultimately, OAT should focus on the resiliency, recover ability, integrity, manageability and supportability of a software or network installation. There should also be separate testing processes for performance, security and data loss/disaster recovery, which are, themselves, specialty areas of huge importance in their own right.

Change Driven Risk Management (CDRM) is the process that determines exactly how much Operational Acceptance Testing is needed. This CDRM process will lead to the appropriate risk assessment strategy for a new installation project. As a result, the OAT process will be more efficient and focused in identifying and addressing operational risks.

As you probably can guess by this description, this process can be rather involved and intricate.

Why Automation of OAT is Necessary

In this age of instant gratification and ubiquitous personal technology, changes to software or a network are expected to be implemented seamlessly. End-users now expect applications and services to be fully functional upon release. Users also expect a steady stream of software updates to be implemented in a way that do not hinder the overall utility of the tool. As a result, automating the OAT process is not only efficient, but it is becoming increasingly unavoidable.

The OAT process usually tests an environments mitigation of the following:

* Backup & Recovery

* Change Implementation

* Change Back-out

* Component Failure

* Shutdown & Resumption

* Operational Support & Procedures

* Alerts

This checklist of testing is determined using CDRM techniques, and is usually implemented in a managed non-functional test environment that is an accurate reflection of production. The process of doing this manually is time-consuming, especially given the constant rate of improvement and development that modern software tools now have. The SDLC of most applications simply doesn't lend itself to lingering testing processes, but steadily-increasing demand and user base requires developers to be as thorough as possible in identifying possible coding issues such as configuration drift.

Developers have traditionally tried to address this by writing run-time tests as they code, essentially implementing manual unit testing for infrastructure while building an application. While this can be effective, it adds additional time to the original development cycle, which undoubtedly leads to cost increases.

The efficiency of an automated network testing tool can shrink testing and recovery time from weeks to days - or even hours, if using the right system. Automating operational testing also can save revenue in the long-term, by reducing both cost of development and cost of maintenance. To that end, automating OAT has been somewhat of a holy grail amongst developers, enterprises and network administrators. Thankfully, several tools have made their way to the market that offer varying levels of testing and management automation. All three offer detailed reporting of testing results, making ALM (Application Life-cycle Management) easier for administrators.

Australian start-up UpGuard is the product of three IT administrators with a software development background. Recognizing that any code was only as good as its testing, they developed the UpGuard service, which allows enterprises to create and run tests using a graphics-based UI. Users define tests based upon their requirements using UpGuard's simplified interface, and the application does the rest, using command-line scripts. Aditionally, UpGuard allows users to create and run collaborative tests remotely, or on their dedicated servers. The plain English interface and ease of use make the service especially appealing to medium-to-large sized enterprises, as complex testing applications can be run by anyone, regardless of coding experience.

Opscode's Chef is an open-source systems framework tool specifically designed for automating cloud-based services. The framework offers users a fully automated infrastructure. Users write abstract definitions, which are then used as source code to build unique parts of the infrastructure. These definitions are then applied to individual servers. The tool is a Configuration Management System (CMS) built upon a systems integration framework, allowing step-by-step automated testing for your network. The over-all functionality of the tool can even be expanded using downloaded plug-ins, known as Cookbooks. As an open-source framework, it is easily expandable and adaptable for and by the community of users. Opscode's software runs a native version of the Ruby on Rails programming language, which means that users must be familiar with the code and its syntax requirements.

Puppet for Enterprise software is an IT automation software that gives system administrators the power to easily automate repetitive tasks (including Operational Acceptance Testing). The tool also quickly deploys critical applications and proactively manages infrastructure changes, whether executed on-premise or remotely from the cloud. Any task along the way of managing your network can be automated using the application. A model-based approach automates repetitive tasks and simply eliminates configuration drift. The software enforces the desired state of your infrastructure that you define, even as you work on other projects within the application. The application uses its own unique coding language, which will require users to be familiar with its syntax. Fortunately, this has been designed to be as user-friendly as possible. An online marketplace known as the Forge offers downloads of hundreds of free configuration modules to be used on a user network.

As more data is consumed across millions of devices and networks, standards of data management and transmission become more important than ever. Tools to manage and test networks are becoming increasingly standardized, as seen with the IT Infrastructure Library (ITIL) Configuration Management Database (CMDB). As this database of databases grows in size exponentially, attempting to manually test networks and applications for compliance becomes more and more unwieldy. As more developers attempt to tackle this problem with innovative solutions, the dream of fully automated operational acceptance tools appears to have finally become a reality. As a result, enterprises save countless resources and significantly improve the quality of ALM services. Use the above-mentioned tools to make your network more autonomous and effective. You customers, employees and network will thank you for it!

Follow UpGuard on Twitter

Topics: ALM, chef, SDLC, puppet

UpGuard Customers