Penetration testing (pen testing) is crucial for developing and maintaining hardened, attack-resilient systems—these can be applications, nodes, or entire networks/environments. Specialized tools are readily avagiilable for discovering vulnerabilities and security gaps in these systems; in this comparison, we'll compare Arachni and OWASP Zed Attack Proxy (ZAP), two popular security suites for application-level pen testing.
Unlike well-known pen testing tools like Kali Linux and Backbox that combine network, host, and software/web application testing capabilities, Arachni and OWASP ZAP are specifically designed to scan web applications for flaws. These tools (and others like them) alert testers of weaknesses that are readily exploitable by cyber attackers (e.g. a SQL Injection flaw or cross-site scripting issue).
Arachni and OWASP ZAP are two of the most popular web application pen testing tools on the market; fortunately, they are also both free and open source. ZAP is maintained by the Open Web Application Security Project (OWASP), a venerable online community and non-profit dedicated to improving software security, while Arachni is supported by Sarosys, the project's corporate arm that provides commercial services around the tool.
OWASP ZAP
OWASP has been developing cutting-edge tools and resources for the general public since 2001, with the goal of improving software application security and overall online security. Its ZAP web application pen testing suite is one the world's most popular solutions for automatically finding vulnerabilities in web applications during development/testing. The project started as a fork of the popular Paros proxy, a Java-based tool for discovering application vulnerabilities and assessing web security fitness.
ZAP is written in Java (alas, Java 7 is required) and is available for Windows, Linux, and MacOS platforms. The suite includes a range of security tools: an intercepting proxy, spider, scanner collection (automated/passive, brute force, port, web sockets), and REST API for custom integrations. ZAP is commonly used with CI/CD tools like Jenkins or Bamboo to add automated pen testing to a firm's continuous delivery lifecycle and CI/CD pipeline.
Arachni
Arachni is a long-standing favorite among software pen testers, especially those partial to the Ruby programming language—the open source pen testing framework was written in Ruby and is highly extensible in this regard. In fact, one of Arachni's most lauded attributes is its scalability and modularity; for example, the tool can be used as a simple command line scanner utility or configured in a high performance scanner grid to support large-scale application security testing routines.
Notable features include responsive/mobile web application auditing, an integrated browser environment for testing modern web technologies (e.g., JavaScript, HTML5, DOM manipulation, AJAX), and a smart, self-learning capability: the tools trains itself by learning from HTTP responses, resulting in more accurate assessments and minimal false-positives.
Side-by-Side Scoring: OWASP ZAP vs. Arachni
1. Capability Set
OWASP ZAP and Arachni are comprehensive and highly capable security testing suites in their own rights—impressive, considering their price tag. That said, the two open source tools have their limitations; firms tend to extract more value by integrating them into their CI/CD pipelines for automated security testing.
2. Ease of Use
Like it or hate it, both OWASP ZAP and Arachni's rudimentary and somewhat outdated UIs make for straightforward usability. Both solutions are easy to operate, but the experience won't be a feast for the eyes.
3. Community Support
Powered by the reputation and reach of OWASP, ZAP commands a larger community of followers and subsequent support resources. The tool came out with top honors in the 2015 Top Security Tools survey held by ToolsWatch.org, beating out tools like Burp Suite and Nmap (Arachni didn't place). Arachni's community resources are not as extensive as ZAP's, but it does offer a support portal with a sizable database of resources.
4. Release Rate
As open source projects, both pen testing suites have seen regular, albeit slow coming releases over the years. A fork of the popular Paros proxy, OWASP ZAP is currently on version 2.5; the Arachni framework is on version 1.5, and its WebUI (0.5.11) has yet to reach a full release.
5. Pricing and Support
A monitoring system won't troubleshoot a configuration error. A configuration test script will.
Both solutions are free, open source solutions. OWASP ZAP is supported by a community of volunteer developers, online donations, and t-shirt sales.
Arachni derives some revenue from commercial services and support provided through Sarosys, its so-called "corporate branch" of the project.
6. API and Extensibility
Arachi comes with a well-documented REST API that enables the remote management of scans over a simple web service. Similarly, OWASP ZAP's REST API allows for interacting with the suite programmatically. And of course, both of their open source codebases are available via GitHub.
7. 3rd Party Integrations
Both offerings can be readily integrated with third party applications, but OWASP has a more comprehensive selection of pre-built integrations at its disposal. For example, its ZAP Jenkins plugin makes it easy to extend the functionality of the ZAP scanner into a CI Environment.
8. Companies That Use It
OWASP ZAP is used by countless organizations across the globe for validating their web application security postures, from governments agencies and educational institutions to large enterprises. Some of these include Mozilla, Microsoft, Ernst & Young, Accenture, and Google. Again, a fairly common ZAP implementation sees the framework integrated with Jenkins to automate security tests in a CI/CD pipeline. Arachni also finds itself integrated with Jenkins quite often (i.e, security tests automated/triggered by a Jenkins build) and boasts a similarly expansive footprint worldwide; some marquee users include Infobyte Security, eBay, Bentley Systems, Manwin, and Katana Security, among others.
9. Learning Curve
An intermediate proficiency with cybersecurity concepts and terminology is assumed with OWASP ZAP and Arachni; that said, it's unlikely that serious pen testing efforts would be left in the hands of infosec neophytes. Other than that, both tools are trivial to get up to speed with.
10. Security rating
OWASP ZAP's web presence scores a security rating of 741—respectable, but less-than-ideal due to security flaws like missing HTTP strict transport security, disabled DMARC, and disabled DNSSEC. Arachni's is poor security rating of 570.
Scoreboard and Summary
For world-class web application pen testing on a budget, either of these leading security tools will suffice. OWASP ZAP is more common in enterprise environments and with SaaS providers, especially as part of an integrated CI/CD pipeline with automated security testing in place. But if you're a Ruby software shop, Arachni's modular, high-performance Ruby framework is likely to be a better fit.
