Pentesting Considerations

Context

Many sites that use Expedited WAF also contract with outside security testing services or consultants who perform a mix of automated and manual penetration tests against their site.

This document covers some of the common pitfalls encountered while doing this and presents some possible solutions.

Terms

penetration test

From our perspective this can be anything from a one off script to test for a specific vulnerability to a entire attack suite of automated vulnerability scanners to manual tests.

firewall domain

The firewall domain is what's pointing to the WAF Edge network. Typically this is your production domain like app.example.com

Heroku application domain

Heroku automatically assigns a domain based upon the application name at app creation, like app-name.herokuapp.com

Permissions and Scheduling

No permission from the Expedited WAF team is required and no prior notice is needed to perform a penetration test.

What is being tested?

If your site is currently behind an Expedited WAF firewall and a penetration test is run against it for practical purposes you aren't testing the web application, you're testing how well Expedited WAF can detect and stop a vulnerability scanner.

To the WAF network there is no functional difference between a hired security consultant launching a vulnerability scan against a site and a person with ill intentions. In large part the WAF is in place directly to prevent the types of requests that would originate from a penetration test.

If it's blocking these assessment attempts it's working to protect your site.

Multiple Blocking Types

When the WAF detects that a request is malicious it will return a 403 HTTP status code along with a message (a "WAF Block").

If there are persistent malicious requests from an IP address the network will shift to temporarily timing out requests from that IP. Against actual attackers this is a much better response than immediately returning a block as it ties up their resources and exhausts the attack.

However, quite often this generally positive behavior can result in inconclusive or misleading reports as the the scans "can't finish".

What We Don't Recommend

1. Don't test directly against the WAF domain

While you are welcome to run scans and tests against your own WAF domain, it's not going to return information useful in hardening your actual web application.

2. Don't add the IP to the Allowlist

In general, the IPs on the allow list are for WAF Blocks and not Network Blocks (timeouts). Consider a trusted IP address that starts sending mass amounts of malicious requests. That IP should temporarily be blocked for the protection of your site.

What We Recommend

Below are some options that we feel confident in recommending as they've been used by many other applications to help balance the need for ongoing scans.

1. Test against a staging application

Deploying a staging application with identical infrastructure (sans the WAF) is relatively simple on Heroku. Running a pentest against a staging environment with an up to date code release allows for good coverage and findings without WAF complications

2. Test against a temporary non-WAF subdomain

Adding a new Domain and DNS entry for a site takes only a few minutes. Some of our customers will coordinate hours where a pentesting.example.com domain is activated and then removed from the production application. This allows for controlled access on production systems.