Handling false positives in automated security testing


Automated security tools have many advantages; they save time, they enable finding security vulnerabilities without the need for user interaction and deliver instant reports. Unfortunately they also have their downsides.

Anyone who uses tools like Burp Suite or OWASP ZAP has to deal with false positives. Luckily however these tools all have options to mark findings as false positive so they will be excluded from the report. This is trivial when one uses these automated tools manually from their computer. But things get tricky when they are integrated in a Systems Development Life Cycle (SDLC) and there is no user interaction involved. How can you mark findings as false positives in that case? I encountered this problem in my previous set-ups. You can read more about them here:

These automated security tools are fully controlled by scripts in the build pipeline. The obvious solution therefore would be to set false positives in advance using a script. Using this approach causes such findings to be ignored and skipped when they are encountered during the active scan.

OWASP ZAP add-on

The OWASP ZAP Alert Filter add-on allows you to automatically override the risk levels of any alerts raised by the active and passive scan rules within a context. You can install this plugin by simply copying the file to the plugins directory in the ZAP installation location.

This add-on creates a new endpoint which is accessible through ZAPs REST API. Thus it can be used through scripting to set false positives before a scan starts!

I wrote a small Proof Of Concept below demonstrating the plugin's usage.

Proof Of Concept

The scenario is as follows: a target application exists which is continously scanned in the development pipeline. The HTML report is saved afterwards in for example Jenkins. You find out that a particular finding is a false positive which shows up in the report each time. With the script below you can provide the specific URL, parameter, and vulnerability type in order to mark it as a false positive.

This add-on requires the target applications to be in the context of ZAP. The tricky part is that since we feed Selenium tests through the proxy before the scan starts we don't know upfront which hosts will be found. I decided to put all found hosts in the Default Context. This can be risky since it will also include external domains in the context. Since I build a logic which checks wether a host is in a specific domain before the actual active scan starts I went ahead and added everything in the Default Context. More info about this logic can be seen in my previous blog.

println "Include all in context....."
path = '/JSON/context/action/includeInContext'
context = "Default Context"
regx = ".*"
query = [apikey: "${apikey}", contextName: context, regex: regx]
zapApiCall(url, path, query)

Below is the script which sets the Path Traversal finding for a specific url with parameter 'myparam' as a false positive. It will therefore be excluded from the findings report.

try {
	setFalsePositive(6, ".*specificurl", true, "myparam", apikey, url) // path traversal
}catch(Exception ex){
	println "Failed! False positives not set!"

Generic method to call ZAPs API.

def ZAPApiCall(url, path, query) {
	def ZAProxy = new HTTPBuilder(url)
	data = ZAProxy.request(GET, JSON) { req ->
		uri.path = path
		requestContentType = JSON
		uri.query = query
		headers.Accept = 'application/json'
		// response handler for a successful response code:
		response.success = { resp, json ->
			assert resp.status == 200
			assert resp.contentType == JSON.toString()
			return json
		response.failure = { resp, json ->
			throw new RuntimeException("${json}")

Since there is a good chance more false positives will be encountered I wrote a generic method for it.

def setFalsePositive(alertRule, alertUrl, bool, alertParam, apiKey, url){
	println "Set false positive rule " + alertRule + " on url " + alertUrl + " with regex " + bool
	path = '/JSON/alertFilter/action/addAlertFilter/'
	query = [apikey: apiKey, contextId: "1", ruleId:alertRule, newLevel:"-1", url:alertUrl, urlIsRegex:bool, parameter:alertParam]
	zapApiCall(url, path, query)

Vragen of feedback?