Automatic vulnerability detection: description and statistics

The time has come to summarize the results of the automatic scan, which was announced 3 weeks ago. Several applications for automatic audit were sent; most sites represented the commercial sector - online stores and corporate sites.

Tool selection

For automatic testing, one of the most popular opensource vulnerability scanners was chosen - w3af.



w3af or Web Application Attack and Audit Framework is a flexible platform for searching and exploiting vulnerabilities in web applications, works on most modern systems, written in Python. This framework is sometimes called Metasploit for Web.

To simplify the process of automated testing, the console version of w3af_console was taken . As in the GUI version, where profiles (consisting of plug-ins) control everything, in the console version, all control and settings also depend on the connected plug-ins in the so-called script file.

How it works

The format for launching a customized script file is quite simple:

./w3af_console -s MyScript.w3af

The script file consists of a sequence of plug-in types of plug-ins and their parameters:

plugins
output console,text_file
output
output config text_file
set output_file report.txt
set verbose True
back
output config console
set verbose False
back
crawl all
crawl
grep all
grep
audit all
audit
bruteforce all
bruteforce
back
target
set target http://targethost
back
start

In the presented example, all links from the targethost website will be collected, requests and answers will be analyzed, comments, error output, checked against the knowledge base / signs for common web vulnerabilities, detected input forms will be subjected to a bruteforce attack (password selection) and the result will be saved to the report.txt file with the full output of the plugins.


^{manual plug-in output in w3af_console}

What does it consist of

Depending on the type of exposure, there are several types of plug-ins:

• attack - to implement an attack, exploiting. In automatic audit were not used.
• audit - identifies vulnerabilities in a web application, contains plugins for searching XSS, SQLi, CSRF, LFI, RFI, open redirects and many others. Some plugins were used in automatic auditing.
• auth - plugins for setting authorization parameters on the investigated resource. In automatic audit were not used.
• bruteforce - plugins for conducting a dictionary matching attack (bruteforce). In automatic audit were not used.
• crawl - plugins for searching and collecting information, sorting out file names and directories, using search engines, determining the type of CMS, analyzing forms for submission to audit / bruteforce / attack departments. Some plugins were used in automatic auditing.
• evasion - plugins to bypass IDS , security policies and WAF . In automatic audit were not used.
• grep - plugins for analyzing web server requests / responses, searching for critical information, displaying errors, comments in the source code, etc. Some plugins were used in automatic auditing.
• infrastructure - plugins for analyzing server settings, misconfigurations, virtual hosts, etc. Some plugins were used in automatic auditing.
• mangle plugins for modifying queries on the fly. In automatic audit were not used.
• output - plugins for output and saving of work results. Some plugins were used in automatic auditing.

You can find ready-made examples of script files for different attack vectors and the type of impact here . Their performance can be tested in a specialized vulnerable moth environment .

Statistics

To conduct an automatic audit, a script file was compiled to search and identify a complete site map, such as a CMS, a web server and search for OWASP TOP-10 vulnerabilities , a system for checking the legitimacy and reporting layout was set up (summarizing recommendations for vulnerabilities found were added manually by our company employees, the machine cannot analyze attack vectors and make an accurate attack scenario).


^{screenshot of the system for adding a task to the autoscan}


Of the total number of sites, the top 5 in terms of the total volume of identified vulnerabilities looks like this:

1. Sensitive data leak - OWASP A6 (sensitive data exposure) - invalid certificate configurations, incorrect HSTS policies are also included here .
2. Insecure configuration - OWASP A5 (security misconfiguration) - listing directories, default settings, outdated software versions.
3. Using components with known vulnerabilities - OWASP A9 (using components with known vulnerabilities) - several unclosed vulnerabilities were found, with public exploits available.
4. Cross-site scripting - OWASP A3 XSS (cross-site scripting) - Several passive XSS were found.
5. Code injection - OWASP A1 (injection) - SQL-injection vulnerabilities are still common.

Resource scanning time took from 10 minutes to 2 days. Several sites could not withstand the load and automatic testing was stopped until the recovery. For all identified vulnerabilities, reports were prepared containing a listing of the identified vulnerabilities, the likelihood of a web application being compromised, and a possible attack scenario.