Public exploits and intrusion detection signatures: Quantitative tests of accuracy

Authors:

  • Teodor Sommestad
  • Hannes Holm

Publish date: 2018-01-23

Report number: FOI-R--4499--SE

Pages: 28

Written in: Swedish

Keywords:

  • intrusion detection systems
  • signature
  • log analysis
  • exploit reliability.

Abstract

This report describes two tests associated to log analysis within cyber security. The first test investigated how often publicly known exploits work, and gives information about the threat environment that a log analyst can use in their decision-making. In total, 1545 exploitation attempts were made with 211 exploits to obtain privileges on the machines identified as vulnerable for the exploit by a vulnerability scanner. Only 70 attempts (4.5%) were successful and only 18 exploits (8.5%) worked against some machine configuration. The second test investigated how often different public signature databases discovers public exploits. Information about this can help log analysts select signature database(s) and understand the limitations of common intrusion detection systems. Signature databases released between 2011 and 2016 from three sources were tested with traffic traces from 246 exploitation attempts with 125 exploit codes. The best signature database produced alerts of the right priority for 61 exploitation attempts; the worst produced alerts of the right priority for 20 exploitation attempts. Newer signature databases performed better; publicly known vulnerabilities were detected more often; attacks on Windows machines were detected more often than attacks on Linux machines; many more exploitation attempts were detected when inactivated signatures were activated.