A Next Generation Entropy Based Framework for Alert Detection in System Logs


Adetokunbo Makanju
A. Nur Zincir-Heywood
Evangelos E. Milios

Author Addresses: 

Faculty of Computer Science
Dalhousie University
6050 University Ave.
PO Box 15000
Halifax, Nova Scotia, Canada
B3H 4R2

{makanju, zincir, eem}@cs.dal.ca


Recent research efforts have highlighted capability of entropy based approaches in the automatic discovery of alerts in system logs. We refer to messages maybe of interest to an administrator as alerts. In the best case, they have been shown to detect all alerts at a false positive rate of 0%.

In this work, we extend the recent research to present the detailed evaluations of three entropy based approaches on new datasets not utilized in previous papers. We also extend the approach with the introduction of a Cluster Membership Anomaly score. This extension of the approach is intended to reduce the false positive rates required to detect all alerts. Previous work has shown that false positive rates required for the detection of all alerts for an entropy based approach could be very high. The results show that the Cluster Membership Anomaly has value for the reduction of false positive rates.

Tech Report Number: 
Report Date: 
December 15, 2010
PDF icon CS-2010-06.pdf966.06 KB