Message Type Extraction Based Alert Detection in System Logs


Adetokunbo Makanju
A. Nur Zincir-Heywood
Evangelos E. Milios

Author Addresses: 

Faculty of Computer Science
Dalhousie University
6050 University Ave.
PO Box 15000
Halifax, Nova Scotia, Canada
B3H 4R2


The task of alert detection in event logs, i.e. determining which events in the event log require action from an administrator, is very important in preventing or recovering from downtime events. The ability to do this automatically and accurately provides significant savings in time and cost of downtime events. In this work we combine message type extraction based alert detection with the entropy based approach of the Nodeinfo algorithm, which is in production use at Sandia National Laboratories, to significantly improve its performance. We show that with Message Type Indexing (MTI) and some modifications to the Nodeinfo framework, we can achieve an ∼99% reduction in the computational effort required for Nodeinfo and an F-Measure score of up to 100% in the identification of regions of the event log which contain alerts. Our work demonstrates a practical application of employing MTI on a real world data set using an alert detection framework that is currently in production use in a major government run national laboratory.

Tech Report Number: 
Report Date: 
November 2, 2009
PDF icon CS-2009-08.pdf1.11 MB