A Lightweight Algorithm for Message Type Extraction in Event Logs


Adetokunbo Makanju
A. Nur Zincir-Heywood
Evangelos E. Milios

Author Addresses: 

Faculty of Computer Science
Dalhousie University
6050 University Ave.
PO Box 15000
Halifax, Nova Scotia, Canada
B3H 4R2


Message type or message cluster extraction is an important task in automatic application log analysis. When the message types that exist in a log file are defined, they form the basis for carrying out other auto- matic application log analysis tasks. In this paper we introduce a novel algorithm for carrying out this task. IPLoM, which stands for Iterative Partitioning Log Mining, works through a 4-step process. The first 3 steps hierarchically partition the event log into groups of event log messages or event clusters. In its 4th and final stage IPLoM produces a message type description or line format for each of the message clusters. IPLoM is able to find clusters in data irrespective of the frequency of its instances in the data, it scales gracefully in face of long message type patterns and produces message type descriptions at a level of abstraction which is pre- ferred by a human observer. Evaluations show that IPLoM outperforms similar algorithms statistically significantly.

Tech Report Number: 
Report Date: 
November 2, 2009
PDF icon CS-2009-07.pdf1.65 MB