Next Article in Journal
Product Lifecycle Management with the Asset Administration Shell
Previous Article in Journal
Uncertainty-Aware Deep Learning-Based Cardiac Arrhythmias Classification Model of Electrocardiogram Signals
Article

The Use of Template Miners and Encryption in Log Message Compression

1
Department of Information Systems, ELTE Eötvös Loránd University, 1117 Budapest, Hungary
2
Department of Informatics, J. Selye University, 94501 Komárno, Slovakia
*
Author to whom correspondence should be addressed.
Academic Editor: George Angelos Papadopoulos
Received: 31 May 2021 / Revised: 18 June 2021 / Accepted: 20 June 2021 / Published: 23 June 2021
Presently, almost every computer software produces many log messages based on events and activities during the usage of the software. These files contain valuable runtime information that can be used in a variety of applications such as anomaly detection, error prediction, template mining, and so on. Usually, the generated log messages are raw, which means they have an unstructured format. This indicates that these messages have to be parsed before data mining models can be applied. After parsing, template miners can be applied on the data to retrieve the events occurring in the log file. These events are made from two parts, the template, which is the fixed part and is the same for all instances of the same event type, and the parameter part, which varies for all the instances. To decrease the size of the log messages, we use the mined templates to build a dictionary for the events, and only store the dictionary, the event ID, and the parameter list. We use six template miners to acquire the templates namely IPLoM, LenMa, LogMine, Spell, Drain, and MoLFI. In this paper, we evaluate the compression capacity of our dictionary method with the use of these algorithms. Since parameters could be sensitive information, we also encrypt the files after compression and measure the changes in file size. We also examine the speed of the log miner algorithms. Based on our experiments, LenMa has the best compression rate with an average of 67.4%; however, because of its high runtime, we would suggest the combination of our dictionary method with IPLoM and FFX, since it is the fastest of all methods, and it has a 57.7% compression rate. View Full-Text
Keywords: log file processing; template mining; compression; encryption log file processing; template mining; compression; encryption
Show Figures

Figure 1

MDPI and ACS Style

Marjai, P.; Lehotay-Kéry, P.; Kiss, A. The Use of Template Miners and Encryption in Log Message Compression. Computers 2021, 10, 83. https://0-doi-org.brum.beds.ac.uk/10.3390/computers10070083

AMA Style

Marjai P, Lehotay-Kéry P, Kiss A. The Use of Template Miners and Encryption in Log Message Compression. Computers. 2021; 10(7):83. https://0-doi-org.brum.beds.ac.uk/10.3390/computers10070083

Chicago/Turabian Style

Marjai, Péter, Péter Lehotay-Kéry, and Attila Kiss. 2021. "The Use of Template Miners and Encryption in Log Message Compression" Computers 10, no. 7: 83. https://0-doi-org.brum.beds.ac.uk/10.3390/computers10070083

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop