Artifact for "Guidelines for Assessing the Accuracy of Log Message Template Identification Techniques"

Many log message template identification techniques have been proposed in the literature to convert raw logs containing free-formed log messages into structured logs to be processed by automated log-based analysis techniques, such as anomaly detection and model inference. This paper describes the artifact related to our novel guidelines for assessing the accuracy of log template identification techniques: (1) use appropriate accuracy metrics, (2) perform ground-truth (oracle) template correction, and (3) perform analysis of incorrect templates. Specifically, the artifact includes the Python implementations for computing template accuracy (TA) metrics and applying oracle template correction and incorrect template analysis. The artifact --- including the complete replication package used to assess the application of the proposed guidelines through a comprehensive evaluation of 14 existing template identification techniques --- is available.