Using Machine Learning to Improve Ticket Classification for IT Service Management System

Abstract

In this research, we propose a maching learning based system for automatic IT support ticket classification. A total of 10,608 tickets (written in Thai) were used where each ticket was processed by several techniques of natural language processing, including word tokenization, word removal and word spelling, among others. The processed data were then fed to six different machine learning models—Multinomial Naïves Bayes, Support Vector Machine, Logistics Regression, Random Forest, Stacking Model with Bagging, and XG-Boosting. Each machine learning model was tested with two feature extraction schemes—Count Vectorization and TF-IDF. Our experiments revealed that the most accurate solution was obtained through the model utilizing ensemble technique, particularly XG-Boosting with TF-IDF feature extraction. Applying this best solution to an unknown dataset, we obtained good results, both quantitatively and qualitatively. In the case of quantitative results, we achieved the highest F1-Score of 81.21%. In the case of qualitative results, this system speeded up the IT ticket classication process from more than one day with manual classification to less than one day with our system. Consecutively, the overall users’ satisfaction regarding IT service management system increased by 8.17%.