Comparative Analysis of Regression Models: A Case Study of KNN Regression vs. Multiple Linear Regression
Article Sidebar
Main Article Content
Abstract
This research paper presents the application of the K-Nearest Neighbors (KNN) algorithm as a regression model for numerical prediction. We propose for KNN regression as a viable alternative for scenarios characterized by non-linear or ambiguously linear data relationships, where conventional linear regression models frequently underperform. Our experimental findings concentrate on evaluating the efficiency of KNN regression in comparison to established models such as multiple linear regression across five datasets. This illustrates the capability of KNN regression to achieve more accurate numerical predictions. In addition, we explore the effects of distance metrics, the inverse distance weighting (IDW) method for neighbor weighting, and K-value selection (number of neighbors) in our in-depth parameter tuning for KNN regression. The results suggest that KNN regression is an efficient and compelling alternative regression model for numerical prediction, particularly when dealing with complicated data and ambiguous linear correlations. Thus relieves the need for more complex models like artificial neural networks.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Journal of TCI is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence, unless otherwise stated. Please read our Policies page for more information...
References
U. Grömping, “Variable importance in regression models,” Wiley Interdisciplinary Reviews: Computational Statistics., vol. 7,
no. 2, pp. 137–152, 2015.
A. Zapf, C. Wiessner, and I. R. König, “Regression analyses and their particularities in observational studies: Part 32 of a series on evaluation of scientific publications,” Deutsches Ärzteblatt International., vol. 121, no. 4, pp. 128, 2024.
A. Khazaee Poul, M. Shourian, and H. Ebrahimi, “A comparative study of MLR, KNN, ANN and ANFIS models with wavelet transform in monthly stream flow prediction,” Water Resources Management., vol. 33, no. 8, pp. 2907–2923, 2019.
D. N. Cosenza, L. Korhonen, M. Maltamo, P. Packalen, J. L. Strunk, E. Næsset, T. Gobakken, P. Soares, and M. Tomé, “Comparison of linear regression, k-nearest neighbour and random forest methods in airborne laser-scanning-based prediction of growing stock,” Forestry: An International Journal of Forest Research., vol. 94, no. 2, pp. 311–323, 2021.
H. A. Abu Alfeilat, A. B. Hassanat, O. Lasassmeh, A. S. Tarawneh, M. B. Alhasanat, H. S. E. Salman, and V. S. Prasath, “Effects of distance measure choice on k-nearest neighbor classifier performance: a review,” Big Data., vol. 7, no. 4, pp. 221–248, 2019.
A. İ. Çetin and A. H. Büyüklü, “A new approach to K-nearest neighbors distance metrics on sovereign country credit rating,” Kuwait Journal of Science., vol. 52, no. 1, p. 100324, 2025.
M. M. Kumbure and P. Luukka, “A generalized fuzzy k-nearest neighbor regression model based on Minkowski distance,” Granular Computing., vol. 7, no. 3, pp. 657–671, 2022.
G. Zhao, Z. Li, and M. Yang, “Comparison of twelve machine learning regression methods for spatial decomposition of demographic data using multisource geospatial data: An experiment in Guangzhou City, China,” Applied Sciences., vol. 11, no. 20, p. 9424, 2021.
S. Uddin, I. Haque, H. Lu, M. A. Moni, and E. Gide, “Comparative performance analysis of K-nearest neighbour (KNN) algorithm and its different variants for disease prediction,” Scientific Reports., vol. 12, no. 1, Article 6256, 2022.
M. J. Hosen and I. Ahmed, “Comparison of Regression, K-Nearest Neighbors (KNN) and Multi-Layer Perceptron (MLP) Models for the Prediction of Weight, Gender and Body Mass Index Status,” International Journal of Computer Applications., vol. 185, no. 29, Aug. 2023.
Z. N. Liu, X. Y. Yu, L. F. Jia, Y. S. Wang, Y. C. Song, and H. D. Meng, “The influence of distance weight on the inverse distance weighted method for ore-grade estimation,” Scientific Reports., vol. 11, no. 1, p. 2689, 2021.
P. Srisuradetchai and K. Suksrikran, “Random kernel k-nearest neighbors regression,” Frontiers in Big Data., vol. 7, Article 1402384, 2024.
A. M. Raheem, I. J. Naser, M. O. Ibrahim, and N. Q. Omar, “Inverse distance weighted (IDW) and kriging approaches integrated with linear single and multi-regression models to assess particular physico-consolidation soil properties for Kirkuk city,” Modeling Earth Systems and Environment., vol. 9, no. 4, pp. 3999–4021, 2023.
