New Criteria for Selection of Regression Models: Case of Small-Size Sample
Keywords:
Model Selection Criterion, Regression Model, Probability of Overfitting or Underfitting, Observed L2 EfficiencyAbstract
The objective of this study was to create a new criterion viz. the New Information Criterion (NIC) for the selection of regression models in a small-size sample case. The performance of NIC was compared to those of three other model selection criteria, namely, KICcC, KICcSB and KICcHM. The conditions for the simulation were the differences in the sample size, number of parameters in the model, regression coefficient, error variance, and distribution of independent variables. The results showed that NIC exhibits the following formula: NIC = log(s2) +log() +
The performance comparison results revealed that KICcHM performed the best. However, such a criterion could identify the true model less accurately. Therefore, this research used the observed efficiency as another model selection criterion. This criterion suggested that NIC was the best criterion.
References
Akaike, H., 1974, “A New Look at the Statistical Model Identification,” IEEE Transactions on Automatic Control, 19 (6), pp. 716-723.
Hurvich, C.M. and Tsai, C.L., 1989, “Regression and Time Series Model Selection in Small Samples,” Biometrika, 76 (2), pp. 297-307.
McQuarrie, A.D.R., Shumway, R.H. and Tsai, C.L., 1997, “The Model Selection Criterion AICu,” Statistics and Probability Letters, 34 (3), pp. 285-292.
Cavanaugh, J.E., 1999, “A Large-Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence,” Statistics and Probability Letters, 42 (4), pp. 333-343.
Cavanaugh, J.E., 2004, “Criteria for Linear Model Selection Based on Kullback’s Symmetric Divergence,” Australian and New Zealand Journal of Statistics, 46 (2), pp. 257-274.
Seghouane, A.K. and Bekara, M., 2004, “A Small Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence,” IEEE Transactions on Signal Processing, 52 (12), pp. 3314-3323.
Hafidi, B. and Mkhadri, A., 2006, “A Corrected Akaike Criterion Based on Kullback’s Symmetric Divergence: Applications in Time Series, Multiple and Multivariate Regression,” Computational Statistics and Data Analysis, 50 (6), pp. 1524-1550.
Montgomery, D.C., Peck, E.A. and Vining, G.G., 2006, Introduction to Linear Regression Analysis, 5th ed., Wiley, New York. p. 67.
Keerativibool, W., 2014, “Unifying the Derivations of Kullback Information Criterion and Corrected Versions,” Thailand Statistician, 12 (1), pp. 37-53.
Keerativibool, W. and Siripanich, P., 2017, “Comparison of the Model Selection Criteria for Multiple Regression Based on Kullback-Leibler’s Information,” Chiang Mai Journal of Science, 44 (2), pp. 699-714.
Al-Subaihi, Ali A., 2007, Variable Selection in Multivariable Regression Using SAS/IML. [Online], Available: https://www.jstatsoft.org/article/view/v007i12/mv.pdf. [28 May 2016]
Whittaker, T.A. and Furlow, C.F., 2009, “The Comparison of Model Selection Criteria When Selecting Among Competing Hierarchical Linear Models,” Journal of Modern Applied Statistical Methods, 8 (1), pp. 172-193.
Acquah, H.G., 2010, “Comparison of Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) in Selection of an Asymmetric Price Relationship,” Journal of Development and Agricultural Economics, 2 (1), pp. 1-6.
Sangthong, M., 2019, “A Study of the Effectiveness of Model Selection Criteria for Multilevel Analysis,” Burapha Science Journal, 24 (1), pp. 156-169. (In Thai)
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 King Mongkut's University of Technology Thonburi
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Any form of contents contained in an article published in Science and Engineering Connect, including text, equations, formula, tables, figures and other forms of illustrations are copyrights of King Mongkut's University of Technology Thonburi. Reproduction of these contents in any format for commercial purpose requires a prior written consent of the Editor of the Journal.
