A variable selection method in multiple linear regression models based on  Tabu Search

Kannat Na Bangchang

Authors

Kannat Na Bangchang Faculty of Science and Technology, Valaya Alongkorn Rajabhat University, Patumthani 13180, Thailand

Keywords:

stepwise regression, Tabu Search, variable selection

Abstract

This research has proposed a variable selection method based on the Tabu Search for multiple linear regression models. In this study two objective functions used in the Tabu Search are mean square error (MSE) and the mean absolute error (MAE). The results of the Tabu Search are compared with the results obtained by the stepwise regression method based on the hit percentage criterion. The simulations cover both cases, without and with multicollinearity problems. Without multicollinearity problem, the hit percentages of the stepwise regression method and the Tabu Search using the objective function of MSE are almost the same but slightly higher than the Tabu Search using the objective function of the MAE. But with multicollinearity problem the hit percentages of the Tabu Search using the objective function of MSE or the MAE are higher than the hit percentage of the stepwise regression method. Additionally, the correlation coefficients between the independent variables X₁ and X₄ are higher; yielding hit percentages that are lower.

References

Bruce, L. B., David, A. D., & Richard, T. O. (1990). Linear Statistical models: an applied approach (2nd ed.). Boston, USA: Duxbury Press.

Efroymson, M. A. (1960). Multiple regression analysis. In A. Ralston & H.S. Wilf (Eds.), Mathematical methods for digital computers (pp. 191-203). New York, USA: Wiley.

Eksioglu, B., Demirer, R., & Capar, I. (2005). Subset selection in multiple linear regression: a new mathematical programming approach. Computer & Industrial Engineering, 49(1), 155-167.

Glover, F. (1990). Tabu search: A tutorial. Interfaces, 20(4), 74-94.

Gujarati, D.N. (2009). Basic Econometrics (4th ed.). New York, USA: McGraw-Hill.

Holland, J. H. (1975). Adaptation in natural and artificial systems. Ann Arbor, MA, USA: The University of Michigan Press. Republished by the MIT Press 1992.

Knox, J. (1989). The application of tabu search to the symmetric traveling salesman problem. A doctoral dissertation. University of Colorado at Boulder, CO, USA.

Montgomery, D. C., Peck, E. A., & Vining, G. G. (2006). Introduction to linear regression analysis (4th ed.). New Jersey, USA: John Wiley & Sons, Inc.

Pacheco, J., Casado, S., & Nunez, L. (2009). A variable selection method based on Tabu Search for logistic regression models. European Journal of Operational Research (EJOR), 199(2), 506-511.

Piriyakul, M. (1986). Regression Analysis. Bangkok, Thailand: Chuanpim Press. (In Thai)

Sacchi, L. H., & Armentano, V. A. (2011). A Computational study of parametric tabu search for 0-1 mixed integer programs. Computer & Operational Research, 38(12), 464-473.

Salhi, S. and Drezner, Z. and Marcoulides, G. (1999) Tabu search model selection in multiple regression analysis. Communication in Statistics: Simulation and Computation, 28(2), 349-367.

Seenoi, P. (2010). Test Statistics for selecting multiple linear regression models. (Master’s thesis, National Institute of Development Administration, Bangkok, Thailand). Retrieved from http://libsearch.nida.ac.th

Shen, Q., Shi, W., & Kong, W. (2010). Modified tabu search approach for variable selection in quantitative structure-activity relationship studies of toxicity of aromatic compound. Artificial Intelligence in Medicine, 49(2), 61-66.

Siary, P., & Berthiau, G. (1997). Fitting of tabu search to optimize functions of continuous variables. International Journal for Numerical Methods in Engineering, 40(13), 2449-2457.

Smith, D., & Richard, N. (1981). Applied regression analysis (2nd ed.). New York, USA: Wiley & Sons, Inc.

Turajlić, N., & Dragović, I. (2012). A hybrid metaheuristic based on variable neighborhood search and tabu search for the web service selection problem. Electronic Notes in Discrete Mathematics, 39(1), 145-152.