BAYESIAN ESTIMATORS OF NORMAL LINEAR REGRESSION MODEL IN THE PRESENCE OF MULTICOLLINEARITY

Abstract

Multicollinearity arises in econometrics when the regressor is linearly related to other regressors in a Normal Linear Regression Model (NLRM). A major drawback of the classical approach to the estimation of NLRM is that it is indeterminate in the presence of extreme perfect multicollinearity. The use of out-of-sample information by the Bayesian approach to resolve this problem has not been fully explored in existing literature on the subject. Therefore, the Bayesian technique was employed to derive estimators for a NLRM and investigate the sensitivity of the estimators to various degrees of collinearity among the regressors.

The NLRM y = Xθ + ε , wherey is (N x 1) vector of the response variable, X is a (N x k) matrix of regressors, θ is (k x 1) vector of parameters and ε is a (N x 1) vector of normally distributed random error with zero mean and variance σ^2 was specified. Six cases of collinearity: case I- High Positive Collinearity (HPC) (0.50≤HPC≤0.99); case II- Moderate Positive Collinearity (MPC) (0.30≤MPC≤0.49); case III- Low Positive Collinearity (LPC) (0.01≤LPC≤0.29); case IV- High Negative Collinearity (HNC) (-0.99≤HNC≤-0.50); case V- Moderate Negative Collinearity (MNC) (-0.49≤MNC≤-0.30); case VI- Low Negative Collinearity (LNC) (-0.29≤LNC≤-0.01) and No Collinearity (NC) were investigated. Two Bayesian out-of-sample priors: Bayesian Informative Prior (BIP) with Normal-Gamma prior and Bayesian Non-informative Prior (BNIP) with a local uniform prior were derived and their estimates compared with classical method, namely, Likelihood Based (LB) method for all the cases of collinearity considered. Data were simulated for all the cases of collinearity for sample sizes 10, 30, 70, 100, 200 and 300. The performances were judged using Standard Error (SE) and Confidence Interval (CI). Therefore, the estimator with the minimum SE and compact CI were considered the most efficient estimator. The derived Bayesian estimators were P(θ|y) ∝ t(θ,h^(-1) Q,v) for BIP and P(θ|h) ∝N(θ,h^(-1) Q) for BNIP, where h, Q and v are precision, un-scaled variance-covariance matrix and degree of freedom, respectively. The SE and CI of BIP, BNIP and LB for HPC were [0.3843, (4.4636≤CI≤5.9949)] [2.1099, (-0.490≤CI≤7.9250)] and [2.1729, (-0.6213≤CI≤ 8.0553)]; for MPC were [0.3870, (4.3608≤CI≤5.8822)], [1.1111, (2.0341≤CI≤6.4023)] and [1.1278, (1.9665≤CI≤6.4700)]; LPC were [0.3963, (4.4893≤CI≤6.0686)], [0.9032, (2.8449≤CI≤6.4475)] and [0.9301, (2.7892≤CI≤6.5033)]; HNC were [0.008, (9.9985≤CI≤10.0015)], [1.6369, (7.1784≤CI≤13.7071)] and [1.6856, (7.0774≤CI≤13.8081)]; MNC were [0.0009, (9.9983≤CI≤10.0017)], [0.4748, (7.5869≤CI≤9.4810)] and [0.4890, (7.5576≤CI≤9.5103)]; LNC were [0.6201, (1.3167≤CI≤3.7879)], [0.7658, (0.4447≤CI≤3.4995)] and [0.7887, (0.3974≤CI≤3.5468)] and NC were [0.5350, (0.9025≤CI≤3.0345)], [0.6958, (-0.0468≤CI≤2.7286)] and 0.7166, (-0.0897≤CI≤2.7716)], respectively. Thus, Bayesian estimators BIP and BNIP were less sensitive with minimum values of SE and narrower CI of parameter estimates for all the cases of collinearity considered compared to LB estimator. The Bayesian estimators outperformed the LB for all the cases of collinearity considered, while BIP outperformed BNIP. The derived Bayesian estimators for normal linear regression model provided better estimates than the classical method at various degrees of multicollinearity. They are also less sensitive to the problem of collinearity and capable of handling perfect correlation.