Principal component extraction with no information loss

Abstract

In Principal Component (PC) analysis of an r × p variance-covariance (V-C) matrix, there is always a loss of information when the first few set of r(< p) PCs are retained. This study derives a new reduced set of PCs (NRPCs) that is simply a constant multiple of the first r original PCs (OPCs). Thus, the OPCs are just a normalization of the NRPCs. The normalizing constant represents the common variance explained by each of the components in the set of r NRPCs. Further features of the NRPCs are examined both analytically and practically in Multivariate Multiple Reduced Rank Regression (MMRRR) modelling. It is found that for the NRPCs extracted from regular (unweighted) V-C matrix, the analytical relationship between the NRPCs and the OPCs are preserved in MMRRR modelling. However, if OPCs are based on weighted V-C matrix, then the analytical relationship between the two types of PCs does not hold practically in MMRRR modelling. The results of the study shows that in order to determine the real spread of PC scores for further analysis, the use of the NRPCs would be more useful.