Linear regression is one of the simplest and widely used machine learning algorithms that has received a lot of attention in many fields. Linear Regression Classification (LRC) algorithm depends on represents each class's training images independently in a linear regression relationship. The algorithm depends on applying the least squares method to find the regression coefficient then decides the class label with the smallest reconstruction error. In this paper, we propose a classification by principal component regression (CbPCR) strategy, which depends on performing regression of each data class in terms of its principal components. This CbPCR formulation leads to a novel formulation of the LRC problem that keeps the key information of the data classes while providing more compact closed-form solutions. We also extend this strategy to the 4D hypercomplex domains to take into account the color information of the image. Our experiments on two color face recognition benchmark databases prove the efficacy of the proposed strategy.