A deep learning convolutional deep neural network (CNN) is employed to build robust classifiers. These classifiers can analyse large amounts of data, find statistical dependencies, learn correlations between features, and generalise their findings. This paper proposes the use of Mean Absolute Error (MAE), Sum of Square of Errors (SSE) and cross entropy loss functions to create two new classification layers, which are employed as the last layer in the proposed convolutional deep neural network -based classifier. A comparative study was conducted to assess the performance of the presented convolutional deep neural network -based classifiers using three different loss functions (cross entropy "conventional", MAE, and SSE) -based classification layers and Adam (adaptive moment estimation), and SGdm (stochastic gradient descent with momentum) optimizers. In addition, the accuracy and loss curves that resulted from the training process are provided for comparison purposes. Handwritten digits dataset was used as a classification case study using the proposed classifiers.