Plant classification and recognition is a vital task for many applications including retail, agriculture, and food processing. Several state-of the-art algorithms were developed in order to address this challenge. While there are several papers that were published to propose classification algorithms for plants and fruits, these algorithms targeted small datasets with 10K or fewer images. This study compares the performance of several Deep Learning models in fruit and plant classification on a large plants and fruits dataset with more than 225k images. This study aims to guide researchers about the performance implications of using popular models at large-scale by testing the scalability and reliability of these models. Fruits-262 is a dataset of over 225k images representing 262 different classes of plants and fruits. To ensure fairness, we trained each model using the same runtime environment on the Fruits-262 dataset. The models were evaluated using four different evaluation metrics; accuracy, loss, validation accuracy and validation loss. We also considered the computational complexity, the training time and the model size in order to evaluate the efficiency, reliability and scalability. Our findings reveal that there are some models that can offer high classification accuracy, yet with high computational resources and long iterations. This paper explores some potential alternatives and highlights some interesting models for future research.