In the agricultural sector, a huge threat is posed by harmful insects and plant diseases. Hence, it is important to conduct early diagnosis and detection of such diseases. Detecting plant diseases is now possible with the great aid from continuously developing deep learning methods which represent a robust tool, rendering remarkably meticulous results. However, the deep learning models' accuracy is dependent on the labeled training data quality and volume. Accordingly, this paper proposes a deep learning-based method for detecting tomate disease, combining recurrent neural network (RNN) architecture with the convolutional neural network (CNN) for this purpose. Xception-GRU is the proposed model as it begins with the Xception pre-trained model and is followed by the GRU layers. After that, transfer learning is employed for training the Xception-GRU model on real and synthetic images for tomato leaves images to be classified into 10 disease categories.Three different classifiers are used on the features extracted from the Xception-GRU model. These classifiers are multi-layer perceptron (MLP), support vector machine (SVM), and the k-nearest neighbor (KNN). With extensive testing and training on PlantVillage dataset, available publicly, the proposed model reached 100%, 98.79%, 99.85%, and 100% accuracy in classifying tomato leaf image diseases into (Early, Late Blight and Healthy), (Early and Late Blight), (Healthy and Late Blight) and (Early and Healthy). Hence, it is shown that the approach proposed is superior over the present methodologies.