The correct classification of the type of brain tumor is critical in the early detection of the tumor, which can mean the difference between life and death. Implementing automated computer-aided approaches can help improve tumor diagnosis. We proposed a method for brain tumor classification via EfficientNetB3, a pre-trained model based on the transfer learning strategy. First, preprocessing images utilizing various methods, followed by classification of the preprocessed images using the fine-tuned EfficientNetB3 model. The suggested technique of fine-tuning pre-trained EfficientNetB3 is executed by first loading ImageNet weights to the EfficientNeB3 model, then adding several layers for the classification of brain tumor classes. A global average pooling (GAP) layer is used in our design to avoid over-fitting and Batch normalization layer to reduce losses. The proposed model was evaluated on 5712 images divided into four classes: glioma, meningioma, pituitary tumors, and normal which are shared publicly on Kaggle website. In addition, Multiple tests were run to assess the reliability of the proposed fine-tuned model in comparison to other traditional pre-trained models as well as other studies in the literature. The proposed framework achieved an accuracy of 97.7% with a minimum loss of 0.17. Also, the proposed method scored 95.6% for precision and F1-score respectively with only 20 epochs with Exponential Linear Unit (ELU) activation function at a threshold of 0.2 and Adam optimizer. We also evaluated the proposed model on two additional datasets to enhance generalizability. This model will certainly minimize detection complications and aid radiologists without requiring invasive procedures.