Cities with high population density have a serious problem with traffic congestion. Intelligent transportation systems try to overcome these problems by finding smart ways to detect traffic congestion. One of the essential issues in these systems is selecting the appropriate features to detect traffic congestion. Most of the current methods utilize motion or texture features only, which have their limitations. In this paper, a deep neural network (DNN), which has two input paths, is proposed for traffic congestion recognition. It handles the evolution of motion as well as texture through its two inputs simultaneously via Long Short-Term Memory (LSTM) layers. Gaussian noise layers are used to increase the generalization ability of the DNN and to enable training on small datasets without over-fitting. Experimental results applied to the UCSD and NU videos datasets assert the robustness of the proposed method. It achieves an accuracy of 98 % which is high in comparison to the state-of-the-art methods.