The COVID-19 pandemic resulted in millions of infections which led to increased demands on health systems around the world. Due to the shortage of diagnostic tools and the stress on radiologists, the need to utilize computer-assisted methods to diagnose COVID-19 has increased. There have been many attempts to use deep learning to accelerate the process of COVID-19 diagnosis. However, there is still an opportunity for further improvements in the results. In this paper, we present a comparative study for COVID-19 diagnosis using multiple convolutional neural networks, as they are the most widely used architectures in classification problems. We trained the convolutional neural networks (CNNs) using 5-fold cross-validation. We used lung ultrasound images proposed in the Point of Care Ultrasound (POCUS) dataset. InceptionV1 achieved the highest results with accuracy and balanced accuracy of 84.3% and 81.8%, respectively. Qualitatively, employed architectures show a variation in performance depending on the internal layers of each architecture. A deep learning architecture can distinguish similar-looking lung ultrasound pathology, including COVID-19, that may be difficult to distinguish by pathologists and radiologists.