A recent study and research have shown that the relay selection schemes for cooperative communication can achieve full cooperative diversity gains and still maintains energy efficiency. Multi-source and multi-relay networks frequently cooperate to transfer their own data to the destination, mainly the access point, in cooperative wireless networks. In this paper, we use a jellyfish optimizer to select the number of relays to transmit packets of the image depending on the order of the data in the image according to its importance and at a constant rate. First, the relationships between the number of relays, peak signal-to-noise ratio, and signal-to-noise ratio are analyzed. Second, the comparison between the performances of Equal Diversity (ED) and Unequal Diversity (UED) with Equal Error Protection(EEP) and Unequal Error Protection(UEP) is introduced using the set partitioning in hierarchical trees (SPIHT) image coder as a source coding. The jellyfish algorithm, as an efficient optimization technique, is applied for solving the problem of optimal relay selection method to reduce the bit error rate and to improve the transmitted image quality. Many useful and important results in the proposed work can be used in the enhancement of image transmission over cooperative networks. Relative to the traditional methods, the proposed method achieves better results for the transmission of images with high quality through cooperative communication networks.