Gastrointestinal (GI) system diseases have increased significantly, where colon and rectum cancer is considered the second cause of death in 2020. Wireless Capsule Endoscopy (WCE) is a revolutionary procedure for detecting Colorectal lesions. It was automatically used to detect the polyps, multiple SB lesions, bleeding, and Ulcer. The acquired video by the WCE can be processed using a Computer-Aided Diagnosis (CAD) system. However, such videos suffer several problems, including burling, high illumination. and distortion. These effects obligate the development of image processing techniques of high accuracy in detection using deep learning-based segmentation. In this paper, a transfer learning-based U-Net was proposed to transfer the knowledge between the medical images in the training phase and the subsequent segmentation using transfer learning to achieve better results and high accuracy results compared to other related studies. The improvement is done by using an algorism written in python code The results showed average segmentation accuracy of 98.67%