Image analysis algorithm was developed to provide the classification parameters for defect of yellow apple fruits. The objectives of this work are: 1) To use and test an image processing system which could classify defects on apple surfaces based on color analysis,and2) Extraction of values of Red, Green and Blue (RGB) coordinates and color properties of Hue, Saturation, and Intensity (HSI) from the infected area. Experimental setup of color analysis was used for taking an image for each apple sample to find and identify apple defect. The results indicated the following points: (1) The value of red color component was decreased (162 - 73.4), (170.4 - 104.4), (210.8 - 98.4), and (202.6 - 113.1 values), and the value of green color component was decreased (148.6 - 73.5), (122.2 - 61.2), (181 - 62.4), and (180.1 - 90.3 values). Meanwhile, the blue color component value changed a little, it is ranged (61.6- 61.9), (61.1- 63.4), (61.4- 62.0), and (61.6- 62.7 values), for color, fungi, bruises and operation defects (which classified into four subclasses of each defect), respectively., (2) The value of red color component was decreased (134.6 - 84.4), (203 - 186.6), (143.8 - 98.4), and (202.8 - 174.9 values), and the value of green color component was decreased (100 - 75.2), (131.4 - 118.), (115.4 - 61.4), and (184.8 - 154.6 values), respectively. Meanwhile, the blue color component value changed a little, it is ranged (62.1- 66.8), (61.6- 62.0), (61.4- 63.4), and (62.4- 62.6 values) for insect, scare, mechanical and russet defects of apple fruit (which classified into three subclasses of each defect), respectively., (3) The value of color component (R,G and B) and color properties (H, S and I) were very important to classify defect of apple fruit., (4) Using of image analysis method in estimation of apple defects, to develop of a sorting system for sorting of apples fruit based on color analysis., and (5) Establishment measuring standard for defects of apple fruits according to image processing technique.