The mean shift algorithm is a widely used tool for robust and real-time object tracking that can use
any feature space such as the color space. Therefore, in this article, we have studied and analyzed
the various effects of using different color space configurations with the mean shift algorithm on the
tracking process in surveillance videos. We have done so by investigating the potency of the gray
space and four other color spaces (RGB, HSV, YIQ, YCbCr [each color space was studied under all
the available seven configurations]). In the end, we have reached a conclusion that deciding which
color space (or which configuration of the color space channels) to be used not only can be a very
critical decision which may affect greatly the results of the tracking process, but also there is no
color space configuration that can succeed all the time in all situations. Therefore, it is better to
select the appropriate color space configuration according to the situation at hand.