This paper aims to present in details the most common challenges faced by slum dwellers and rise during the upgrading and replacement projects in both Makkah and Jeddah in Saudi Arabia. The research focuses on social, economical and constructional challenges from the dwellers point of view. The slum problem is growing rapidly in Saudi Arabia in general. Makkah and Jeddah are two main cities which have the slums problems very clear. Many researches explored the slums phenomena in the two cities, but they did not cover all aspects of the problem. Many social, economical and constructional challenges need to be studied. To achieve this goal, an interviews survey was conducted with 36 family heads living in slums in Jeddah and Makkah. The required data were formed in a survey consisting of 26 questions about the different challenges. The questionnaire was classified into four sections including general information, current available facilities, presence of upgrading or replacement projects, and finally the social challenges impact on slum dwellers. The questionnaire results are analyzed and findings are extracted. Regarding replacement projects, the most emphasized survey result is the unsatisfaction of the dwellers toward the properties compensation amount due to high prices of lands and construction costs of new alternative houses. The most impressive social challenge is the negative impact of desertion of the place in which dwellers were born and lived for most of their lives which lead to the rejection of slum projects. Sometimes, this rejection took the form of violent resistance of dwellers to evacuate slums as a preparatory stage for the demolition. Therefore, the majority of such projects were usually delayed and stopped for months. Other obstacles were represented in the closure of roads during the upgrading projects and consequently, the halting of the economic activities such as local commercial shops. It is highly recommended to revise the properties compensation system. Also, the quality of life in the upgraded slums should be maximized through the improvement of different infrastructure services including pure water, sanitary, and electricity services. Some other services can be introduced such as the construction of new parks, playgrounds, sporting clubs and other means of entertainment for the sake of the current dwellers and the next generations.