Traffic jam is a crucial issue affecting cities around the world. They are only getting worse as the population and number of vehicles continues to increase significantly. Traffic signal controllers are considered as the most important mechanism to control the traffic, specifically at intersections, the field of Machine Learning offers more advanced techniques which can be applied to provide more flexibility and make the controllers more adaptive to the traffic state. Efficient and adaptive traffic controllers can be designed using a multi-agent reinforcement learning approach, in which, each controller is considered as an agent and is responsible for controlling traffic lights around a single junction. A major problem of reinforcement learning approach is the need for coordination between agents and exponential growth in the state-action space. This paper proposes using machine learning clustering algorithm, namely, hierarchal clustering, in order to divide the targeted network into smaller sub-networks, using real traffic data of 65 intersection of the city of Ottawa to build our simulations, the paper shows that applying the proposed methodology helped solving the curse of dimensionality problem and improved the overall network performance.