Load balancing is one of the main issues related to task scheduling in cloud environment. Load balancing aims at avoiding the situation whenever some virtual machines are overloaded and others are under loaded, or idle. It may cause Service Level Agreement (SLA) violation and further may lead to the reduction of customer satisfaction level. In this study, we propose Improved Active Monitoring Load Balancer with Hill Climbing Algorithm (IAMLBHC) to improve the performance metrics such as response time and processing cost. The proposed approach is simulated and implemented in CloudAnalyst simulator. The performance of the proposed approach is compared with the most recent existing load balancing algorithms via the simulation. The experimental results show that the proposed approach IAMLBHC gives better performance in terms of response time, and processing cost compared with the existing similar algorithms.