Adaptive Cruise Control (ACC) is an autonomous driving technology that enables vehicles to maintain a preset velocity and keep a safe distance from the lead vehicle by modifying the longitudinal acceleration of the ego vehicle. Reinforcement learning (RL) is a deep learning approach that learns by interacting with environments through trial and error. It receives rewards from the environment to evaluate its actions and improve its policies through learning algorithms. In this paper, an intelligent ACC system is proposed to enhance the system's overall performance by utilizing two main sub-systems. First, an ACC is implemented using the Twin Delayed Deep Deterministic (TD3) policy gradient algorithm with a continuous action space to control the acceleration of a vehicle. Second, a Lane Keeping Assist (LKA)is implemented using the Deep Q-Network (DQN) algorithm with a discrete action space for steering angle control. The system architectures and vehicle dynamics of the two sub-systems are modelled and simulated using Simulink. The TD3 and DQN algorithms are trained to perform ACC with LKA through cooperative behavior. Many experiments are carried out to evaluate the performance of the proposed system. The obtained results demonstrate the system's ability to follow a preset velocity, keep a safe distance from a lead vehicle, maintain the ego vehicle centered in its lane, and mitigate the risk of collision that may arise from lane changes.