The aim of this study is to highlight how specific parameters, especially nonlinear feedback control, can enhance system stability and optimize vibration control. Our findings contribute to the ongoing development of advanced strategies for managing vibrations in nonlinear dynamic systems. In this research paper, we investigate the reduction of vibrations in a hybrid Rayleigh-van der Pol-Duffing oscillator using negative cubic velocity feedback control. This system is modeled as a single-degree-of-freedom oscillator that incorporates both cubic and fifth-order nonlinear terms, along with an externally applied force. To derive a solution from the initial approximation, the multiple scales method was utilized, providing an effective analytical approach for examining the nonlinear behavior of the system. We conducted a comprehensive analysis both graphically and numerically, examining the system's behavior before and after implementing negative cubic velocity feedback, with a particular focus on the primary resonance condition. MATLAB was used as the main computational tool to explore the effects of different parameters, including the impact of negative cubic velocity feedback on the primary system's response.