Background: In early 2020, COVID-19 emerged as a health crisis and began affecting various dimensions of people's quality of life (QoL). This study assessed QoL during the COVID-19 outbreak in the Al-Baha region of the Kingdom of Saudi Arabia.Methods: A cross-sectional survey was conducted between January 2023 and February 2023 to gather information on the participants' sociodemographic characteristics and QoL. To assess QoL, 12 items were adapted from the World Health Organization Quality of Life Instruments (WHOQOL-BREF). To describe QoL scores, median and interquartile ranges were used. A multinomial regression analysis was performed between associated factors and QoL score quartiles, and the statistical significance level was set at p < 0.05. Results: This study involved 624 participants, with a median QoL score of 37 and an interquartile range of 32–43. The multinomial regression analysis results illustrated that male (OR = 1.64; 95% CI: 1.05–2.57) participants aged 18–25 years (OR = 2.31; 95% CI: 1.18–4.52) and those who were married (OR = 2.12; 95% CI: 1.25–3.57) were likelier to be in the first quartile of the QoL scores.Conclusions: The COVID-19 pandemic significantly affected several aspects of individuals' QoL. To mitigate the negative influences of the pandemic and enhance the QoL and health of the population, community-based interventions are required.