In this study, we constructed a structural equation model (SEM) for predicting the quality of life (QOL) in elderly Koreans with chronic musculoskeletal pain (CMP) and examined the differences between sexes. Data were earlier collected in a prior study of 307 participants (101 men and 206 women) with CMP, aged 65 years and above, who used geriatric welfare centers located in two cities. The effects of pain, functional limitation, perceived health status, pain coping, and social support on the QOL were estimated with a multigroup SEM. For both men and women, the results show sequential causality from pain to functional limitation, perceived health status, and QOL. However, the relationships among pain, pain coping, functional limitation, and QOL differ between men and women. The multigroup SEM provides a better understanding of the sex differences in the QOL of elderly with CMP. The results suggest that in order to improve QOL among the elderly with CMP, a customized strategy should be applied that takes into account differences between the sexes.