Despite the high prevalence and socioeconomic impact of chronic low back pain (cLBP), treatments for cLBP are often unsatisfactory, and effectiveness varies widely across patients. Recent neuroimaging studies have demonstrated abnormal resting-state functional connectivity (rsFC) of the default mode, salience, central executive, and sensorimotor networks in chronic pain patients, but their role as predictors of treatment responsiveness has not yet been explored. In this study, we used machine learning approaches to test if pre-treatment rsFC can predict responses to both real and sham acupuncture treatments in cLBP patients. Fifty cLBP patients participated in 4 weeks of either real (N = 24, age = 39.0 ± 12.6, 16 females) or sham acupuncture (N = 26, age = 40.0 ± 13.7, 15 females) treatment in a single-blinded trial, and a resting-state fMRI scan prior to treatment was used in data analysis. Both real and sham acupuncture can produce significant pain reduction, with those receiving real treatment experiencing greater pain relief than those receiving sham treatment. We found that pre-treatment rsFC could predict symptom changes with up to 34% and 29% variances for real and sham treatment, respectively, and the rsFC characteristics that were significantly predictive for real and sham treatment differed. These results suggest a potential way to predict treatment responses and may facilitate the development of treatment plans that optimize time, cost, and available resources.