Chronic low back pain (cLBP) is associated with widespread functional and structural changes in the brain. This study aims to investigate the resting state functional connectivity (rsFC) changes of visual networks in cLBP patients and the feasibility of distinguishing cLBP patients from healthy controls using machine learning methods. cLBP (n = 90) and control individuals (n = 74) were enrolled and underwent resting-state BOLD fMRI scans. Primary, dorsal, and ventral visual networks derived from independent component analysis were used as regions of interest to compare resting state functional connectivity changes between the cLBP patients and healthy controls. We then applied a support vector machine classifier to distinguish the cLBP patients and control individuals. These results were further verified in a new cohort of subjects. We found that the functional connectivity between the primary visual network and the somatosensory/motor areas were significantly enhanced in cLBP patients. The rsFC between the primary visual network and S1 was negatively associated with duration of cLBP. In addition, we found that the rsFC of the visual network could achieve a classification accuracy of 79.3% in distinguishing cLBP patients from HCs, and these results were further validated in an independent cohort of subjects (accuracy = 66.7%). Our results demonstrate significant changes in the rsFC of the visual networks in cLBP patients. We speculate these alterations may represent an adaptation/self-adjustment mechanism and cross-model interaction between the visual, somatosensory, motor, attention, and salient networks in response to cLBP. Elucidating the role of the visual networks in cLBP may shed light on the pathophysiology and development of the disorder.