Keeping high levels of physical activity is a challenge among chronic patients. In this regard, self-efficacy has been identified as a crucial variable to reduce sedentarism and physical inactivity in women with fibromyalgia. The current study aimed to evaluate the associations among objective physical activity levels, self-efficacy, activity patterns, and the impact of the disease, as well as to compare those variables between women with fibromyalgia with different self-efficacy levels. For this purpose, in this cross-sectional study, the physical activity levels of 123 women with fibromyalgia were assessed by accelerometers, together with self-efficacy, the impact of the disease, and activity patterns. Results revealed that self-efficacy for light or moderate physical activity was directly related to light ( < 0.01), moderate ( < 0.01), and vigorous physical activity ( < 0.05), as well as inversely related to sedentary time ( < 0.01). Moreover, the main differences were observed between those with low self-efficacy levels and the rest of the sample, while there were no differences between the high and the medium self-efficacy groups ( > 0.05). Thus, self-efficacy for walking and light physical activity seems to be more relevant than self-efficacy for moderate and vigorous physical activity to achieve higher levels of physical activity.