Cluster headache (CH) shows a more severe clinical picture than migraine (Mig). We tested whether brain changes can explain such difference. Multimodal MRI was acquired in attack-free patients with CH (n = 12), Mig (n = 13) and in normal controls (NC, n = 13). We used FSL for MRI data analysis and nonparametric permutation testing for voxelwise analyses (p < 0.01, corrected). CH showed lower grey matter (GM) volume, compared to Mig and NC, in frontal cortex regions (inferior frontal gyrus and frontal pole [FP], respectively) and, only compared to Mig, in lateral occipital cortex (LOC). Functional connectivity (FC) of CH was higher than Mig and NC within working memory and executive control networks and, only compared to Mig, between cerebellar and auditory language comprehension networks. In the attack-free state, the CH brain seems to be characterized by: (i) GM volume decrease, compared to both Mig and NC, in pain modulation regions (FP) and, only with respect to Mig, in a region of visual processing modulation during pain and working memory (LOC); (ii) increased FC at short range compared to both Mig and NC and at long range only with respect to Mig, in key cognitive networks, likely due to maladaptation towards more severe pain experience.