The structural impact of chronic pain on amygdala in chronic pain (CP) patients remains unclear although major depression and anxiety are known to be associated with its increase and decrease in size, respectively. This study aimed at examining the relationship between emotional stress and amygdala size in CP patients. The effects of mediating and moderating variables were also examined. The PubMed, Embase, and Web of Science databases were searched for English clinical trials from inception to February 2022 using the appropriate keyword strings. We compared the differences in amygdala size assessed with magnetic resonance imaging between CP patients with emotional stress and healthy counterparts. Of the 49 full-text articles identified, 13 studies enrolling 1551 participants including 738 CP patients with emotional stress and 813 controls were analyzed. Emotional stress evaluated with questionnaires based on Beck depression inventory, Hamilton depression/anxiety scale, state-trait anxiety inventory, and hospital anxiety and depression scale revealed significant differences between CP patients with emotional stress and controls, indicating a subclinical but significant level of emotional stress in CP patients. The results demonstrated an amygdala shrinkage among CP patients with emotional stress compared to the controls, especially the right side (p = 0.02). Besides, pain from a single body region was more likely to impact the amygdala size compared to diffuse pain (p = 0.02). Regression analysis revealed no significant association between continuous variables (age, gender, pain duration/intensity) and amygdala size. Our findings demonstrated that emotional stress was associated with a reduced right amygdala size in CP patients.