The email from the industry was effusive. In a cock-a-hoop, caps lock-happy frenzy it bellowed “ALL MANUAL MEDICINE PROVIDERS SHOULD BE AWARE OF THIS STUDY”. The study in question, soon to be published in the journal “Spine” is a RCT that specifically looks at whether patients with chronic back pain benefit from a sustained period of “maintenance spinal manipulation” following their initial treatment period and concludes that SMT is indeed effective but that “maintenance manipulations” add benefit after the initial intensive therapy has concluded. No wonder the mailer was excited; here was a study that purportedly demonstrates a real benefit to spinal manipulation in chronic back pain and that seems to validate that common but controversial practice of regularly seeing patients between flare-ups for a “quick click” to keep the spine tip-top.
This was a small study of 60 patients divided into 3 groups: one received 12 sessions of “sham” manipulation over one month, one received 12 sessions of real manipulation over a month (one of those general “give the back a good click” manips), and the third group received the same plus an additional “maintenance spinal manipulation” every 2 weeks for nine months. The authors report that both groups that received real manipulation did better than the sham group but the group that got the maintenance manipulation did the best of all.
Now 60 people in 3 groups should not really provide sufficient statistical power to demonstrate a difference, even if one is there. So it would suggest that the effect of this manipulation is either uncharacteristically (for a manual therapy) consistent, or large when it is present. Put into context the recently updated Cochrane review of all trials of manipulation for chronic back pain suggests a tiny effect size for manipulation that doesn’t really tickle the undercarriage of clinical significance.
Closer inspection of the paper revealed a few devils lurking amidst the details. There are serious challenges to devising a true sham manipulation. In fact depending on what one thinks the “active ingredient” of a manual therapy is it may well be considered impossible. The authors did not investigate whether or not the sham was convincing to the folk who took part but there is a possible clue in the fact that half of those in the sham manipulation group dropped out during the study compared with fewer dropouts in the other groups. The authors have tried, fairly, to account for this in the analysis but this level of uneven dropout makes any comparisons with the sham group unreliable. We cannot be clear that manipulation was truly better than the sham.
In terms of the effects of maintenance manipulation there is a further issue. Given that this group is basically compared to a “no maintenance treatment” group there is no control for non-specific effects of care such as placebo, attention, etc. Perhaps the manipulation is a red herring; maybe, if the observed effects are real, it just helps patients to be seen regularly. There is not much data about but another new trial, this time for chronic neck pain might shed some light. Here, maintenance manipulation was compared with manipulation plus exercise and with a control group that met with their chiropractor to discuss their symptoms but who received no manipulation or exercise, thereby controlling for the effects of attention. They found no benefit to maintenance manipulation or manipulation plus exercise for any outcome, although this study also suffers from its small size.
Beyond these issues a few other concerns spring out at me that question not just the interpretation of the effect but whether it is really there at all. The level of variability in this sample of patients seems remarkably low. In fact calculating and comparing the standard deviations with those of the studies included in the recent Cochrane meta-analysis of manipulation for chronic back pain they are around half the size and sometimes less. The criteria for including patients don’t seem remarkable so what else could explain such a consistent population?
The results at first appear statistically super significant with a p-value of p>0.001 for many comparisons. Simply put, this indicates that the chances of us getting the same positive result by chance despite the treatment actually being useless are less than 1 in 1000. Unfortunately the authors decided to use a statistical test called the t-test for every comparison. If we include all of the outcomes and all of the follow-up points that means 72 of them! This is something of a methodological no-no because it means that each time we take the test we increase the chances of finding a false positive. A quick, dirty and imperfect correction for this suggests that for this study p<0.001 actually equates to p<0.072 which we would not accept as statistically significant. This choice of statistical approach seems a touch bizarre, and I can’t really think of a good reason that it might have been chosen.
It is of course possible that the results of this study are accurate and maintenance manipulations are effective, but these problems make it difficult to judge. The message from this one back pain trial might seem appealing and I can see why the email was so enthusiastic. But by focusing on one particular cherry that seems so ripe and juicy we might miss the bigger picture from the rest of the tree. And there is always the chance that the tastiest cherries contain a few artificial sweeteners. Personally I would lay off the caps lock for now.
About Neil
Neil O’Connell is a researcher in the Centre for Research in Rehabilitation, Brunel University, West London, UK. He divides his time between research and training new physiotherapists and previously worked extensively as a musculoskeletal physiotherapist. He also tweets! @NeilOConnell
Neil is currently fighting his way through a PhD investigating chronic low back pain and cortically directed treatment approaches. He is particularly interested in low back pain, pain generally and the rigorous testing of treatments. He also tends to get all geeky over controlled trials.
Senna MK, & Machaly SA (2011). Does maintained Spinal manipulation therapy for chronic non-specific low back pain result in better long term outcome? Spine PMID: 21245790
Rubinstein SM, van Middelkoop M, Assendelft WJ, de Boer MR, & van Tulder MW (2011). Spinal manipulative therapy for chronic low-back pain. Cochrane database of systematic reviews (Online), 2 PMID: 21328304
Martel J, Dugas C, Dubois JD, & Descarreaux M (2011). A randomised controlled trial of preventive spinal manipulation with and without a home exercise program for patients with chronic neck pain. BMC musculoskeletal disorders, 12 PMID: 21303529