We did an experiment on you. Well, it wasn’t really on you. Although it did involve you. Anyway, here it is: Some time ago, we did an experiment to see if all this work we are putting in at BiM actually has an effect on dissemination of the research we are bringing to you all. Our intent was to try to measure what is widely accepted as fact – that social media release increases impact of research.
Here is what we did: we randomly picked 16 articles that had been published in PLOS journals and had investigated something broadly within the clinical pain sciences. We then randomly picked two dates from a list of possible post dates. We called one of those dates ‘The Release Date’ and the other “The Control Date’. Then we compared the HTML views and PDF downloads in the week before and the week after each date.
For those experimental design fans, this is sort of like a randomised repeated measures case-control comparison. For those of you who are not experimental design fans, we reckon the study was as rigorous an investigation as we could have feasibly undertaken. Anyway, we were looking for an idea of how many people who, after they were alerted to the article via a blog post that reviewed the article, then went to look at it and then download it.
Our original title for the paper was ‘A drop in the ocean?’ because we really were not confident that our humble website would make a measurable difference to accesses to the article. Our results showed that we underestimated you all! Figure 4 in the paper shows the effect.
The lines on the left show PDF downloads the week before and the week after The Control Date – no change. The lines on the right show PDF downloads the week before and the week after The Release Date – clearly an increase. In absolute numbers – our release meant that, in the week following the Release, about 12 extra people per day viewed the article at PLOS and about 3 extra people per day downloaded it. In stats-speak, we had an effect size of about 1. If we were talking a new drug for headache, we would be very excited by such an effect size.
We reckon that this is the first attempt to apply empirical experimental methods to test whether or not a blog works. We reckon our experiment shows it does. Of course HTML views and PDF downloads are not necessarily the primary outcome of interest for BiM, but it is the closest we could get and the results are encouraging to say the least. Anyway, we would love you to check out the article itself. It is free and it is here and if you do have a look, it boosts our social media-based metrics (check out the last post to know what that really means).
About Lorimer Moseley
Lorimer is NHMRC Senior Research Fellow with twenty years clinical experience working with people in pain. After spending some time as a Nuffield Medical Research Fellow at Oxford University he returned to Australia in 2009 to take up an NHMRC Senior Research Fellowship at Neuroscience Research Australia (NeuRA). In 2011, he was appointed Professor of Clinical Neurosciences & the Inaugural Chair in Physiotherapy at the University of South Australia, Adelaide. He runs the Body in Mind research groups. He is the only Clinical Scientist to have knocked over a water tank tower in Outback Australia.
Link to Lorimer’s published research here. Downloadable PDFs here.
References
[1] Allen HG, Stanton TR, Di Pietro F, & Moseley GL (2013). Social media release increases dissemination of original articles in the clinical pain sciences. PloS one, 8 (7) PMID: 23874810