I can only speak for myself, but the idea of peer review usually fills a researcher with the diametrically opposed feelings of fear of failure and a genuine excitement that the imminent suggestions will help one’s paper shine. It doesn’t help that the time of judgment usually comes after one has spent countless hours working on presenting an idea!
The reviewer has two roles. Firstly, that of a colleague who can approach the paper with a “fresh set of eyes” and provide helpful commentary. Secondly, that of a member of the jury who may condemn the researcher to more community service (hours of extra work) or even temporary professional death (“Sorry, we’re just not that interested in your paper. Perhaps you could try…”). Unsurprisingly, the peer review process has, on more than one occasion, wriggled its way into BIM meeting discussions. So, it is with interest, that I read this new systematic review of peer-review by Larson and Chung on the eve of finalising a paper for submission.
Did you know that apparently the first medical journals were published some 300 years ago? And that the idea of peer review arose about the same time? Even the earliest intellectuals realised that publishing any old article may not be such a good idea. And, of course, the newly formed journals really did not want to be immediately discredited by allowing poor quality material (or, worse, fraudulent material!) through their doors.
But is peer review as important as we normally accept it to be?
To begin with, we can question the validity of the very premise that an educated community of scientists and clinicians are not able to independently judge the quality of a research paper. The clear consensus seems to be that allowing this is not a good idea! Researchers and clinicians do not always have the time and sometimes do not have the requisite skills and knowledge to make those decisions. In our busy lives, we appreciate the presence of a seemingly impartial filter that screens out ““irrelevant, trivial, weak, misleading or potentially harmful content” while also “improving the clarity, transparency, accuracy, and utility” of potential papers”” (Larson and Chung, 2012). In days gone by, some journals distinguished peer-reviewed articles by granting a title of approval like “Academician”. On the other hand, those published “sit penes authorum fides” allowed the authors to shoulder responsibility for their own work. This distinction was deemed to be adequate. However, now we think that simply allowing an author to take responsibility for their own article is inadequate as we also need to protect the reader from ignoring that warning! And, to be honest, it is nice to not have to read through too much irrelevant or bad science to find the gems. Implicit in this statement is a trust that the reviewers generally get it right.
What about conflicts of interest? The world of research has become very large and we sometimes see potential conflicts of interest in allowing research to be reviewed by external parties. We may wonder: Is the reviewing committee independent? Is it possible for conflict of interest to creep into the modern peer review (when was the last time you asked if the peer review process was blinded?)? Could the reviewer gain from our work (apparently this has happened!)?
So, as the authors of the review suggest, there are weaknesses in the peer review process. Firstly, it limits the publication of material that may be of significant interest to some. Perhaps an infinitely large database of research publicly available is not such a bad idea. Secondly, there is no uniform approach to peer review. The most common versions (the single-blind, the double-blind, open, peer agreement, and author suggestion-based) each suffer from positives and negatives.
So what’s the take home message from the article? As a minimum, reviewers should be appropriately skilled for the review but not so skilled that they are in active competition with the submitting authors and perhaps even steal their ideas! Ouch!
But, we should ask if there is a better way? Is peer review considered to be a significant factor when analysing the quality of an article? Is there an ideal way of getting peer review done? Perhaps peer review, in its various guises, is just the best system we currently have to ensure that a certain standard is maintained across the thousands of labs around the world. Or perhaps it’s just there so that we can feel more secure, rightly or wrongly, that what we read is a truthful and honest representation of how things really are. Either way, the messages are that we should keep questioning the appropriateness of these systems and, when reading papers, to not leave the critical parts of our brains on ‘stand-by’.
Luke Parkitny
Luke Parkitny is a PhD student at Neuroscience Research Australia. He is researching some of the factors that play a role in the development of complex regional pain syndrome (CRPS). Luke joins the Body in Mind team with a background of clinical practice and research in Western Australia. He has rapidly cultivated an interest in all things pain and has very successfully exploited every opportunity to share this knowledge with other health professionals and lay-persons.
Link to Luke’s published research and here he is in person talking about what he does.
Reference
Larson, B., & Chung, K. (2012). A systematic review of peer review for scientific manuscripts HAND DOI: 10.1007/s11552-012-9392-6