July 21, 2014
Terms like ‘open peer review’ and ‘post publication peer review’ are more and more popular and at the same time quite misleading. The Internet provides an opportunity to make work available to a broad audience at very low cost, thus there is less reason to wait for formal reviews, leading to a thin line between published and unpublished work.
Peer review is the system of quality control in modern academic publishing. The idea behind it is simple: each paper or book has to be checked by one, two or more specialists working in the same field as the author(s) of the book or paper. Reviewer(s) verify the scientific value of the work and decide whether it should be published or not.
Considering anonymity issues, we may distinguish semi-blind peer review, double blind one and so-called open peer review. The first one is when author(s) do not know the identity of referee(s), but referee(s) know whose work they are evaluating. The second is when the editor sends work to referee(s) without any personal information about the author(s). In this case the referees’ identity is also withheld from the author(s). In the last situation both sides know the identity of one other. In this case, the name of reviewers or even the review itself might be published together with evaluated work.
There are not many formal criteria for peer review and usually, all parties believe that being a “peer” (i.e. performing the same everyday problem-solving tasks as an author) is sufficient to spot all errors, frauds or uninteresting material. In fact this process is slow and fallible. Referees are often not able to spend enough time searching for necessary literature and analyzing research output. It is also common that finding a real ‘peer’ is hard as some scientists only share their interests with coworkers.
Some authors like to stress that peer review is the heart of science and is totally inevitable. That might be true, but it is worth remembering that the peer review system is much younger than science itself and Nature, for instance, introduced this procedure in 1960 (source), which means that it was able to operate without it for whole century.
Reviewers usually prefer to keep anonymity, thus double blind or semi-blind system is the most popular (some data has shown that it is harder to find reviewers without guaranteeing anonymity), although the idea of open, written discussion is much older than peer review itself and probably is closer to “heart of science”. The reason why reviewers prefer to be anonymous is not that they do not like to openly discuss and criticize one other, it is rather that they do not want to decide openly whether the work of their colleagues will be published or not.
Nowadays scientific publishing is changing thanks to the Internet (which is a platitude!) and we have an opportunity to return to the idea of an open and boundless discussion. The singular decision whether work should be published or not is not so crucial since the difference between published and unpublished work is much more discretionary. I agree with Joseph Esposito that it is hard to say nowadays what it means that something is published or reviewed.
I am not talking only about preprint servers like ArXiv, although it is worth stressing that papers published there are sometimes widely discussed (thus reviewed) on Twitter and blogs and even gain citations in journals without formal peer review and formal publication. It is true that these papers offer mixed quality – some of them are brilliant and others are not. I personally have wasted some time reading weaker papers on scientometrics on ArXiv, but I have also found some valuable ones too (I often tweet it at @OpenScienceNews).
Meanwhile, there are more comprehensive and systematic ways of conducting open discussions. The discussion that starts naturally before the first sentence is even written, and traditionally goes on within a research group, but thanks to the Internet might become global. Good examples are Open Access, history books from De Gruyter: ‘Historyblogosphere‘, already published in German and available both on-line and in paperback, ‘Arbeit im nationalsozialismus‘ and ‘Geschichte lernen im digitalen Wandel‘ to be published this year. All three of them started as blogs but were designed to become books. Registered blog users (each account had to be verified by administration but in fact there was no formal criteria for registration – only personal data was necessary, so anyone could join really) had an opportunity to comment on each paragraph of the book in preparation.
Open discussion is much fairer for authors, more transparent and trustworthy and what might be surprising, sometimes even more reliable. The Internet is a mass medium and it can be a more efficient tool for finding real “peers” (I mean people with similar research interests and with at least the same level of knowledge as the author(s) who engage in review, that traditionally means the direct approach of one scientist to another.
Also, the already well-known model of the Atmospheric Chemistry and Physics journal is quite interesting. ACP calls it “public peer review”. In fact, ACP offers two services at once. Peer review and open discussion, which is running simultaneously to peer review and which can be joined by anyone. Each paper submitted to ACP is quickly evaluated by editors and if they decide to send it to peer review it can become available on the journal website in ‘Papers in open discussion section’. The open discussion can be joined by any registered user (and there is no formal criteria for registration). Two of its participants are referees chosen by editors, who conduct classical review, with publicly available results. At the end of the discussion, the paper can be published in one of the journal’s regular on-line issues – together with its discussion version and comments.
What is even more interesting is that the term ‘final peer-reviewed version of an article’ is slowly losing its meaning, since every paper can by improved at any moment following an on-line discussion. CrossMark gives an opportunity to easily track all changes and always find the most recent version of the article.
Will non-traditional forms of article review gain popularity? I think so, but it will take a long time. Anyway at this moment it is an interesting topic for every scholar, since it offers an interesting opportunity to discuss and evaluate one’s work.