Your guide to Open Access publishing and Open Science

Multi-Stage Open Peer Review (Pöschl, 2012)

Author: 4 Comments Share:

One of the supposed conflicts in academic publishing is ensuring quality reviewed research in an environment of rapid scientific exchange. Traditional peer review, for instance, is a prime example of scientific quality: it allows for the dissemination of knowledge to pass through a filter of peers that self-regulates the suitability of a paper for publication. Despite not being perfect, it is currently accepted as the best system we’ve got, but one major stumbling block is the speed of publication – for anyone who has submitted a paper, the peer review process can be excruciatingly slow (to the point of competing with glacial displacement).

With that in mind, a recent paper by Pöschl (2012) argues we can have the best of both worlds: open access can provide the speed of publication we need and also improve upon scientific communication, quality assurance, evaluation and maintain sufficiently high standards of peer review:

(1) Open Access is fully compatible with traditional peer review, and in addition it enables interactive and transparent forms of review and discussion open to all interested members of the scientific community and the public (open peer review).

(2) Open access gives reviewers more information to work with, i.e., it provides unlimited access to relevant publications across different scientific disciplines and communities (interdisciplinary scientific discussion and quality assurance).

(3) Open access facilitates the development and implementation of new metrics for the impact and quality of scientific publications (combination of citation, download/usage, commenting, and ranking by various groups of readers and users, respectively.

(4) Open access helps to overcome the obsolete monopoly/oligopoly structures of scientific publishing and statistical analysis of publication contents and citations/references, which are limiting the opportunities for innovation in scientific publishing and evaluation.

Central to Pöschl’s argument is the discussion of multi-stage open peer review:  a two-stage process of open access publishing combined with iterated steps of peer review and interactive public discussion (see figure 1).

Like any successful product, multi-stage open peer review combines what worked previously with new, innovative practices afforded by open access and digitisation. Pöschl makes a convincing case for why we should adopt this approach more widely across academia and the sciences. In particular, what makes it more successful than other forms of closed or open peer review is that it enables the following:

(1) Publication of discussion papers before full peer review and revision: free speech, rapid publication, and public accountability of authors for their original manuscript foster innovation and deter careless submissions.

(2) Integration of public peer review and interactive discussion prior to final publication: attract more comments than post-peer review commenting, enhance efficiency, and transparency of quality assurance, maximize information density of final papers.

(3) Optional anonymity for designated referees: enables critical comments and questions by referees who might be reluctant to risk appearing ignorant or disrespectful – especially when providing a voluntary community service in which they have little to gain for investing lots of effort and time.

(4) Archiving, public accessibility, and citability of every discussion paper and interactive comment: ensure documentation of controversial scientific innovations or flaws, public recognition of commentators’ contributions, and deterrence of careless submission.

For me at least, multi-stage open peer review sounds like a sensible move in the scientific community. Pöschl leaves plenty of room for manipulation of the various stages so that they are tailored for the demands and assurances of different disciplines. Still, he does outline the essential elements and scales of evaluation, as figure 3 from the paper illustrates:

If I had anything to add, then it would be my own pet interest: to try and reduce the positive results bias in science. Multi-stage peer review could feature in this, but I would also add another stage to the process. What that stage is will have to wait for one of my future posts. For now, I’ll leave you with Pöschl’s (lengthy) abstract:

The traditional forms of scientific publishing and peer review do not live up to all demands of efficient communication and quality assurance in today’s highly diverse and rapidly evolving world of science. They need to be advanced and complemented by interactive and transparent forms of review, publication, and discussion that are open to the scientific community and to the public. The advantages of open access, public peer review, and interactive discussion can be efficiently and flexibly combined with the strengths of traditional scientific peer review. Since 2001 the benefits and viability of this approach are clearly demonstrated by the highly successful interactive open access journal Atmospheric Chemistry and Physics (ACP, www.atmos-chem-phys.net) and a growing number of sister journals launched and operated by the European Geosciences Union (EGU, www.egu.eu) and the open access publisher Copernicus (www.copernicus.org). The interactive open access journals are practicing an integrative multi-stage process of publication and peer review combined with interactive public discussion, which effectively resolves the dilemma between rapid scientific exchange and thorough quality assurance. Key features and achievements of this approach are: top quality and impact, efficient self-regulation and low rejection rates, high attractivity and rapid growth, low costs, and financial sustainability. In fact, ACP and the EGU interactive open access sister journals are by most if not all standards more successful than comparable scientific journals with traditional or alternative forms of peer review (editorial statistics, publication statistics, citation statistics, economic costs, and sustainability). The high efficiency and predictive validity of multi-stage open peer review have been confirmed in a series of dedicated studies by evaluation experts from the social sciences, and the same or similar concepts have recently also been adopted in other disciplines, including the life sciences and economics. Multi-stage open peer review can be flexibly adjusted to the needs and peculiarities of different scientific communities. Due to the flexibility and compatibility with traditional structures of scientific publishing and peer review, the multi-stage open peer review concept enables efficient evolution in scientific communication and quality assurance. It has the potential for swift replacement of hidden peer review as the standard of scientific quality assurance, and it provides a basis for open evaluation in science.

Reference

Pöschl U (2012). Multi-Stage Open Peer Review: Scientific Evaluation Integrating the Strengths of Traditional Peer Review with the Virtues of Transparency and Self-Regulation. Frontiers in computational neuroscience, 6 PMID: 22783183

Previous Article

How To Get Impact Factor

Next Article

Open and Shut: #OpenScience #OpenData #OpenAccess around the web

You may also like

4 Comments

  1. In my opinion Open Access to paper depends on the scientific area. For instance, in Social Science it´s possible. However, in the prive sector where they devolop products to sell for market, it almost impossible to apply and open access system to science. The economic force will drive this situation.

    Thank you for the oportunity
    Sérgio Franklin
    S

    1. Cheers for the comment Sergio. I wouldn’t say the private sector is necessarily incompatible with Open Access. They obviously need to sell something, and part of the challenge facing publishers is how do they make a profit. Having said that, making the journal freely available doesn’t mean that they can’t charge for other services, or generate revenue in some other manner (e.g. PLoS One charges authors). It’s about adapting to the new publishing ecology: the best journals will be the ones that offer new and innovative services that complement their more traditional roles; thus making them more attractive to prospective authors.

      Still, I agree that there will be tensions, but it’s more likely to be during the transition to an information-economy in academic publishing. We’re already seeing signs of potential conflict (e.g. Green vs Gold OA). How the dust will settle is anyone’s guess.

  2. As food for thought I would like to also offer the reference of a paper discussing an entirely journal-independent and author-guided model of open and transparent peer review. In fact, we have taken one step further and this October we plan to launch a free, multidisciplinary research platform that will be the first to test this innovative peer review model. Check it out here: http://libreapp.org and read more about or join our open, not-for-profit organisation here: http://www.openscholar.org.uk

    Natural Selection of Academic Papers. Perakakis, P., Taylor, M., Mazza, M., Trachana, V. (2010). Scientometrics 85. url: http://www.academia.edu/1222917/Natural_selection_of_academic_papers

Leave a Reply