Priorities in academic publishing: quality vs quantity

A recent Twitter exchange got me thinking about academic publishing again. It seems to me that much of the current debate about peer review, publication bias and open access boils down to a conflict between quantity and quality, and I have a favourite: quantity.

Quality (the problem)

This is why we have peer review, to ensure that only the good stuff gets published. Clearly this doesn’t work, but I do still feel there is a place for peer review; not in the selection of the ‘best’ papers, but in the filtering-out of erroneous work. In the UK, the REF, and by extension universities, encourage quality over quantity. I have often heard school heads and research group leads trumpet the need for fewer, higher-quality papers. No wonder, if that’s what brings in the money.

Quality is important, for sure (even if our ways of defining quality are weak). However, in my opinion, these incentives are totally unnecessary for ensuring quality. The reason an economist might give half their right thumb to publish in American Economic Review over any other journal is not simply because of quantifiable career benefits and employability. No doubt the prestige gained (or the envy induced) is a sufficient incentive.

Quantity (the solution)

Isn’t this what current campaigns are striving for? We want to reduce publication bias through the publication of uninteresting or negative results. We want datasets and detailed methodologies made available. Yet academics are encouraged not to waste their time on these things and instead strive for that publication in AER/Science/Nature/NEJM. We want academics to stop prioritising prestigious journals with unscalable paywalls, yet this is exactly what they are currently incentivised to do.

Incentives for quantity should be appended to my previous suggested solution to the problems of academic publishing. The REF should reward quantity instead of quality, for example. Some research suggests that academics face a quality/quantity trade-off, while others suggest that the two may go hand-in-hand; no doubt this depends on the field of research. Nevertheless, a re-alignment of incentives towards quantity and away from (self-sustaining, immeasurable) quality would surely be better for academia as a whole.

doi: 10.6084/m9.figshare.1138635

Advertisements

One suggestion for a new model of academic publishing

Problems exist with various aspects of academic publishing: open access, peer review, publication bias. I’m not going to go through all the issues here, as they’ve been covered eloquently elsewhere. I suggest reading The Scholarly Kitchen. Anyway, here’s me weighing in with a possible new model that could solve most of the problems with academic publishing*.

A simple regulatory framework

Regulation is costly to maintain and enforce, so it’s often best to try and keep it to a minimum. Nevertheless, academic publishing is big business, so pussyfooting around won’t achieve much; strong policies need to be drawn up. In my proposed model, research resulting from government funding (and government funding resulting from research) should be subject to two conditions:

  1. All outputs must be published with open access and
  2. <peer review of all publications must be handled by an independent third party.

These two requirements would be enforced by bodies that fund research, such as the NIHR and Research Councils UK, or the NIH in the US. They would also be necessary criteria for inclusion in REF (and its international equivalents). We’re close to achieving the first criteria, but I’m not aware of any discussion of the second.

The ideal outcome

I think this environment could result in desirable outcomes. The new pressure upon academics from universities and funding bodies to conform to the two conditions above would lend to a system in which academics cease to submit articles to journals. Instead, journals would ‘bid’ for papers. They would compete in various ways; on quality, distribution and price. The price would be in the form of the publication fees charged; The American Economic Review could demand a higher fee than The Journal of Health Economics. New journals looking to build credibility could even pay the academics for the right to publish their paper. Universities would pay publication fees on behalf of academics. It’s also likely that they would fund the peer review process through subscription fees to the companies providing it. A lot of time and money would be saved by enabling the transferability of reviews and removing the need for academics to make multiple submissions. I also suspect that, not having to go through this rigmarole, academics would be more inclined to publish negative results.

In this world, the process of publication would go as follows:

  1. Academic writes paper;
  2. Academic submits paper to peer review company;
  3. Peer review company obtains reviews from numerous academics, including reviews of reviews;
  4. Journal editors are given access to the manuscript and all reviews;
  5. Journal editors ‘bid’ for the right to publish the article;
  6. Academic selects journal for publication;
  7. Paper is published.

The necessary organisations for this model already exist. Creative commons provides a variety of open access licences. Companies like OAK have popped up to deal with the administration of publication fees. Peerage of science is a company providing portable peer review, with growing interest from open access publishers. As the market expands, other companies would inevitably enter the arena.

Maybe the ‘simple regulatory framework’ set out above wouldn’t result in the ‘ideal outcome’ described… but I reckon it might.

*this model may very well have been proposed elsewhere, but I am unaware of it.

doi: 10.6084/m9.figshare.1101321