Peer-reviewing peer-review: Publons edition

On this blog and elsewhere I have discussed some of the problems with academic publishing; whether it be determining what actually counts as an academic publication or thinking about an alternative model. I will be continuing to prod at our decaying system of scholarly publishing in my new role as a Publons Advisor. As my first act I would like to advise all of you academic types to join Publons!

What is it?

Publons is a website where you can record and share your peer-review activity. Each time you review a paper – whether it be pre-publication or post-publication – you should tell Publons about it. Publons also provides a place for you, as an expert, to discuss published literature. If you want more detail about Publons and the ideas behind it, you can read a paper published by the founders.

Why do we need it?

Peer-review is in crisis. There is no evidence that – in its current form – it actually does any good. As such we need better approaches to the assessment of the quality of scholarly output. Peer-review still has an important role to play; it’s vital that experts in any given field assess the quality of reports on research findings. Furthermore, the quality of these assessments should also be assessed. Yet in the vast majority of cases, peer-review continues to be a secretive affair. Rarely do we know how good the peer-review process actually was. Publons can solve this problem by linking a paper with its reviews, and ideally these reviews will be free for anybody to evaluate.

Why should I sign up?

Academics spend a lot of their time contributing to the public good; doing things for which they receive little or no personal benefit. Peer-review is the prime example of such unrecognised labour. Publons makes it possible to get credit for your peer-review activity. Your Publons profile provides proof of your activity as a reviewer and even the quality of the reviews you share. In the age of metrics, Publons gives you a score indicative of the quantity and quality of your review work. You can take a look at my stats page to get an idea of the kind of supporting information that Publons can provide.

I hope you’re convinced. Adding a review to Publons takes less time than it took you to read this blog post. You can get the credit you deserve and improve scientific discourse at the same time. Visit the Publons website to sign-up, and if you have any questions please leave me a comment below.

Priorities in academic publishing: quality vs quantity

A recent Twitter exchange got me thinking about academic publishing again. It seems to me that much of the current debate about peer review, publication bias and open access boils down to a conflict between quantity and quality, and I have a favourite: quantity.

Quality (the problem)

This is why we have peer review, to ensure that only the good stuff gets published. Clearly this doesn’t work, but I do still feel there is a place for peer review; not in the selection of the ‘best’ papers, but in the filtering-out of erroneous work. In the UK, the REF, and by extension universities, encourage quality over quantity. I have often heard school heads and research group leads trumpet the need for fewer, higher-quality papers. No wonder, if that’s what brings in the money.

Quality is important, for sure (even if our ways of defining quality are weak). However, in my opinion, these incentives are totally unnecessary for ensuring quality. The reason an economist might give half their right thumb to publish in American Economic Review over any other journal is not simply because of quantifiable career benefits and employability. No doubt the prestige gained (or the envy induced) is a sufficient incentive.

Quantity (the solution)

Isn’t this what current campaigns are striving for? We want to reduce publication bias through the publication of uninteresting or negative results. We want datasets and detailed methodologies made available. Yet academics are encouraged not to waste their time on these things and instead strive for that publication in AER/Science/Nature/NEJM. We want academics to stop prioritising prestigious journals with unscalable paywalls, yet this is exactly what they are currently incentivised to do.

Incentives for quantity should be appended to my previous suggested solution to the problems of academic publishing. The REF should reward quantity instead of quality, for example. Some research suggests that academics face a quality/quantity trade-off, while others suggest that the two may go hand-in-hand; no doubt this depends on the field of research. Nevertheless, a re-alignment of incentives towards quantity and away from (self-sustaining, immeasurable) quality would surely be better for academia as a whole.

doi: 10.6084/m9.figshare.1138635

Reddit for academics

I’ve finally figured out reddit, and it’s a great tool for academics. You should really give it a try.

Over the past few years I’ve dabbled in reddit, failing to really ‘get it’. I thought it was just a place for silly GIFs and celebrity AMAs (Ask Me Anything), but I was wrong. It’s a place to share links to interesting internet stuff, but more importantly it’s a place where these links can be discussed. So if like me you’ve tried it before and failed to grasp it, or if you fancy giving it a go, here’s my suggested route in.

  1. Read this.
  2. Sign-up. Choose a username. You can remain anonymous if you wish.
  3. Go to reddit.com/subreddits and unsubscribe from any stupid subreddits to which you’ve been automatically subscribed.
  4. Go and find some subreddits to join. For academics I suggest academicpublishing, DepthHub and Scholar. And if you’re a fellow health economist, try academiceconomics, Economics, healthcare and HealthEconomics.
  5. Have a look at your homepage. By this stage it should be full of articles that interest you. Follow the links. Vote-up the ones you like. Join the discussion by commenting on them.
  6. Take a look at your preferences. They can alter the experience somewhat.
  7. Head over to metareddit.com to find more subreddits you might like.
  8. If you use Chrome, download the reddit companion.
  9. Start submitting your own links and comments to subreddits and watch the discussion unfold. You can submit links to academic papers, blogs, silly pictures… whatever you like!

A word of warning. DO NOT use reddit exclusively as a means of self-promotion. You will be Shadow Banned, as I have been. Stick to the rule of thumb of no more than 1 in 10 of your link submissions being some way self-promotional. The number of links you can submit is, in some secretive way, defined by your ‘karma’. You get karma by posting links and comments that other people like. Just be helpful and nice and you won’t fall foul of the reddiquette police.