Double-blind peer review – list of journals

Subconscious discrimination against member of underrepresented groups, such as women, is incredibly common in educational and academic settings. Elementary school teachers grade math tests more harshly when they believe that the tests have been taken by girls. And, much later in life, a female researcher’s manuscript is more likely to be accepted for publication in a refereed journal if the reviewers do not know her gender. This occurs during “double-blind review,” in which an author does not know the names of reviewers, and reviewers do not know the names of authors.

Double-blind review allows reviewers to focus on the content of the research that they are evaluating, rather than the identity (institutional affiliation, demographic attributes, etc.) of the researcher who produced the work.

A few journals have started to use double-blind review lately, but the practice generally seems to be catching on rather slowly. And while it’s easy to find lists online of journals in a given field along with the journals’ publishers or impact factors, I’m not aware of any list of journals that includes the use of double-blind review.

So I am trying to compile a list of journals and funding sources in the life sciences that will include whether each journal/funding source always, never, or sometimes uses double-blind review. This information isn’t always readily available so I would greatly appreciate any tips on which journals use double-blind review – please send me an e-mail at schachat@stanford.edu to help. Thank you!

OA – Open Access

FP – For-profit publisher (Wiley, Elsevier, Springer, etc.)

DB – Double-blind

JOURNAL Mandatory OA FP DB review
Behavioral Ecology No No Yes
Conservation Biology No Yes Yes
Ecology and Society Yes, $975 No Yes
Entomological Research No Yes Yes
Nature No Kind of Optional
Palaeontologia Electronica Yes, $0 No Yes
PeerJ Yes, $695 Yes No
PLoS ONE Yes, $1,495 No No

Experiment.com does for scientific funding what beauty pageants have done for scholarships

tl;dr: Beauty pageants claim to be good for women because they award scholarship money. But there are many reasons to doubt this claim – in the end, beauty pageants are superficial and anti-intellectual, so they cause more harm than good by reinforcing oppressive ideas about a woman’s worth. A new website, Experiment.com, is purportedly a good thing for science – and women in science – because it provides small research grants. But, much like a beauty pageant, Experiment.com is based on superficial selection criteria and in the end it probably just reinforces biases that are bad for science (e.g., dinosaurs and other big amniotes are the only fossils that matter) and bad for women (there are countless examples of prejudice against women during review/evaluation, which point to the need for gender-blind grant review; by forcing researchers to insert their photo and biography into proposals – thus highlighting gender and other demographic attributes – Experiment.com is a step in the wrong direction).

During the 1940s, women in the United States attended college at far lower rates than men. There are many reasons why this may have been the case, ranging from priorities within individual families (parents were often willing to invest more in their sons’ education than in their daughters’ education) to prejudices at the national, if not global, level (due to societal biases against women, those in power were predisposed to judge female students more harshly). Given the multitude of reasons why women were denied educational opportunities, it may have seemed that a complex solution was required.

But maybe one part of the solution was actually quite simple. Maybe part of the solution was simply to give women more money to pursue an education, in the form of scholarships. Even better, the money could be distributed in a way that was somehow democratized – with a panel of caring adults voting on who should receive the scholarships.

In the mid-1940s, beauty pageants stepped in to fill this role. To this day, beauty pageants continue to sell themselves as being great for women’s education. Not only do they give scholarships to women – they preferentially give scholarships to those women who do the best job of explaining the importance of world peace!

So, are beauty pageants a good thing for women? Have they democratized our educational system? There are many reasons why this may not be the case:

  • The only women who can enter these “scholarship contests” are those who can afford to compete in the first place – and the initial costs of entering these competitions are quite high.
  • While a contestant’s ability to explain world peace (or evolution) is probably taken into account, contestants are primarily judged on more superficial criteria that have little to do with intellectual merit.
  • Beauty pageant judges may not be particularly qualified to decide who does and does not deserve scholarship money.
  • The relatively small amount of scholarship money disbursed by beauty pageants may not make up for the fact that beauty pageants reinforce cultural norms that are harmful for women – a woman’s value to society is determined first and foremost by her physical appearance, etc.

Now the scientific community has its own form of the beauty pageant: Experiment.com, a crowdfunding website where research funding has been “democratized” – more accurately, plutocratized. (It’s basically Kickstarter for science.) Researchers can create videos about the work they’d like to do, upload nice pictures of themselves on the project webpages, and then hop onto social media and self-promote like there’s no tomorrow.


Here are a few reasons why Experiment.com is bad for science in general:

The barriers to entry are very real. Yes, it is true that major funding agencies will require many weeks’ worth of work for a single grant application, and then dismiss that grant application after just a few minutes of review. And it is also true that many funding agencies are overly risk-averse, and reluctant to fund truly innovative research. However, these issues don’t really apply to small grants for young researchers (Experiment.com’s stock in trade). A number of the paleontology proposals on Experiment.com are from graduate students who need $1,000 or less for fairly run-of-the-mill studies. These students could get all of the funding that they need from Sigma Xi or any number of specialized scientific societies such as the Paleontological Society, assuming that their proposals are of merit. Small grants are relatively democratic: it costs zero money to type up a proposal, it shouldn’t take more than a few days to write the whole thing out, and grant-writing is a skill that all graduate students need to develop anyway.

At Experiment.com, however, the barriers to entry are quite a bit different. Many proposals include lots of photos of the researcher – if you want to do that science, you had better be good-looking and photogenic! Plenty of proposals also include videos about the proposed research. I am certainly not against scientists making videos about their work (as an undergraduate, I learned about digital animation in an art class and entered one of my videos into an environmental preservation film festival). However, video creation is not as essential skill for young researchers – I possess this skill myself, and have not used it in almost five years – and among the researchers who lack the time or experience to make their own videos, the only option is to pay somebody else to do it. And according to Experiment.com, the best videos are those recorded with “a multi HD-camera setup with lighting.” How many students have access to such a setup? In contrast, a student who wants to submit a proposal to Sigma Xi just needs access to a word processor, something that is available in pretty much every school library in the developed world.

And unlike Sigma Xi and other scientific societies, Experiment.com’s funding process gives a direct advantage to researchers who come from wealthy families. Experiment.com is like Kickstarter in that funding is all-or-nothing: If you set a target of $3,000 but only raise $2,896 by the deadline, you will receive nothing. Now imagine two proposals for $3,000 that are of equal merit, both of which receive $2,896 of pledged backing when it’s 5 minutes before the deadline. One proposal comes from a student whose parents are wealthy – the parents pledge $150 with 3 minutes to go, and so the wealthy student gets all $3,000. The other student’s parents are broke – this student never raises over $2,896 in pledged backing, and therefore gets nothing. In other words, Experiment.com lets wealthy students take their family’s money and multiply it. This is a very real phenomenon. I won’t name any names or provide links, but there are many Experiment.com researchers with uncommon last names whose backers just so happen to share the same last name.

Experiment.com claims to support research that other funding bodies wouldn’t normally consider. In the paleo section of the website this does not seem to be the case. Of the first 3 Experiment.com projects that my boyfriend looked at, two had odd requests ($50 of domain names and a $15 college ruled notebook), but I’m not too upset about the fact that these sorts of requests aren’t normally funded.

Experiment.com panders to – and reinforces – the public’s preexisting misconceptions about what scientists should be studying in the first place. In paleobiology, for example, tons of great work has been done on “charismatic megafauna” – dinosaurs, large mammals and birds, etc. But here’s the thing – charismatic megafauna, while important, do not comprise the vast majority of fossils that are worth studying. (A parallel from the beauty pageant example: there are plenty of exceptionally good-looking women who deserve scholarships, but exceptionally good-looking women do not comprise the vast majority of deserving female students.) A tremendous preference for great-looking women is ultimately damaging to the vast majority of women who deserve scholarships, because this preference reinforces unfair prejudices about a woman’s “worth.” And a tremendous preference for charismatic megafuana is ultimately damaging to the field of paleobiology, because this preference reinforces unfair prejudices that have prevented the public from appreciating the scientific value of fossil invertebrates, plants, microorganisms, etc. Just take a look at Experiment.com’s paleontology page and see if you can spot any biases in the projects on there. There are fossils other than charismatic megafauna on Experiment.com, but they are absurdly under-represented.

Experiment.com conflates self-promotion with scientific communication. By and large, young researchers really do need to learn how to communicate their science with the public. Experiment.com supposedly helps with this because researchers must successfully engage the public (or perhaps more realistically, their family) in order to even receive any funding. However, I looked at many of the paleontology proposals on Experiment.com and found that most of the researchers there have not done a very good job at all of communicating their research. In projects that were successfully funded, researchers jumped from generalities/platitudes (it’s important to understand the natural world!) to specifics (to which of the following very closely related lineages does fossil X belong?) with little or nothing in between. There’s plenty of jargon. Links between different aspects of a single projects are not well explained. And I didn’t notice any correlation between effective scientific communication and funding success on the site. I rarely was able to tell whether the proposed research was novel, or whether it would be of interest to others. Experiment.com clearly doesn’t teach young researchers to communicate science effectively.

“Science communication” used to mean teaching the public about science. But now, according to Experiment.com, “[u]sing your networks” to promote yourself and ask for money “is the communication part of ‘science communication’.” If only Oliver Sacks were still around to participate in the ascent of Experiment.com – he could have ditched all that book-writing and concentrated on asking for cash via Twitter!

The middleman is creeping in. When a researcher receives a large grant (say, $100,000) from a major funding source such as the National Science Foundation, the researcher’s institution will keep a large part of that money – “overhead.” The institution will use some of the overhead to keep the lights on and the water running, but may also use a large chunk of the overhead to pay unneeded administrators and cover other unnecessary costs. Experiment.com promotes itself by saying that “[b]ackers fund directly to the scientists, so there is no middleman or overhead involved (compared 50-60% when receiving a grant at a university).

This might lead a young researcher to believe that if they receive $800 through traditional means (such as a grant from Sigma Xi) they’ll only get to keep $400, but if they receive $800 from Experiment.com they’ll get to keep all $800. None of this is true.

First of all, universities don’t usually take overhead from small grants awarded to early-career researchers. During my M.S. I received $5,000 from NSF on two occasions – the money was funneled through my university both times (NSF gave the money to my university, and my university gave the money to me) and no overhead was ever taken out. I also received small grants/fellowships from Sigma Xi and from the Smithsonian, and again, nobody took anything for overhead.

Also, when overhead is taken out, the rate is often far below 50%. It’s easy to imagine Experiment.com’s motivation for making misleading statements about overhead rates, but still…is this organization sufficiently trustworthy that it deserves to play a role in the scientific community?

Lastly, while Experiment.com claims that “there is no middleman,” Experiment.com is the middleman – it takes an 8% cut from all funded projects. (The fee used to be 5%, so it’s growing – what else would you expect from a funding source that is backed by venture capitalists?) So a student who gets $800 from Sigma Xi gets to keep all $800 whereas a student who gets $800 from Experiment.com only keeps $736. Experiment.com is a step in the wrong direction for young researchers who want to keep all of the funding that they raise.


Now here’s why Experiment.com is especially bad for women and for members of other underrepresented groups that face discrimination in the sciences:

Experiment.com reinforces the role of gender (and other demographic factors) in grant review. There’s tons of evidence that women are judged more harshly than equally-competent men in academic/educational settings, from their days in elementary school to their time as professors and seemingly all life stages in between. Unsurprisingly, this problem also exists in the scientific review process. And so a number of organizations have instituted double-blind peer review. With double-blind peer review, if I submit a grant proposal to a funding agency or a manuscript to a journal, I will never know the names of the reviewers, and the reviewers will never know my name (and therefore will not be able to infer my gender, race, religion, institutional affiliation, etc.). We already know that double-blind peer review is great for women: an individual does not know whether a researcher is male or female is better able to focus on the merit of the research in question. Double-blind review is a bit more difficult for grants than it is for manuscripts, because most funding entities will evaluate a researcher’s qualifications to do the proposed work (by looking at a CV, for example), in addition to reading the research proposal. However, there are still ways around this – for example, grants can be reviewed in two rounds: a double-blind round in which proposals are evaluated, followed by a single-blind round in which CVs and letters of recommendation are evaluated.

It makes sense that new developments in funding and publishing should move science in the direction of double-blind peer review. But Experiment.com does the opposite – reminding potential funders of a researcher’s identity (gender, etc.) by requiring at least one photo of each researcher in every proposal, by including each researcher’s photo and institutional affiliation on the “discover” page, etc. Experiment.com floods potential funders with the sort of personal information that is so often used subconsciously – if not consciously – against members of underrepresented groups.

By constantly reminding the public of researchers’ identities, Experiment.com shifts the focus from research to demographics in a way that is not helpful to members of underrepresented groups, such as women.