Bullied Into Bad Science

Leading individuals and institutions in adopting open practices to improve research rigour

The letter

The Bullied Into Bad Science campaign is an initiative by early career researchers (ECRs) for early career researchers who aim for a fairer, more open and ethical research and publication environment.

We are postdocs and a reader in the humanities and sciences at the University of Cambridge. We are concerned about the desperate need for publishing reform to increase transparency, reproducibility, timeliness, and academic rigour of the production and dissemination of scholarly outputs (see Young et al. 2016, Smaldino & McElreath 2016).

We have identified actions that institutions and managers can take to better support ECRs (below). These actions are crucial for our success because we are eager to publish openly and at places that keep profits inside academia in accordance with many modern online publication venues (Logan 2017). However, ECRs are often pressured into publishing against their ethics through threats that we would not get a job/grant unless we publish in particular journals (Carter et al. 2014, Who is going to make change happen?, Kent 2016; usually these journals are older and more familiar, have a print version, a high impact factor, and are not 100% open access). These out of date practices and ideas hinder ECRs rather than help us: evidence shows that publishing open access results in increased citations, media attention, and job/funding opportunities (McKiernan et al. 2016). Open dissemination of all research outputs is also a fundamental principle on which ECRs rely to fight the ongoing reproducibility crisis in science and thus improve the quality of their research.

To support ECRs in this changing publishing landscape, we encourage funders, universities, departments, and politicians to take the following actions (below) and to announce these actions in public statements. We consider these actions essential for enabling ECRs to do and disseminate our research as we intend it, in an open, modern, and rigorous way. We feel that failure to adequately support ECRs, which are a vulnerable group, will prevent us from delivering outstanding academic outputs and becoming the academic leaders of the future, and thus decrease our nation’s reputation for world-leading research.

If you, too, have felt pressured into taking professional actions that are against your ethics, please mark which actions you agree with and join our effort to change academic culture. We will send letters that include the number of ECRs who signed each action (and their names and affiliations, plus some anonymised anecdotes about ECR experiences) to relevant institutions, focusing on UK politicians, universities, and funders, and to the press to generate publicity. Our aim is to instigate institutions into taking actions that are relevant to us to improve academic culture for ECRs. You can stay updated with the progress of this effort and view the letters with the actions and signatories at www.CorinaLogan.com and www.BulliedIntoBadScience.org. The actions and their signatories will be available for reference by others who wish to create change in academic culture beyond the UK.

ECRs: sign the letter

We urge institutions and individuals to better support ECRs by taking these actions:

  1. Sign the Declaration on Research Assessment (DORA) as an independent university/institution/individual (following the example of Imperial and UCL), indicating that you will: a) “Be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published; and b) For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice” (http://www.ascb.org/dora/). These concerns about access to data should not be narrowly construed as concerning the ‘hard sciences’. The increasing reliance on the analysis of digital information in the humanities and the arts means that such issues matter across academia. As digital humanities become more widespread, access to open data and publications become more crucial. ECRs will benefit more than other career stages from DORA being signed because there is more pressure to publish in journals with high impact factors earlier in their career.
  2. Positively value a commitment to open research and publishing practices that keep profits inside academia when considering candidates for positions and promotions (in alignment with DORA).
  3. Endorse immediate open publishing, favouring publications in journals that are 100% open access (see https://doaj.org for a list of high-quality 100% open access journals), which are more beneficial to universities and researchers because:
  1. Endorse posting of preprints in recognised preprint servers to avoid publishing delays that are detrimental to ECR career progress (http://asapbio.org; recognising that some disciplines will be restricted from doing so). This is particularly timely as funders, such as the MRC and the Wellcome Trust, now accept preprints in grant proposals (http://asapbio.org/funder-policies). This action helps institutions implement the UK Concordat on Research Integrity.
  2. Endorse, support and promote the open publication of data and other scientific outputs such as software, which is inherent to modern scientific practice and results in an increase in citations for each open product (McKiernan et al. 2016). This action helps institutions implement the UK Concordat on Open Research Data.
  3. Educate researchers about publishing practices via public statements, mandatory courses, and inductions that cover: open research/data/access, mandates, hidden costs of traditional publishing (e.g., publishing delays, page charges and color figure charges in addition to article processing charges), and how to protect ECRs against exploitative publishing practices (e.g., avoiding hybrid open access protects ECRs from being overcharged [Shamash 2016, Pinfield et al. 2015, Solomon & Björk 2016, Kingsley 2016]). This action helps institutions implement the UK Concordat to Support the Career Development of Researchers and the UK Concordat on Research Integrity.
  4. Increase transparency by reporting to the public how much institutions pay for research to be published (i.e., journal subscriptions and article processing charges) to raise awareness about the significant drain on public funding. This will help researchers decide where to invest their future publishing efforts, especially because research funding is decreasing. This action helps institutions implement the UK Concordat on Research Integrity.
  5. Make all postdocs voting members of their institutions. This will ensure that institutions increase diversity and stay connected to the changing needs of this underrepresented group that tends to be more connected to modern publishing practices. Increasing the diversity of career stages in decision making processes will result in higher group performance, particularly where innovation is concerned (Roberge & van Dick 2010). This advances two Athena SWAN Charter points (five and nine; because, for example, most female researchers at the University of Cambridge are ECRs [University of Cambridge Databook 2016, p.26]) by removing obstacles faced by women at major points of career progression and by mainstreaming structural and cultural changes that advance gender equality. This will improve working conditions for researchers, which impacts research quality, and will help universities obtain or retain EC HR Excellence in Research awards. This action helps institutions implement the UK Concordat to Support the Career Development of Researchers.

You are an ECR (PhD student, Post-doc [including researchers in industry and those that are unemployed], Lecturer, Reader) and want to join the campaign: sign the letter

Note: although ‘science’ is in our name, we actually mean ‘research’, and researchers from all fields are welcome. The only reason we put science in our name is because it sounded catchier.

Survey results

As of Sun Dec 02 2018, we have 158 signatories, and the response rate for our 8 actions above is shown in the table below:

Action 1 Action 2 Action 3 Action 4 Action 5 Action 6 Action 7 Action 8
Number 156 157 157 157 158 157 158 157
Percentage 99 99 99 99 100 99 100 99

The signatories are:

We have collected anectodes from the ECRs who filled out the survey - click on Details below.

Was told that only certain journals count at previous institution and pressured towards working on papers for those journals. / Was also dissuaded from applying for certain individual fellowships as they did not provide enough overhead costs.

Asked to reproduce a former post-doc’s results in order to get the paper into Science. When myself, and two other senior post-docs couldn’t reproduce said results. I was blamed and the paper was submitted to a lower impact journal. / The post-doc in question had left the group, and it was well known how difficult it was to reproduce this person’s results for other projects. Those results eventually ended up published in Nature. Nobody said anything (obviously).

There is a lot of tacit pressure in where to submit work, that many just accept or even do not know exists (e.g., submitting to closed journals when open alternatives exist).

There has not been any direct pressure, but it feel like people in my field (atmospheric science) are very reluctant to publish in fully open access journals (which is a bit of a contradiction for me - as the research we are conducting are meant to bring useful information to policy makers to make decision about climate change, air pollution, urban planning…) and I feel that people undervalue the research being published in OA journals although they went through rigorous peer-review.

I have been hearing of publication count since I started my PhD. We often talked about the publication numbers rather than quality. As a scientist, I need to solve practical problems or provide some theoretical answers. Now if I have made one publication that is groundbreaking, and ten not so good publication. The count will suggest 10 is better than 1. But in reality, the one was more impactful. I believe the University and policymakers should look beyond only numbers. Besides, I don’t think the recruitment process of ECR (e.g., PhD) have been served us well. Again, we minimised the criteria to grades and publication. We missed dedicated researchers in the process. I signed up because I believe the process should be more transparent and justified. Science is more than grades and number of publications.

Pressured not to publish pre-prints

I was told that if it wasn’t Nature or Science it wasn’t worth publishing. I was actively prevented from publishing valid, good science in lower impact journals and putting all of my eggs in the one Nature basket. I feel completely disillusioned about academia, and publishing, and I’m questioning putting 11 years of my life into this. When I mentioned open source opens I was laughed at and ridiculed.

We are often told that papers published in open-access, low impact factor journals “do not count” for our career assessments, even if they have demonstrable impact. In fact, many official evaluation criteria explicitly exclude publications in journals not included in SCI, or value candidates based directly on the Impact Factor of the journals where they have published. Same with sharing preprints, datasets and code: not valued, even opposed to, because losing data ‘ownership’ and publication credits. Finally, being encouraged to p-hack or hide weaknesses in order to get published in high-IF journals. / / For too many scientists, the goal is to get papers published, and in some particular journals, not being correct or really advance science.

Too many to list, including several other even more egregious coercions into conducting completely corrupt and flawed science. All such experiences will be exposed in my upcoming non-fiction narrative book called “Dangerous Deceptions: The Social Psychology of Bad Science” (see etiennelebel.com and corresponding blog for updates)

My supervisor told me that he is looking out for our career development, and that accordingly we need to publish in high impact journals as this will raise my profile among his peers. Having repeated papers in high impact journals would be crucial during applications for jobs and grants.

I was forced to publish in a higher impact journal than the one which I originally wanted. Both were OA, so that was okay, but the price then changed from free to $6000.

I was never pressured to publish in particular journals but I was asked by senior colleagues to keep track of “impact factor” of the journals I am interested in publishing just to make sure I am not sabotaging the reputation or my own career opportunities. I was even intimidated by being perceived less serious and credible the moment I advocate for open practices.

Supervisors would subtly ‘encourage’ me to go for the highest impact factor journal, regardless of other things (such as theme, readership,etc).

I have founded 1 and co-founded a second OA journal at University of Edinburgh, and work with strong editorial teams, we have a strong peer-review culture. And yet, only ‘activist’ scholars from within Edinburgh departments will publish with us. There is a fear, I feel, of diluting their ‘brand’ all-the-while acknowledging that the for-profit publishing culture is a huge problem. The incentives, I feel, need to originate in the research funding bodies and at the institutional level. There is too much fear that OA will impact careers.

My account can be found here: https://socialbat.org/2015/08/12/goals-of-science-vs-goals-of-scientists-a-love-letter-for-plos-one/

In my case pressures were not high, but there was a consistent difficulty in convincing older colleagues and supervisors that doing Open Science (e.g. publishing preprints, sharing the data, publishing on OA journals) actually improves science. They simply refuse and are obnubilated by indexes and metrics, without realizing that these are choking the core of research and science.

I was promised a promotion if I managed to publish (again) in a high profile journal

During my transition from one postdoc to another, I wrote a perspective paper with two other colleagues that were unemployed. When our paper was accepted, we were asked to pay USD1,000+ for publication fees, and none of us has fundings at the time to cover this cost. Eventually, our new and old employer agreed to pay with the agreement that we included them in the acknowledgement. If our old or new employers refused to pay, it would be impossible to publish when we were either transitioning or unemployed.

My supervisor has repeatedly blocked finished work from being submitted for publication so that it may be supplemented with more work to make it suitable for higher-impact journals. Papers that were ready for submission have been delayed, some for several years, on the basis that an unlikely result from an extremely difficult experiment may improve the impact of the research. The supervisor of course does not suffer the consequences of this delay, while myself and PDRA colleagues miss out on an entry in our CV that would greatly help with fellowship and job applications.

My experience is written in my OpenCon2017 submission: https://goo.gl/15ZiDT

its a somehow unwritten law at the University of Vienna that you wont get a permanent job without a Nature or Science paper

When deciding on and proposing a journal to send a paper to, when suggesting an open access journal, often even without impact factor yet, like PeerJ or Royal Society Open Science, or worst when suggesting to publish a preprint on a recognized server, an answer I often got from the PI was “it’s not good for your stats” “I don’t need more papers but you do so choose your journal wisely”… it’s all very disheartening and demeaning. I played the game several times. I don’t anymore. I send my papers where I think they fit, and where I think the policies of the journal fit my ethics.

I have been told explicitly that I would never be hired if I published in open-access journals – though there are almost none in my field in any case.

The institute uses impact factor as one of the performance metrics of research groups and in turn uses it to determine the quality of funding. This places pressure on the group leader and therefore his/her group to publish in these co-called “high impact” journals. We also experience that these said journals publish articles from a select group of individuals and suppress early career researchers despite the latter group producing high quality research.

Achieving a suitable REF return according to my institution standards means aiming for journals that are not the appropriate target expected within the discipline, and rarely (if ever) open-access. Assessments of professional performance should not be based so narrowly on the high impact factor publications, and should allow researchers (the experts in the field) to decide where their work should be submitted.

I entered academia because I saw my mother, a law professor as a rolemodel. When I signed up in 2006, essentially all PhDs would get tenure and move on to be professors, provided they made the research. I remember when my mother prepared five boxes containing all her life’s work, my father painstakingly taking printouts on dot matrix printers. These efforts were made because the jury who would evaluate my mom would actually read and evaluate the material in the documents. They had the potential to understand the contents. Then some guy I won’t mention by name made a decree so powerful, it must have made the Pope jaleous: “Science Citation Index is the currency of science”. Of course, he became a very rich man. Fast forward to 2017, and my state-funded university expects me to publish 3 SCI or SCIE articles, in a journal which is listed by a private company. I observe professors, who haven’t published a single SCIE paper gather up and say “Youngsters with just 3 SCI publications apply for Associate Professorship. We must increase quality.” and draft all these new requirements, which, for some reason apply only to young researchers. The list I mentioned contain journals that pay for a fee-but they are peer reviewed, if you can believe it-those that only publish research from their own country(universality of science anyone?) and those with leisurely pictures and articles that wouldn’t make it to your local computer magazine. As I write these lines, I am glancing lustily at the Waves book from Prof. Feynman’s lecture series. I cannot start to read it. Instead, I must publish in whatever topic is hot/popular right now. Drones, iot, data mining, social media, whatever will get me an article. Then I must pay hefty fees for publication. I did not sign up for this. I respect secretaries, but I did not sign up to become a secretary. I want to learn and make real research that may come to fruitition in may be 10 years from now. But I can’t. Now I should stop writing and go back to publishing. I’m in a Kafkaesque world.

2/5 of my NERC Independent Research Fellowship reviews discriminated against me for publishing in open access journals

Often the decision on where to publish is done by the PI without consultation with ECR - I believe this practice need to be considered critically too

I have been constantly harassed by superiors to modify data in order to provide better agreement with known experimental values in order to make the paper look better for publishing at prestigious journals.

It’s the reason why I left to focus on open science advocacy and development!

I have so many journal ancedotes I do not know where to begin. My overwelming impression is that editors read the author name and institute and do not get as far as the title before deciding what to do with a paper. I have published 53, but the sort of comments that come back give me no confidence in the system.

If you want to go into tenure track, you NEED to publish in Nature or Cell. / This is what I was told.

I don’t have experience yet, but I have been told how important it is to get at least some papers published in high ranking journals to get a post-doc.

Non-ECRs: support the campaign

You are an established researcher (Professor) or in a position connected with academia and want to show your support: fill out this form.

Press coverage

Contacts

  • Dr Corina Logan, Senior Researcher, Max Planck Institute for Evolutionary Anthropology, Germany.
  • Dr Laurent Gatto, Associate Professor, de Duve Institute, UCLouvain, Belgium.
  • Dr Ross Mounce, Open Access Grants Manager, Arcadia Fund
  • Dr Adrian Currie, Lecturer, Sociology, Philosophy and Anthropology, University of Exeter.
  • Dr Stephen Eglen, Reader, Department of Applied Mathmatics and Theoretical Physics, University of Cambridge, UK.
  • Dr Lauren Maggio, Associate Professor, Department of Medicine, Uniformed Services University of the Health Sciences, USA.

Additional actions

These actions are in addition to the 8 we initially outlined in the above petition. Since we can’t add actions to the petition after so many people have already signed on to these particular 8 actions, we are making these important additions here.

  • Stop pressuring ECRs into conducting/writing up underpowered, non-preregistered, p-hacked, HARKed studies. In other words: stop teaching/advising/pressuring people to mutilate data into a “publishable” form when that distances it from actual science. (Added by Anne Scheel)

How we are implementing change