ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Correspondence

Biased under-reporting of research reflects biased under-submission more than biased editorial rejection

[version 1; peer review: 3 approved]
PUBLISHED 02 Jan 2013
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

This article is included in the Research on Research, Policy & Culture gateway.

This article is included in the All trials matter collection.

Abstract

Stephen Senn challenges Ben Goldacre’s assertion in ‘Bad Pharma’ that biased editorial acceptance of reports with ‘positive’ findings is not a cause of biased under-reporting of research. We agree with Senn that biased editorial decisions may contribute to reporting bias, but Senn ignores the evidence that biased decisions by researchers to submit reports for possible publication are the main causes of the problem.

Keywords

peer review

Stephen Senn challenges Ben Goldacre’s assertion in ‘Bad Pharma1 that biased editorial acceptance of reports with ‘positive’ findings is not a cause of biased under-reporting of research, and concludes that "the prospects for disentangling cause and effect when it comes to publication bias are not great"2. Senn apparently overlooks the studies – including controlled experiments - which have investigated reporting biases. These are summarised in an article3 from which the following is an excerpt:

Who is responsible for biased reporting of clinical research?

Reporting bias can be due to researchers and sponsors failing to submit study findings for publication, or due to journal editors and others rejecting reports for publication. Numerous surveys of investigators have left little doubt that almost all failure to publish is due to the failure of investigators to submit reports for publication4,5, with only a small proportion of studies remaining unpublished because of rejection by journals6, although positive-outcome bias has been demonstrated among peer reviewers7. Qualitative studies of editorial discussion indicate that a study’s scientific rigour is the area of greatest concern8. Researchers report that the reason they do not write up and submit reports of their research for publication is usually because they are "not interested" in the results ("editorial rejection by journals" is only rarely given as a cause of failure to publish). Even those investigators who have initially published their results as (conference) abstracts are less likely to submit their findings for full publication unless the results are ‘significant’9.

Investigations of biased reporting of research began with surveys of journal articles, which revealed improbably high proportions of published studies showing statistically significant differences1014. Subsequent surveys of authors and peer reviewers showed that research that had yielded ‘negative’ results was less likely than other research to be submitted or recommended for publication1518. These findings have been reinforced by the results of experimental studies, which showed that studies with no reported statistically significant differences were less likely to be accepted for publication7,1921".

Senn’s use of the term ‘publication bias’ in his commentary suggests that he is restricting it to editorial bias whereas, as indicated above, the origins of reporting bias are largely due to researchers’ decisions not to submit, not editorial decisions not to accept. The analyses of observational data cited by Ben Goldacre in his book ‘Bad Pharma1 do not detect editorial bias, but neither do they support a confident conclusion that no editorial bias exists. However, we believe Goldacre is correct to castigate researchers and research sponsors as being more culpable than editors in betraying their responsibility to the patients who have participated in trials.

The controlled experiments suggest that it is the results of studies, not their quality, that predisposes them to editorial bias. Senn believes that any editorial bias that exists can be ‘very plausibly explained’ by preferential publication of ‘positive’ studies, and that it "seems plausible that higher quality studies are more likely to lead to a positive result". Unless he is using the word ‘positive’ to mean something other than ‘a beneficial effect’, however, Senn appears to be overlooking substantial evidence challenging the plausibility of his belief (see, for example, reference22). Given the estimated likelihood of new treatments proving superior to standard treatments23 it surprises us that, "as a statistician" Senn would find this evidence "unpalatable".

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 02 Jan 2013
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Chalmers I and Dickersin K. Biased under-reporting of research reflects biased under-submission more than biased editorial rejection [version 1; peer review: 3 approved]. F1000Research 2013, 2:1 (https://doi.org/10.12688/f1000research.2-1.v1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 02 Jan 2013
Views
28
Cite
Reviewer Report 09 Jan 2013
Steven Julious, Medical Statistics Group, School of Health and Related Research, University of Sheffield, Sheffield, UK 
Approved
VIEWS 28
I would just put one anecdotal observation and that is of second studies that replicate the findings of a study published in a journal. An editor may turn down the second study as 'nothing new' is being said although most ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Julious S. Reviewer Report For: Biased under-reporting of research reflects biased under-submission more than biased editorial rejection [version 1; peer review: 3 approved]. F1000Research 2013, 2:1 (https://doi.org/10.5256/f1000research.653.r651)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
16
Cite
Reviewer Report 08 Jan 2013
Luigi Naldi, Department of Dermatology, Centro Studi GISED, Bergamo, Italy 
Approved
VIEWS 16
I confirm that I have read this submission and believe that I have an ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Naldi L. Reviewer Report For: Biased under-reporting of research reflects biased under-submission more than biased editorial rejection [version 1; peer review: 3 approved]. F1000Research 2013, 2:1 (https://doi.org/10.5256/f1000research.653.r601)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
26
Cite
Reviewer Report 08 Jan 2013
Riekie de Vet, EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands 
Approved
VIEWS 26
The authors comment on a article by Stephen Senn who questions Ben Goldacre’s assertion in the book “Bad Pharma” that editorial process is not the main cause of publication bias. They present a large amount of evidence from the literature
... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
de Vet R. Reviewer Report For: Biased under-reporting of research reflects biased under-submission more than biased editorial rejection [version 1; peer review: 3 approved]. F1000Research 2013, 2:1 (https://doi.org/10.5256/f1000research.653.r600)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 02 Jan 2013
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.