Researchers frequently make inappropriate requests to statisticians: Removing/altering data to support the hypothesis; interpreting the findings on the basis of expectation, not results; not reporting the presence of key missing data; & ignoring violations of assumptions

Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians. Min Qi Wang, Alice F. Yan, Ralph V. Katz. Annals of Internal Medicine, http://annals.org/aim/article-abstract/2706170/researcher-requests-inappropriate-analysis-reporting-u-s-survey-consulting-biostatisticians

Abstract

Background: Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers.

Objective: To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations.

Design: Online survey.

Setting: United States.

Participants: A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians.

Measurements: The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting.

Results: Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: a completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as “most severe” by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians.

Limitations: The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey.

Conclusion: This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study.