Questionable research practices

What are Questionable Research Practices?

A range of activities that intentionally or unintentionally distort data in favour of a researcher’s own hypotheses - or omissions in reporting such practices - including; selective inclusion of data, hypothesising after the results are known (HARKing), and p-hacking. (Fortt 2021)

Questionable research practices (QRPs) are one factor which can lead to low degrees of reproducibility in scientific research. Caution should be taken when evaluating QRPs, which range in seriousness, and can be highly context specific.

QRPs range from the occasionally problematic, such as HARKing, to practices such as the “creative” use of outliers or p-hacking which can have a more serious impact on the validity of a study’s conclusions.

Increased transparency

Avoiding the use QRPs can be encouraged through increased research transparency. For example preregistration clearly shows readers when an analysis has been performed post-hoc. Open data reveals where data has been manipulated to produce a particular result. Reforming scholarly publishing with interventions such as registered reports are designed to disincentivise the use of QRPs, removing pressure for researchers to produce positive results, often seen as a requirement to publish in prestigious journals.

Examples of Questionable research practices

• P-hacking: Performing multiple statistical analyses in search of a “significant” P value
• Salami slicing: Manipulating data from a single study to generate multiple publications
• HARKing: “Hypothesising after the results are known” is the presentation exploratory analyses as confirmatory analyses
• Selective reporting: While non-publication of negative results is currently unavoidable, the resultant “publication bias” leads to an incomplete scientific literature.