Questionable research practices in linguistics: where do we fish, HARK and p-hack?

A graphical zine
Authors

Tetiana Hnatiak & Merve Begüm Ceylan

The aim of this zine is to explain as simply as possible Questionable Research Practices, such as ‘fishing’, ‘HARKing’, and ‘p-hacking’ in the context of linguistic research.

Public View Link

You can view the zine following The Canva Link for free public viewing or download it directly as a .pdf file here.

The contents of the zine were designed with BA, MA, PhD linguistics students in mind, just like they were designed by ones (Tetiana Hnatiak & Merve Begüm Ceylan, as printed on the cover of the zine).

Zine PDF

Download PDF file.

QRPs in Linguistics: Zine created by Hnatiak, T., & Ceylan, M. B. on canva with licence-free items and AI generated items on Perplexity (see the licensing section)

We created this zine within the scope of the Open Science seminar (Summer Semester 2025, University of Cologne) taught by Dr. Elen Le Foll with Open Science values (more about Open Science values) and the aim of spreading the knowledge among our target audience or anyone whom it may concern.

Bibliography description

Throughout the research process we cited the following works:

Bishop (2019) - as the basis and the starting point of our research, the paper from which we started contemplating QRPs in the context of linguistics.

Andrade (2021) helped us research further and cover the definition of ‘fishing expeditions’, its theory specifically, and a hands-on example in the context of research.

With Kerr (1998) and Isbell et al. (2022) we learnt more about HARKing.

Simmons, Nelson, and Simonsohn (2011) - “an elegant, comic paper in which the authors crafted analyses to prove that listening to the Beatles could make undergraduates younger” to cite Dorothy Bishop (Bishop 2019).

Yamada (2018)‘s article helped us cover the ’Redemption Arc’ page of our zine, explaining how to be a better researcher.

These distinctions — CHARKing, RHARKing, SHARKing, and THARKing — are outlined in Hollenbeck and Wright (2016).

Image Credits and Licensing

All visuals used in this project were created or selected from the following sources:

  • Canva Stickers: licence-free elements available under Canva’s Free Plan, which permits educational and non-commercial use without attribution.
  • AI-Generated Images: all AI-generated visuals were created using Perplexity for the purposes of this project, using platforms that allow academic and non-commercial reuse.
  • Screenshot from Bishop (2019): the screenshot included from Dorothy Bishop’s article “Rein in the four horsemen of irreproducibility” (Nature, 568, 435) is used under the principle of fair use for educational and non-commercial purposes. The source is fully cited and no modifications were made beyond highlighting and cropping for illustration and critique.

For attribution, please cite this work as:

Hnatiak, T., Ceylan, M. B. 2025. “Questionable Research Practices in Linguistics: Where Do We Fish, HARK and p-Hack?” July 10, 2025.

References

Andrade, Chittaranjan. 2021. “HARKing, Cherry-Picking, p-Hacking, Fishing Expeditions, and Data Dredging and Mining as Questionable Research Practices.” The Journal of Clinical Psychiatry 82 (1). https://doi.org/10.4088/jcp.20f13804.
Bishop, Dorothy. 2019. “Rein in the Four Horsemen of Irreproducibility.” Nature 568 (7753): 435–35. https://doi.org/10.1038/d41586-019-01307-2.
Hollenbeck, John R., and Patrick M. Wright. 2016. “Harking, Sharking, and Tharking: Making the Case for Post Hoc Analysis of Scientific Data.” Journal of Management 43 (1): 5–18. https://doi.org/10.1177/0149206316679487.
Isbell, Daniel R., Dan Brown, Meishan Chen, Deirdre J. Derrick, Romy Ghanem, María Nelly Gutiérrez Arvizu, Erin Schnur, Meixiu Zhang, and Luke Plonsky. 2022. “Misconduct and Questionable Research Practices: The Ethics of Quantitative Data Handling and Reporting in Applied Linguistics.” The Modern Language Journal 106 (1): 172–95. https://doi.org/10.1111/modl.12760.
Kerr, Norbert L. 1998. “HARKing: Hypothesizing After the Results Are Known.” Personality and Social Psychology Review 2 (3): 196–217. https://doi.org/10.1207/s15327957pspr0203_4.
Simmons, Joseph P., Leif D. Nelson, and Uri Simonsohn. 2011. “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.” Psychological Science 22 (11): 1359–66. https://doi.org/10.1177/0956797611417632.
Yamada, Yuki. 2018. “How to Crack Pre-Registration: Toward Transparent and Open Science.” Frontiers in Psychology 9 (September). https://doi.org/10.3389/fpsyg.2018.01831.