What is an executable paper?

At the Open Science Fellows Program, I had the pleasure to meet Dr. Jana Lasser. In her “open science” fellowship she is working on how executable papers can be a tool for more reproducibility and transparency in the natural sciences. The topic is initially irritating for a social science blog (with a qualitative orientation). For me it is important to enable interdisciplinary exchange and to keep thinking outside the box. Therefore, I am very pleased that Jana has written three blog posts about her experience with executable papers:

  1. What is an executable paper?
  2. How to create an executable paper and make it accessible?
  3. Tips and tricks for creating an executable paper

Dr. Jana Lasser received her PhD in physics at the Max Planck Institute for Dynamics and Self-Organization in 2019 and is now working as a postdoctoral researcher at the Complexity Science Hub Vienna. The work presented in this article was made possible by the Wikimedia foundation, Stifterverband and Volkswagen foundation. @janalasser

What is an executable paper?

Enhanced by the current Covid-19 pandemic, Open Science practices have taken an even more central role in the way we conduct research. This does not stop at the data being available, but includes transparency of the methods employed to reach conclusions, and the dissemination of the results. Too often, even if data sets are openly available, analysis code is hidden away in cryptic scripts that are “available upon request” and not thought for other researchers to read and use them. A prominent example of this problem is the simulation based research that informed large parts of the UK’s policy decisions regarding Covid-19. The research received heavy criticism because the underlying simulation code was not accessible and not understandable at first. A way to remedy this problem and support data availability, transparency of methods is the “executable paper” – i.e. formatting a scientific publication as a dynamic piece of software that combines text, raw data, and the code used for the analysis, that a reader can interact with and that makes the process of the generation of insights transparent.

In principle, the idea is to enable the reader to reproduce each step taken to arrive at a publication’s conclusion, from the raw data to the polished plot. Therefore, a fundamental part of producing an executable paper is making all material, from the actual paper to the data, available to readers by hosting it on an accessible platform. The executable paper itself should be linked to a repository containing the raw data and should be able to

  • display nicely formatted text, including references and links (just like a journal article),
  • display figures, plots, and possibly also videos and interactive elements,
  • display the code used for creating such plots from data,
  • interpret such code and user input in case of interactive elements, and
  • be composed of completely Open Source components and hosted under a free license to be easy to share and re-use.

Traditional journals increasingly support uploading online-only supplements, including analysis codes, images and even videos. Nevertheless, these materials are sometimes incomplete and clearly separated from the main publication. This makes the reproduction of the research from the raw data to the polished figure shown in the paper difficult or even impossible. This can be remedied by an executable paper that collects and combines all relevant information in one master document. This way, following the details necessary to reproducing the research does not require multiple switches between different documents and software, thus becoming more transparent.

Sadly, the concept of executable papers is not established in traditional journals and only a handful of computer science and math journals support a variation of the format – the “reproducible paper”. This publication type is centered around a simulation code that can be run by the reader to generate data and perform their analysis, thus reproducing the whole research process down to the results. The main difference to the executable paper is that, while the executable paper relies on empirical data, the reproducible paper’s code generates the data, thus putting a larger emphasis on the description of the code.

Creating an executable paper takes a significant amount of time, but I can only recommend the experience. It gave me new confidence about my own research, and forced me to revisit choices I had made in the past, sometimes in a rush and with less understanding than I have today. Creating an executable publication also gave me the chance to use many open tools and workflows I already knew about, and to pull them together into a more open and transparent way of academic working. In retrospective, creating an executable paper is a journey through almost all aspects of Open Science: from open and accessible data to open and transparent tools and software, ending with an open publication composed of open and transparent content.

Examples

In the next part of this series, I will give an introduction how to create an executable paper.

Cite this article as: Jana Lasser, "What is an executable paper?," in Sozialwissenschaftliche Methodenberatung, 18/06/2020, https://sozmethode.hypotheses.org/1045.

Autor: Jana Lasser

I am a PostDoc at the Computational Social Science lab at the Graz University of Technology and the Complexity Science Hub Vienna. Drawing from my experience in nonlinear dynamical systems, my current research activity centres around emergent phenomena in complex social systems. I employ methods from machine learning, data science, sociophysics to understand a diverse range of topics, from emotion dynamics to the spread of diseases and misinformation in the field of Computational Social Science. For my PhD in physics, I conducted research on pattern formation in salt deserts at the Max Planck Institute for Dynamics and Self-Organization and received my degree from the Georg-August-University in Göttingen, Germany in 2019. Next to my research I care deeply about how the scientific community functions and dysfunctions. I try to improve reproducibility and transparency of research by following Open Science practices and being an outspoken and active proponent of Open Science practices. During my time as the president of the Max Planck PhD association I learned about the rampant mental health problems of many academics and joined the COST action on researcher mental health to help change the academic system for the better.

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.