At the Open Science Fellows Program, I had the pleasure to meet Dr. Jana Lasser. In her “open science” fellowship she is working on how executable papers can be a tool for more reproducibility and transparency in the natural sciences. The topic is initially irritating for a social science blog (with a qualitative orientation). For me it is important to enable interdisciplinary exchange and to keep thinking outside the box. Therefore, I am very pleased that Jana has written three blog posts about her experience with executable papers:
- What is an executable paper?
- How to create an executable paper and make it accessible?
- Tips and tricks for creating an executable paper
Dr. Jana Lasser received her PhD in physics at the Max Planck Institute for Dynamics and Self-Organization in 2019 and is now working as a postdoctoral researcher at the Complexity Science Hub Vienna. The work presented in this article was made possible by the Wikimedia foundation, Stifterverband and Volkswagen foundation. @janalasser
As I explained in the first post about executable papers, an executable paper is a dynamic piece of software that combines text, raw data, and the code used for the analysis that enables the reader to reproduce the research. In the following, I will introduce the Jupyter Notebook, the tool that I used to create my first executable paper (Lasser 2020). In addition, I will show how to make an executable paper accessible and citable via GitHub.
In general, the format of the executable paper makes the analysis more open, by allowing readers to re-create the analysis while reading the scientific publication. The Jupyter Notebook makes this process accessible by combining text and executable code into a single document that is easy to follow, without jumping between several programs and windows. The Jupyter Notebook also allows even readers that are potentially unfamiliar with the programming language used for the analysis to run code without locally installing any additional software (more on that below).
A Jupyter Notebook is a document that is composed of text elements and code blocks. The text is written in Markdown, a markup language that is very accessible and easy-to-learn. The markup language is used to quickly format text in a human readable and web-friendly way. As a result, similar to a website, you can display Jupyter Notebooks in a browser. The screenshot below shows how a text element looks in Markdown (left) and how it is displayed by the Browser (right).
Next to the text, the document contains code blocks that can interpret code from a variety of programming languages. The screenshot below shows an example of such a code block, in which a data set is loaded and displayed as a table.
The code displayed in this specific code block is written in the programming language Python. Python is a good choice for an executable paper, because it has a large user base and a large variety of well-documented and well-maintained libraries for scientific computing. With a little bit of effort and by writing readable instead of elegant code, Python can be read almost as a description of what the code is actually doing. This allows readers who are unfamiliar with the syntax of the programming language to follow the steps of the analysis and validate them by running the code themselves.
As a side note: Jupyter Notebooks also support programming languages other than Python (here is a full list of supported languages). If you are already familiar with the programming language R (instead of Python), then the tool R Markdown is also a good alternative to Jupyter Notebooks, because it natively interacts with other R-components.
A fundamental part of producing an executable paper is making all material, from the actual paper to the data, available to readers by hosting it on an accessible platform. In the future, journals adopting this kind of format will hopefully take over all the tasks associated with hosting. Until that is the case however, we have to publish the paper ourselves.
To publish our executable paper, we have to overcome three challenges in terms of hosting: First all the resources (code, images, data) need to be accessible to the reader – we are going to use GitHub to achieve this. Second, since the executable paper is not a static artifact, but rather an interactive and dynamic piece of software, it needs to be hosted such that the user can interact with the document (run the code), preferably without having to install additional software. Here binder helps to turn the static Git repository into a dynamic document ready for user interaction. Lastly, since we are creating a scientific publication after all, we want to assign a persistent DOI to the executable paper to make it citable. For this, we are going to use Zenodo.
The first challenge is easily solved by using Git and GitHub. Putting the text and code under version control in a Git repository presents additional advantages: it allows to save the history of changes, and manages changes by different people working on the same document. Sending the code to a remote repository on GitHub is a minor additional step, and has the benefit of creating a backup in a remote server. GitHub also allows for an easy setup of a project description through the README file, and the creation of a license.
As regards the second challenge, fortunately, for interpreted programming languages (such as Python) online hosting solutions such as binder are available. GitHub allows for easy integration of binder into an existing repository with a simple badge (just like a license badge). Just copy-paste the URL of your GitHub repository into the first field on the binder webpage and insert the resulting URL into the README file of your GitHub repository. This will display a badge that leads readers who click on it to an interactive version of your executable paper. This way, readers can execute the code without the need to download any data or install any software. Also, readers cannot “break” anything by deleting or modifying code, because every reader gets their own private instance of the executable paper. Large projects (such as the executable paper I created) can take a couple of minutes to load on binder, but that is a minor tradeoff when compared to the need to locally install a new programming language.
If you have all your materials for the executable paper uploaded on GitHub, it is easy to create a release (i.e. a “frozen” version of the current state) of the paper and add a persistent DOI to the release to make it citable, for example by using Zenodo. The workflow and interaction between GitHub and Zenodo is nicely described in this guide and takes about 5-10 minutes to complete. Ultimately, publishing an executable paper on GitHub is equivalent to publishing a preprint on a preprint server.
In the last part of this series, I will share a few more tips and tricks for an effective use of the tools introduced above for scientific publications.
 Lasser, J. (2020). JanaLasser/salt-polygons-are-caused-by-convection: Final version for open science fellowship project “executable papers” (Version v1.0). Zenodo. https://doi.org/10.5281/ZENODO.3831237