This webpage is dedicated to advocating the idea of REproducible Research In Transportation Engineering (RERITE; pronounced as Re-write)
Why bother?
How to validate a scientific finding has bothered me for a long time in my not-so-long academic life so far. Replication is generally regarded as the gold standard of validating a scientific study. Unfortunately, replicating a study is often very difficult if not impossible at all due to various reasons (hint: dollar sign is often involved). These days, budget cut is one of the terms that most frequently appears in the media. For many academics (I'm one of them), securing financial support to do an original study is difficult enough, not to mention the would-be difficulty of asking for support to replicate a previous study. On the other hand, research validation is becoming important and urgent more than ever. Just think about the huge amount of data we are collecting, the high complexity of the algorithms we are developing, the unprecedented computational power we are using on a daily base, the fast-paced (it's getting even faster!) publication industry, just to name a few. Good news: Reproducible research comes to rescue. Well, kind of. Reproducible research is a compromise of replication, rather than a replacement of replication. Instead of validating a study (this is the main mission of replication), reproducible research aims to validate a study's data analysis. Simply speaking, reproducible research is an extra (but very important) effort from the authors of a publication to share their data, codes, and instructions on how to piece them together for the purpose of enabling a third party to obtain the identical results reported in their paper. If you are not happy with my explanation, please take a look at the definition given by Wikipedia:
"The term reproducible research refers to the idea that the ultimate product of academic research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. that can be used to reproduce the results and create new work based on the research. This is the essential part of open science." (accessed at 30/11/2021; url: https://en.wikipedia.org/wiki/Reproducibility#Reproducible_research)
Update:
"The term reproducible research refers to the idea that the ultimate product of academic research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. that can be used to reproduce the results and create new work based on the research. This is the essential part of open science." (accessed at 30/11/2021; url: https://en.wikipedia.org/wiki/Reproducibility#Reproducible_research)
Update:
- The RERITE Working Group has launched its website; please click here to visit. (17/06/2024)
- Honoured to be one of the two inaugural Open Research Fellows at the University of Queensland. (08/01/2024)
- Talk at AIMOS2023 Conference organised by Association for Interdisciplinary Meta-Research and Open Science. (21/11/2023)
- Our workshop proposal "Reproducible Research (RR) – Why, What, and How" sponsored by eight committees has been selected by TRB24. Please join us at 9:00AM - 12:00PM, Thursday, Jan 11, 2024 in Washington D.C. More detail can be found here. (12/10/2023)
- We have converted the Matlab codes into Python for our TR Part C paper (Title: "About calibration of car-following dynamics of automated and human-driven vehicles: Methodology, guidelines and codes", which was co-authored with Dr Vincenzo Punzo and Dr Marcello Montanino), thanks to the great effort from my PhD student Pengcheng Fan. You can download the Python script with related data files from here. (12/10/2023)
- The data collected in the driving simulator experiment for my DECRA can be downloaded from this link. In this folder, besides the data files you will also find several other files: a data dictionary, a copy of the questionnaire, the responses to the questionnaire, and disclaimer. (31/03/2023)
- I'm coordinating a Lectern Session focusing on Reproducible Transportation Research at TRB 2023. (01/11/2022)
- SimCCAD, an open-source integrated simulation platform for Conventional, Connected and Automated Driving (CCAD) can be downloaded from here. (1/10/2022)
- I'm coordinating a Call for Papers "Towards Reproducible Transportation Research" at TRB 2023. More detail can be found here. (01/06/2022)
- Codes and data used in our TR Part C paper (Title: "A pattern recognition algorithm for assessing trajectory completeness") can be downloaded from here , and the codes can be used to assess vehicular trajectory's completeness in terms of driving regimes; please read the disclaimer before you use any part or any version of the codes). (28/09/2021)
- Our recent TR Part C paper (Title: "About calibration of car-following dynamics of automated and human-driven vehicles: Methodology, guidelines and codes", which was co-authored with Dr Vincenzo Punzo and Dr Marcello Montanino) is reproducible! the codes (in Matlab) and data used in this paper can be downloaded from here , The Python script can be downloaded from here, and the reproducible version of this paper using R Markdown can be downloaded from here; please read the disclaimer before you use any part or any version of the codes). (15/08/2021; 12/10/2023)
- My paper on how to conduct reproducible research in Transportation Research (title: "Reasons, Challenges and Some Tools for Doing Reproducible Research in Transportation Research") was published in Communications in Transportation Research. The published version of the paper can be downloaded from here, and the source code (in R Markdown) for reproducing the paper can be downloaded here. (16/08/2021)
How to make a paper reproducible
- a "Hello World" example (Download the codes for generating the example; download the example: The existence of capacity drop phenomenon): it's my intent to create a simple example to demonstrate how a reproducible research looks like by making one of my recent publications reproducible. Yes, you guessed it, the original form/published version of my papers is not reproducible.
- Additional resources
- Quarto - a new tool for reproducible research. Click here for more information.
- An online course on reproducible research offered by Johns Hopkins University through Coursera. Here is the course link.
- Reproducible Research with R and R Studio (Book; Second Edition) by Christopher Gundrud. Its Amazon link.
- Dynamic Documents with R and knitr (Book; Second Edition) By Yihui Xie. Its Amazon link.
- Quarto - a new tool for reproducible research. Click here for more information.