RepairBox is an ongoing effort to provide a series of controlled environments for performing experiments on automated program repair of C programs. RepairBox provides a number of bug scenarios, adapted from existing publicly-available datasets, including ManyBugs, GenProg's earlier benchmarks, and several datasets from the Software Infrastructure Repository.

As well as providing the files necessary to study each bug, RepairBox also provides a minimal, isolated environment for execution, in the form of an accompanying Docker container. By providing a container for each bug, rather than a single, monolithic virtual machine for a group of bugs, RepairBox incurs fewer overheads (resulting in a much higher performance, crucial for rigorous experimental trials), makes dependencies easier to manage, and results in a smaller disk footprint.

For more information on Docker, the interested user is referred to https://docs.docker.com/learn/.

To install the RepairBox platform, we advise that users produce a shallow clone of this repository using the command below:

$ git clone git://github.com/squaresLab/RepairBox --depth 0

Producing a shallow clone takes far less time, and the git protocol both reduces that further and helps to prevent any hangs you might otherwise experience using the HTTPS protocol.

To use RepairBox, the following packages must be installed:

• Python 2.7+
• Docker

Additionally, RepairBox requires the following Python packages:

• tabulate
• requests
• yaml

These may be installed via the following pip command:

$ pip install tabulate requests yaml

RepairBox provides a simple, self-documenting command line interface, via the repairbox executable. Below, we briefly explain and illustrate each of the commands provided by this interface.

Provides a detailed list of available commands.

$ repairbox -h

usage: repairbox [-h] [--version]
                 {build,upload,download,install,uninstall,list,launch} ...

______                 _     ______           
| ___ \               (_)    | ___ \          
| |_/ /___ _ __   __ _ _ _ __| |_/ / _____  __
|    // _ \ '_ \ / _` | | '__| ___ \/ _ \ \/ /
| |\ \  __/ |_) | (_| | | |  | |_/ / (_) >  < 
\_| \_\___| .__/ \__,_|_|_|  \____/ \___/_/\_\
          | |                                 
          |_|                                 

A platform for reproducible empirical studies of program repair

Download at: https://github.com/squaresLab/RepairBox

positional arguments:
  {build,upload,download,install,uninstall,list,launch}

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit

$ repairbox -h build

TODO: add description!

Produces a list of all repair boxes, programs, datasets and tools within RepairBox.

$ repairbox list bugs

Bug                                            Latest    Installed    Remote
---------------------------------------------  --------  -----------  --------
manybugs:libtiff:2005-12-14-6746b87-0d3d51d    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2005-12-21-3b848a7-3edb9cd    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2005-12-27-6f76e76-5dac30f    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2006-02-23-b2ce5d8-207c78a    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2006-02-27-6074705-e6d0c32    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2006-03-03-a72cf60-0a36d7f    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2006-03-03-eec4c06-ee65c74    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2007-07-08-bd2f947-ccc10c7    0.0.0.0   0.0.0.0      0.0.0.0
manybugs:libtiff:2007-07-08-c766cb7-0cc95fb    0.0.0.0   0.0.0.0      0.0.0.0
...

$ repairbox list programs

Program              Bugs
-----------------  ------
manybugs:php          103
robots:arducopter       1
manybugs:libtiff       24
manybugs:python        15
...

$ repairbox list datasets


Dataset      Programs    Bugs
---------  ----------  ------
manybugs            3     142
robots              1       1
tse-2012            0       1
...

$ repairbox list tools


Tool          Image                           Installed
------------  ------------------------------  -----------
genprog       christimperley/genprog:latest   True
searchrepair  searchrepair:latest             True
shuriken      christimperley/shuriken:latest  True
smallcov      squareslab/smallcov:latest      True
...

Installs a given repair box by first attempting to download it, if the remote version is up-to-date, before resorting to building the repair box locally:

$ repairbox install bug manybugs:libtiff:2005-12-14-6746b87-0d3d51d

downloading bug: manybugs:libtiff:2005-12-14-6746b87-0d3d51d
...
downloaded bug: manybugs:libtiff:2005-12-14-6746b87-0d3d51d

This command may also be used to install all bugs belonging to a given dataset:

$ repairbox install bug manybugs

downloading bug: manybugs:libtiff:2005-12-14-6746b87-0d3d51d
...

Or it may be used to install bugs belonging to a given program in a dataset:

$ repairbox install manybugs:python

downloading bug: manybugs:python:69223-69224
...

Installs a registered tool by attempting to download its associated image from DockerHub.

$ repairbox install tool genprog
...

Uninstalls a given repair box, a group of repair boxes belonging to a dataset or program, or a registered tool from the local machine:

$ repairbox uninstall bug manybugs:libtiff:2005-12-14-6746b87-0d3d51d
$ repairbox uninstall bug manybugs:python
$ repairbox uninstall bug manybugs
$ repairbox uninstall tool genprog

Downloads a given repair box onto the local machine, provided that the currently installed version is out of date or not installed. As with the install and uninstall commands, download may accept the name of a single bug, a program, or a dataset:

$ repairbox download bug manybugs:libtiff:2005-12-14-6746b87-0d3d51d
...
$ repairbox download bug manybugs:python
...
$ repairbox download bug manybugs
...

Downloads a registered tool onto the local machine, provided the tool isn't already installed.

$ repairbox download tool genprog
...

Builds a repair box from scratch, provided that repair box isn't already installed on the local machine. Once again, this command accepts the name of a bug, program, or dataset.

$ repairbox build manybugs:libtiff:2005-12-14-6746b87-0d3d51d
...
$ repairbox build manybugs:python
...
$ repairbox build manybugs

To interact with a minimal repair box, the following command can be used:

host> repairbox launch manybugs:libtiff:2005-12-14-6746b87-0d3d51d
docker> echo $PWD
/experiment

By itself, this ability isn't particularly interesting. Let's import some tools into the repair box! Supported tools (found via the list tools command) can be imported via the --with flag:

host> repairbox launch manybugs:libtiff:2005-12-14-6746b87-0d3d51d --with genprog --with smallcov
docker> genprog problem.json

Files can also be shared between the repair box and the host machine (in both directions) via volume sharing. The volume sharing flag -v is used to mount a given directory on the host machine (which will be created if it doesn't already exist) at a given path in the repair box; both of these paths should be absolute. An example usage of the -v flag is given below.

host> repairbox launch manybugs:libtiff:2005-12-14-6746b87-0d3d51d -v /home/foo/bar:/experiment/bar
docker> genprog problem.json

Note, multiple volumes may be mounted by specifying a separate '-v' flag for each.

Whereas the launch command is designed for debugging tools and inspecting bug scenarios, the execute command is designed for conducting controlled, repetable experiments. The execute command runs headlessly, rather than interactively, making it suitable for long-running experiments on remote servers.

Most of the functionality of the execute command is the same as the launch command, except that users are required to specify a shell instruction at the end of the command.

$ repairbox execute -v /home/foo/bar:/experiment/bar \
                    --with genprog \
                    manybugs:python:69223-69224 \
                    genprog problem.json

Discuss how one goes about adding a new tool.

version: 0
tool: genprog
image: christimperley/genprog:latest
environment:
  PATH: "/opt/genprog3:${PATH}"

Discuss how one constructs a compatible Docker container.

Add a link to a page in the Wiki explaining how all of this works under the hood.

To add your own dataset of bugs into RepairBox, you'll first need to supply a .dataset.yaml. We suggest forking the repository and creating a directory for your dataset at the project root. The .dataset.yaml file should be placed in this directory, and should share the same name as the dataset.

For example, let's say that we wanted to put together a new dataset of bugs, known as the notsomanybugs dataset. We would create a notsomanybugs directory at the root of the project, then we would add a notsomanybugs.dataset.yaml file inside the notsomanybugs directory.

The notsomanybugs.dataset.yaml file

Discuss how one might go about adding a bug scenario.