Ostrich2 is a benchmark suite developed in the Sable Lab at McGill University with the objective of studying the performance of languages used for numerical computing. It is a port of the C and JavaScript implementations of the Ostrich benchmarks to the Wu Wei Benchmarking Toolkit with support for more languages such as MATLAB on the way.
We aim to make the suite:
- Consistent and Correct by providing self-checking runners for every language that automatically ensure that the computation result of the benchmarks are consistent across all language implementations and correct with regard to the algorithm for known inputs;
- Representative of the most important and popular numerical algorithms with a proper choice of representative input data with a wide range of benchmarks across known numerical categories (Dwarfs) ;
- Extensible across numerical languages and benchmarks;
- Friendly to language implementation research by factorizing the core computation from the runners to minimize the non-core functions necessary to validate the output of compilers;
- Easy to use by automating the deployment of benchmarks, their test on virtual (web browser and others) and native platforms, as well as the gathering and reporting of relative performance data;
- Fast by making the setup (data generation and loading) and teardown as quick as possible so that most of the time is spent in the core computation in every language;
- Small by minimizing the amount of data needed to download the suite;
- Simple by minimizing the amount of external dependencies and tools required to run the suite;
Although we tried our best to minimize external dependencies, the suite still depends on the following external tools:
- Node.js
- Python/Numpy/Scipy
| Benchmark | Split Runner/Core Computation | Languages | Language-Independent Verification of Results |
|---|---|---|---|
| bfs | no | c,js | no |
| backprop | no | c,js | no |
| crc | no | c,js | no |
| fft | no | c,js | no |
| hmm | no | c,js | no |
| lavamd | no | c,js | no |
| lud | yes | c,js,matlab | yes |
| nqueens | yes | c,js,matlab | yes |
| nw | no | c,js | no |
| pagerank | no | c,js | no |
| srad | no | c,js | no |
| spmv | no | c,js | no |
Please read our wiki for more details on obtaining the suite, description of the benchmarks and instruction on running the benchmarks.
Copyright (c) 2014, Erick Lavoie, Faiz Khan, Sujay Kathrotia, Vincent Foley-Bourgon, Laurie Hendren and McGill University.
- Ostrich: MIT Licence
- OpenDwarfs: GNU Lesser General Public License
- Rodinia: Rodinia Licence
- V8: BSD 3 Licence