Skip to content

dsavenko/ck-env

 
 

Repository files navigation

Fighting software and hardware chaos in research

Feel free to provide your favorite software descriptions (see examples here). Do not hesitate to ask for instructions and help here!

logo logo License

Researchers nowadays suffer from continuously changing software and hardware stack when sharing, customizing and reproducing experimental results:

SW/HW chaos

This is an extension repository to the Collective Knowledge Framework to let users automatically detect or install multiple versions of various software (compilers, libraries, tools, models, data sets) across diverse hardware with Linux, Windows, MacOS and Android operating systems. This, in turn, allows researchers implement portable, customizable and reproducible experimental workflows as described here.

The community gradually describes various software using CK entries with JSON meta and simple Python scripts to detect coexisting installations. See already shared software descriptions and package descriptions.

Please, check CK portable experimental workflows for more details!

Contributors

  • General Motors and dividiti use CK to crowdsource benchmarking and optimization of CAFFE with different versions of compilers, libraries, models and data sets: public CK repo
  • ARM and the cTuning foundation use CK to systematize SW/HW co-design: HiPEAC Info'45 page 17, ARM TechCon'16 presentation and demo, public CK repo
  • The community gradually provides description of all existing software in the CK format: GitHub, Wiki
  • The community gradually adds various packages to automatically rebuild software: Wiki

Acknowledgments

logo logo logo logo logo logo logo logo logo logo logo logo logo

License

  • BSD, 3-clause

Prerequisites

Installation

 $ (sudo) pip install ck
 $ ck pull repo:ck-env

 $ ck list soft
 $ ck list package

Usage

You can easily detect and register in the CK all the instances of GCC and LLVM as following:

 $ ck detect soft:compiler.gcc
 $ ck detect soft:compiler.llvm

You can now see multiple versions of the detected software registered in the CK as following:

 $ ck show env

You can then compile and run unified CK benchmarks shared by the community using any of the above compiler instances (GCC, LLVM, ICC ...) and their versions simply as following:

 $ ck pull repo:ck-autotuning
 $ ck pull repo:ctuning-programs

 $ ck compile program:cbench-automotive-susan --speed
 $ ck run program:cbench-automotive-susan

If you have Android NDK and SDK installed, CK can automatically detect it together with compiler versions (GCC, LLVM), register them and let you compile and run benchmarks on Android simply via:

 $ ck compile program:cbench-automotive-susan --speed --target_os=android21-arm-v7a
 $ ck run program:cbench-automotive-susan --target_os=android21-arm-v7a

You can find further details about our customizable and cross-platform package/environment manager here.

Publications

The concepts have been described in the following publications:

@inproceedings{ck-date16,
    title = {{Collective Knowledge}: towards {R\&D} sustainability},
    author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
    booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
    year = {2016},
    month = {March},
    url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}

Feedback

About

CK extensions - managing environments for software and hardware

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 45.5%
  • C++ 17.9%
  • C 10.7%
  • CMake 9.6%
  • HTML 8.0%
  • Shell 7.1%
  • Other 1.2%