IEGen 2009-6-1: Automatic Inspector/Executor Generation for Irregular Computations

Alan LaMielle (lamielle at cs dot colostate dot edu)
Michelle Strout (mstrout at cs dot colostate dot edu)

TODO: Add simple example usage and refer to ./run_tests for more complicated usage
TODO: Format so it looks a bit nicer

INSTALLING:

**************************************************
-QUICK START:  Execute the following commands from the root of the project
               directory to build/install IEGen. These commands assume that
               Polylib and CLooG are installed in $HOME/software. Read farther
               below for the details of these commands.

# This didn't work due to $'s or something, 3/19/09, MMS
#setenv LD_LIBRARY_PATH $LD_LIBRARY_PATH:.:$HOME/software/lib

setenv SWDIR /s/bach/e/proj/rtrt/software/
setenv LD_LIBRARY_PATH $SWDIR/cloog/lib:$SWDIR/polylib/lib

# this didn't work, 3/19/09, MMS
#setenv PYTHONPATH $PYTHONPATH:$PWD

setenv PYTHONPATH $PWD

./waf configure --prefix=$PWD --polylib-loc=$SWDIR/polylib --cloog-loc=$SWDIR/cloog
./waf build
./waf install
./run_tests
./run_example moldyn_fst cpack
**************************************************

=============
Usage Notes

    The main code generator is iegen.  To see what options are available
    including debug options use the --help command line option.

        ./iegen/bin/iegen --help

    Example usage of iegen is as follows:

        ./iegen/bin/iegen --output-progress examples/moldyn_fst_notrans.spec
    
    The run_example script constructs a call to iegen and compiles and runs
    the resulting code.  For example,
        ./run_example moldyn_fst cpack_align notrans trans compile run

    does the following:
        iegen/bin/iegen examples/moldyn_fst_notrans.spec -o examples/moldyn_fst_notrans.c --inspector-name=inspector_notrans --executor-name=executor_notrans
        iegen/bin/iegen examples/moldyn_fst_cpack_align.spec -o examples/moldyn_fst_trans.c --inspector-name=inspector_trans
        g++ examples/moldyn_fst_cpack_align.c -g -I./src/dev src/dev/ExplicitRelation.c src/dev/RectDomain.c src/dev/RectUnionDomain.c src/dev/Tuple.c src/dev/util.c src/dev/ERG_cpack.c -o examples/moldyn_fst_cpack_align
        examples/moldyn_fst_cpack_align

    The debug output is summarized as follows:
    -o, --output-code   generated code
    --output-progress   progress messages 
        which is on by default
    --output-modified   components as they are modified
        access relations and statements
    --output-detail     more detailed messages
        full iteration space, transformations being applied, some simplification detail
    --output-info       general information messages
        doesn't seem to provide any info
    --output-debug      debug messages
    --debug
        debug information about all operations like compose, union, inverse, etc. being performed
    --output-error      error messages


=============
-Dependencies: Python version >= 2.4.0, CLooG version >= 0.14.0, and Polylib
|
|-As IEGen is mostly written in Python, Python is a necessary requirement.
|
|-IEGen depends on a polyhedral loop scanning tool called CLooG: http://www.cloog.org/
| This tool will need to be installed before you can use IEGen.
|
|-Indirectly IEGen depends on the Polylib library as CLooG uses it; this will
  need to be installed as well.

-IEGen has currently only been built and tested on 32 and 64 bit x86 Linux machines
 running Fedora 9/10, Gentoo, and Ubuntu. If you have success building and running this software
 on other operating systems or architectures, please let us know.

-IEGen uses a really slick Python based build system called waf.

-Use the following commands to configure, build, and install IEGen using waf:

-1)      ./waf configure --prefix=location_to_install_to --polylib-loc=path_to_polylib --cloog-loc=path_to_cloog
   (exs: ./waf configure --prefix=$PWD --polylib-loc=$HOME/software/polylib/ --cloog-loc=$HOME/software/cloog
         ./waf configure --prefix=$PWD --polylib-loc=$HOME/software --cloog-loc=$HOME/software)

   -Note: the Polylib and CLooG paths should be a directory CONTAINING the
          lib/ and include/ directories where Polylib/CLooG installed their
          libraries and headers respectively. It is NOT a path to a lib/ or
          include/ directory.

          The prefix is the location where IEGen will be installed. A single
          directory, named 'iegen', will be installed to this location. See
          below for a detailed description of what is conatined in this
          directory.

-2) ./waf build

-3) ./waf install

-4) ./run_tests (Optional)

-5) ./run_example moldyn_fst cpack

-Further building/installation related notes:

 -To be sure that the dynamic linker can find the CLooG and Polylib
  libraries at loadtime, be sure that the environent variable
  LD_LIBRARY_PATH includes the location to the CLooG and Polylib libraries.

 -Step 4 will run a test script that exercises the functionality of IEGen.

 -This is a python script that imports the iegen python module that is the
  root of the IEGen system.

 -As such, Python must be able to find this module:

  -If IEGen is installed to the example location, $PWD, then Python will
   surely be able to find the iegen module when running ./run_tests.

  -If IEGen is installed elsewhere, be sure that you set the environment
   variable PYTHONPATH to include the location where the 'iegen' directory
   is located.

 -The ./run_example script runs IEGen, generates code based on an example spec
  file, compiles the generated code and associated driver, and runs the
  compiled code.  Run ./run_example for usage details.

Writing a new example spec file/test driver:
-The examples live in the examples/ directory.
-Create a new spec file called '[spec_name]_[transformation_suffix].spec'
-For example, the moldyn_fst spec file that applies a single cpack
 transformation is called 'moldyn_fst_cpack.spec'.
-Corresponding to each runnable example spec file is a C driver that calls
 the generated code and tests it.
-The name of the C drivers should match the name of the corresponding spec
 file: moldyn_fst_cpack.spec should have a moldyn_fst_cpack.c

-What gets installed:

iegen: Root module for IEGen
|-- __init__.py: iegen module init
|-- _iegen.py: Main IEGen classes/functions
|-- ast: Set/Relation AST Module
|   |-- __init__.py: ast module init
|   |-- _ast.py: Set/Relation AST classes
|   - visitor: Set/Relation AST Visitor module
|       |-- __init__.py: visitor module init
|       |-- _trans_visitor.py: Translates a Set AST to a CLooG domain matrix
|       |-- _visitor.py: Depth First Visitor of Set/Relation ASTs
|-- parser: Set/Relation Parser Module
|   |-- __init__.py: parser module init
|   |-- _parser.py: Set/Relation parser
|-- pycloog: Python module for interfacing with CLooG
|   |-- __init__.py: pycloog module init
|   |-- _pycloog.so: Shared library that interfaces with CLooG
|   - _pycloogpy.py: Python code that loads and interacts with _pycloog.so
|-- test: C tests
|   |-- driver_moldyn_FST_hand
|   - test_Hypergraph
- util: IEGen Utility Module
    |-- __init__.py: util module init
    |-- _util.py: IEGen utility classes/functions

(Note: This directory tree was obtained from the useful unix command 'tree')

-Alan LaMielle, 2008/2009