Skip to content

davidozog/PRESTO

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

72 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PRESTO: Parallel Runtime Environment for Scalable Task-Oriented Compuations
===========================================================================
Presto is a distributed framework whose goal is to support various parallel
computational models (e.g. master/worker, pipeline, graph-based, MapReduce, 
etc.) with various programming adaptors (Matlab, Java, C++, etc.) 
Currently, this repository includes code that supports the master/worker 
model with Matlab, Java, Python, or C++ adaptors.


INSTALLATION
============
Please read the INSTALL file.


USING PRESTO WITH MATLAB
========================
After installing, you can use the following command to count the 
number of available generic compute nodes on the Aciss cluster:

  interactive_job --count

( Note: if you are not using Aciss, then you may be able to
  customize this script in $PRESTO/bin/ for your PBS scheduler.)

Then you can launch PRESTO with some number of MATLAB instances 
less than the number printed above.  For example, if 4 or more 
compute nodes are available, then you can issue the following 
command to launch an interactive session in the resource queue 
with 4 processes:

  interactive_job -n 4

Then you simply launch presto:

  presto

For more custom usage information, type:

  presto -h

This will launch a single interactive Matlab environment on the 
current node, and 3 other Matlabs running in the background on 
the other 3 compute nodes.

When using PRESTO with Stingray, you need to use a special branch 
that allows workers to use the Matlab Parallel Computing Toolbox:

  export STINGRAY=$STINGRAY/branches/PRESTO


USING PRESTO WITH JAVA
======================
If are using Java computational engines, then do the following 
to launch PRESTO:

  presto --engine java --app <JavaAppName>

where <JavaAppName> is the name of your Java application.


TESTING PRESTO
==============
Besides running Stingray with the .m files above, you should be 
able to test the installation with this simple example.  

First, begin by launching PRESTO as described above with 4 compute 
nodes. If the launch worked, you should see 3 lines confirming 
the connection of the workers to the master process:

  Worker connected:  ('192.168.1.22', 33466)
  Worker connected:  ('192.168.1.22', 33467)
  ...

If so, type the following Matlab commands into the interative 
session:

  m = getenv('PRESTO');
  load([m, '/src/PRESTO/my_split.mat'])
  load([m, '/src/PRESTO/my_shared.mat'])
  [x y] = send_jobs_to_workers('Rinth_testfunc', {'aStation', 'tlMisfit_sub'}, {'srModel'})

When the quick computation is finished, the x.A0 object should 
contain:

    >>  x.A0

    ans =
    
        34    34
        34    34
    
    
    ans =
    
        35    35
        35    35
    
    
    ans =
    
        36    36
        36    36

and the y object should contain:

    >> y(1)
    
    ans = 
    
            bvec: 31
             res: 31
             ptr: 31
           ttime: 31
           inode: 31
        ray_minz: 31
           niter: 31
          icntTT: 31
           zssqr: 31
           wssqr: 31
           sumsd: 31
           nssqr: 31
          nelsum: 31
          nramat: 31
    
    >> y(2)
    
    ans = 
    
            bvec: 32
             res: 32
             ptr: 32
           ttime: 32
           inode: 32
        ray_minz: 32
           niter: 32
          icntTT: 32
           zssqr: 32
           wssqr: 32
           sumsd: 32
           nssqr: 32
          nelsum: 32
          nramat: 32
    
    >> y(3)
    
    ans = 
    
            bvec: 33
             res: 33
             ptr: 33
           ttime: 33
           inode: 33
        ray_minz: 33
           niter: 33
          icntTT: 33
           zssqr: 33
           wssqr: 33
           sumsd: 33
           nssqr: 33
          nelsum: 33
          nramat: 33


To safely shutdown PRESTO, type the following commands:

    >> kill_workers
    
    >> exit

About

Parallel Runtime Environment for Scalable Task-Oriented Computations

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published