Parallel Runtime Environment for Scalable Task-Oriented Computations
License
davidozog/PRESTO
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
PRESTO: Parallel Runtime Environment for Scalable Task-Oriented Compuations =========================================================================== Presto is a distributed framework whose goal is to support various parallel computational models (e.g. master/worker, pipeline, graph-based, MapReduce, etc.) with various programming adaptors (Matlab, Java, C++, etc.) Currently, this repository includes code that supports the master/worker model with Matlab, Java, Python, or C++ adaptors. INSTALLATION ============ Please read the INSTALL file. USING PRESTO WITH MATLAB ======================== After installing, you can use the following command to count the number of available generic compute nodes on the Aciss cluster: interactive_job --count ( Note: if you are not using Aciss, then you may be able to customize this script in $PRESTO/bin/ for your PBS scheduler.) Then you can launch PRESTO with some number of MATLAB instances less than the number printed above. For example, if 4 or more compute nodes are available, then you can issue the following command to launch an interactive session in the resource queue with 4 processes: interactive_job -n 4 Then you simply launch presto: presto For more custom usage information, type: presto -h This will launch a single interactive Matlab environment on the current node, and 3 other Matlabs running in the background on the other 3 compute nodes. When using PRESTO with Stingray, you need to use a special branch that allows workers to use the Matlab Parallel Computing Toolbox: export STINGRAY=$STINGRAY/branches/PRESTO USING PRESTO WITH JAVA ====================== If are using Java computational engines, then do the following to launch PRESTO: presto --engine java --app <JavaAppName> where <JavaAppName> is the name of your Java application. TESTING PRESTO ============== Besides running Stingray with the .m files above, you should be able to test the installation with this simple example. First, begin by launching PRESTO as described above with 4 compute nodes. If the launch worked, you should see 3 lines confirming the connection of the workers to the master process: Worker connected: ('192.168.1.22', 33466) Worker connected: ('192.168.1.22', 33467) ... If so, type the following Matlab commands into the interative session: m = getenv('PRESTO'); load([m, '/src/PRESTO/my_split.mat']) load([m, '/src/PRESTO/my_shared.mat']) [x y] = send_jobs_to_workers('Rinth_testfunc', {'aStation', 'tlMisfit_sub'}, {'srModel'}) When the quick computation is finished, the x.A0 object should contain: >> x.A0 ans = 34 34 34 34 ans = 35 35 35 35 ans = 36 36 36 36 and the y object should contain: >> y(1) ans = bvec: 31 res: 31 ptr: 31 ttime: 31 inode: 31 ray_minz: 31 niter: 31 icntTT: 31 zssqr: 31 wssqr: 31 sumsd: 31 nssqr: 31 nelsum: 31 nramat: 31 >> y(2) ans = bvec: 32 res: 32 ptr: 32 ttime: 32 inode: 32 ray_minz: 32 niter: 32 icntTT: 32 zssqr: 32 wssqr: 32 sumsd: 32 nssqr: 32 nelsum: 32 nramat: 32 >> y(3) ans = bvec: 33 res: 33 ptr: 33 ttime: 33 inode: 33 ray_minz: 33 niter: 33 icntTT: 33 zssqr: 33 wssqr: 33 sumsd: 33 nssqr: 33 nelsum: 33 nramat: 33 To safely shutdown PRESTO, type the following commands: >> kill_workers >> exit
About
Parallel Runtime Environment for Scalable Task-Oriented Computations
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published