jTReqS
=====

This is the source code of jTReqS the smart and shiny Tape Request Scheduler
for HPSS. This implementation is in Java.


* Prerequisites

 - Required libraries (included in the sources):
   o Logback (Classic and Core).
   o SLF4J.
   o Commons CLI.
   o Commons Collections.
   o Commons Configuration.
   o Commons Logging.
   o Commons Lang.
   o MySQL connector for java.
   o (Optionally) DB2 connector for Java.

 - Compilation Environment:
   o Linux.
   o Java (wih JAVA_HOME set).
   o gcc compiler.
   o Maven (optional but highly recommended).
   O (Optionally) IBM DB2 SQLJ compiler. 

 - Other:
   o a proper HPSS keytab.


* Compilation Instructions

(With Maven)

 - Set the environment.
     $ export HPSS_ROOT=${HPSS_ROOT:-/opt/hpss}
 - Remove any existent maven artifact
     $ rm -rf ~/.m2/repository/
 - Compile the project with Maven.
     $ mvn
   There could be many test failures, and they are due to the local database
   configuration. If you change the file:
      java/scr/test/config/jtreqs.conf.test.properties
   you will not have these errors.
 - Copy/Move and extract the file
      release/target/jtreqs.tar.gz
   to a good location.

(Without Maven)

 - Create a building directory.
     $ mkdir bin
 - Copy the necessary files.
     $ cp java/src/test/scripts/jtreqs.sh bin
     $ cp release/src/main/config/jtreqs.conf.properties bin
     $ cp release/src/main/config/logback.xml bin
 - Create a file with the project version (if known).
     $ echo jTReqS Server \${project.version} > bin/version.txt
 - Run javac over the source directory.
     $ javac -d bin -encoding UTF8 -cp \
vendor/apache/commons-cli-1.2/commons-cli-1.2.jar:\
vendor/apache/commons-collections-3.2.1/commons-collections-3.2.1.jar:\
vendor/apache/commons-configuration-1.6/commons-configuration-1.6.jar:\
vendor/apache/commons-lang-2.5/commons-lang-2.5.jar:\
vendor/apache/commons-logging-1.1.1/commons-logging-1.1.1.jar:\
vendor/mysql/mysql-connector-java-5.1.13/mysql-connector-java-5.1.13-bin.jar:\
vendor/slf4j/slf4j-1.6.1/slf4j-api-1.6.1.jar \
java/src/main/java/fr/in2p3/cc/storage/treqs/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/activator/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/controller/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/dispatcher/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/exception/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/process/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/selector/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/control/starter/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/hsm/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/hsm/hpssJNI/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/main/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/model/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/model/dao/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/model/exception/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/persistence/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/persistence/helper/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/persistence/mysql/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/persistence/mysql/dao/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/persistence/mysql/exception/*.java \
java/src/main/java/fr/in2p3/cc/storage/treqs/tools/*.java
 - Create the native library.
     $ javah -classpath bin/ -d bin/ \
-jni fr.in2p3.cc.storage.treqs.hsm.hpssJNI.NativeBridge
     $ gcc -I /opt/hpss/include/ -DLINUX -fPIC -Wall -o bin/HPSSBroker.o \
-c native/src/main/c/HPSSBroker.c
     $ gcc -I ${JAVA_HOME}/include/ -I /opt/hpss/include/ -I bin/ \
-DLINUX -Wall -fPIC -o bin/NativeBridge.o -c native/src/main/c/NativeBridge.c
  This is the MOST important line of code of this project (because of this, this
  project could have been thrown to the garbage.)
     $ ld -shared -L/opt/hpss/lib -lhpss -lhpssunixauth \
-o bin/libNativeBridge.so bin/HPSSBroker.o bin/NativeBridge.o


* Installation and configuration

** Database (MySQL)

 - Create the database and grant proper permissions to user <jtreqs>.
   > CREATE DATABASE jtreqs;
   > CREATE USER 'jtreqs'@'localhost' IDENTIFIED BY 'jtreqs';
   For tests.
   > GRANT ALL ON jtreqs.* TO 'jtreqs'@'localhost';
   For production.
   > GRANT SELECT, INSERT, UPDATE ON jtreqs.* TO 'jtreqs'@'localhost';

   Then, you can create the tables in the database. In the 'doc' directory,
   there is a script called 'mysql.sql' that contains the dump of the database.
   However, the application itself can show the queries to create the tables
   by calling the application with the option -db
      jtreqs.sh -db
   And, if you want, the application can create the tables if it is called like
   this:
      jtreqs.sh -dbc
   For more information about executing the application, please check the
   'Execution' section.

** Database (DB2)

 - Create the user jtreqs in the operative system.
   > sudo useradd jtreqs
   > sudo passwd jtreqs
 - Create the database and grant the proper permissions to user <jtreqs>.
   > db2 CREATE DATABASE jtreqs
   > db2 CONNECT TO jtreqs
   > db2 GRANT DBADM ON DATABASE TO USER jtreqs
   
   Then, you can create the tables in the database. In the 'doc' directory,
   there is a script called 'db2.sql' that contains th dump of the database.
   However, the application itself can show the queries to create the tables
   by calling the application with the option -db. Remember to change the
   data source in order to use DB2 instead of MySQL.
      jtreqs.sh -db
   For more information about executing the applications, please check the
   'Execution' section.

** Table's configuration

   Once, the tables are created, you need to indicate the quantity of drives
   that the application can use to mount tapes. Remember that the HSM (HPSS)
   could use many drives to do migrations, thus, it is NOT recommended to put
   the totality of drives for the applications. Depending on the
   reading/writing ratio, this value should be configured. If you are going to
   put 15 T10K-B drives you should insert a row like:
      insert into JMEDIATYPES (id, name, drives) values (1, "T10K-B", 15);
   When using Fair-Share, you have to decide the percentage of drives that will
   be used for a given client when all drives are requested. This will assure a
   minimum quantity of drives per user when all drives are being used. For
   example, if you want to reserve 20% of drives T10K-B for an experience called
   Atlas, you will do:
      insert into JALLOCATIONS (id, user, share) values (1, "atlas", 0.20);

** System

*** User

   The application needs a user to be executed, thus it is necessary to create a
   user in the system that will have the following rights:
   - To execute the application (+x in the bin/jtreqs.sh)
   - To read the keytab (+r in the keytab path)
   - To write in the log files of the application. If the default values are
   chosen, then the application has to have the rights to write in
      /var/log/jtreqs.

*** Files and Directories

   Depending on the configuration, you must create a directory to write the
   logs. If the default configuration for 'logback' is selected, then you have
   to create the directory
      /var/log/jtreqs

   In order to start the application as a service, there is a daemon called
   jtreqsd and it is found in the 'bin' directory. You can create a symbolic
   link to this file, and then you can do the following actions: start, stop,
   query the status. You just have to do
      ln /etc/init.c/jtreqds /opt/jtreqs/bin/jtreqds
   We supposed that the installation of the application is in /opt/jtreqs. You
   will need to modify this file, in order to define the JAVA_HOME, the home
   directory for the application, and other parameters.

   The application's monitoring is done with a watchdog, and you can add it in
   a cron. The file checks if the applications is running well, its name is
   watchdog.sh and it can be found in the bin directory. You will need to
   modify this file to configure the database access.
   There is a file called jtreqs.cron that show how the cron can be configured.

** Application

 - Modify the file
     jtreqs/etc/jtreqs.conf.properties
     (or bin/jtreqs.conf.properties without maven)
   in order to connect to the database and connect to HPSS with a good set of
   credentials.
 - Setup the logger configuration in the file
     jtreqs/etc/logback.xml
     (or bin/logback.xml without maven)
   if neccessary.


* Execution

 - Start jtreqs.

   (When using Maven)
   Once the file has been inflated, then execute these commands from the root
   dir of the installation:
   The install directory could be something like /opt/jtreqs
     $ export HPSS_ROOT=${HPSS_ROOT:-/opt/hpss}
     $ export LD_LIBRARY_PATH=${HPSS_ROOT}/lib
     $ sh jtreqs/bin/jtreqs.sh

   (When not using Maven)
     $ export HPSS_ROOT=${HPSS_ROOT:-/opt/hpss}
     $ export LD_LIBRARY_PATH=`pwd`/bin:${HPSS_ROOT}/lib
     $ cd bin
     $ sh ./jtreqs.sh

   However, if the cron is set, then it will start the application
   automatically.

* Usage

 - With the treqs-client available at: http://git.in2p3.fr/public/treqs-client.


For test in development:

 - Many tests use Assertion, thus, you have to activate them by -ea in the VM
 arguments.
 - When running in Eclipse, it is necessary to indicate that the Working
 directory is the bin directory, or where the .class files are ggenerated.
 - Many tests use some configuration files and scripts. For this reason it is
 necessary to add the java > src > test > config and script directories to the
 path.