OLVER Core test suite performs functional and conformance testing of
programming interfaces of the main Linux system libraries against the
LSB Core requirements.  The tests are generated automatically from
test scenarios and formal specifications of the requirements.

This readme file describes how to build and execute the tests.

If you have just installed binary package of the test suite you may
skip "Building the Test Suite" section of this readme.

All commands below are relative to root dir of the Test Suite.
Default Root Directory for packages is /opt/lsb/test/olver-core.

Contents
---------------------------------------------------------------------
1. Software and Hardware Requirements
2. Building the Test Suite
3. Running the Test Suite
4. Report Generation
5. Viewing Test Results
6. Troubleshooting

---------------------------------------------------------------------
1. Software and Hardware Requirements
---------------------------------------------------------------------
Currently all seven LSB archtectures (IA32, IA64, x86_64, s390, s390x, PPC64,
PPC32) are supported.

We've checked our test suite on the following Linux distributions and
architectures:

    Debian GNU/Linux 4.0 (etch) on IA32, IA64.
    Debian GNU/Linux 5.0.3 (lenny) on x86_64.
    Moblin release 2.0 on IA32.
    Moblin release 2.1 on IA32.
    openSuSE 11.2 on x86_64.
    RedHat Enterprise Linux 4 Update 2 on IA32 и x86_64.
    Red Hat Enterprise Linux Server release 5.4 (Tikanga) on x86_64.
    SuSE Linux Enterprise Server 10.0 on IA32, IA64, PPC32, PPC64, s390, s390x.
    SuSE Linux Enterprise Server 11.0 on IA32, x86_64.
    Ubuntu 9.04 on IA32.
    Ubuntu 9.10 on IA32.
    Fedora release 8 (Werewolf) on IA32.
    Fedora release 10 (Cambridge) on x86_64.
    Fedora release 12 (Constantine) on IA32.

All failures found on these systems have been documented and published.

For building and running the test suite additional software requirements should
be met listed in "Building the Test Suite" and "Running the Test Suite" sections.

---------------------------------------------------------------------
2. Building the Test Suite
---------------------------------------------------------------------
Prerequisites

To build OLVER one need to have the following components installed in the
system:

1. LSB 4.1 Software Development Kit (SDK). You may find it on
       http://www.linuxfoundation.org/collaborate/workgroups/lsb/download
2. GNU C/C++ Compiler 3.3.5 or later
3. Java Runtime Environment version 5 or later (Sun or IBM preferred). 
   Note that the 'java' command should be in your path.
4. autoconf
5. automake
6. libtool

You should prepare Test Suite to build by running configure script. You
will be asked about tester account preferred and the architecture you
want to test against.

	./configure.sh

In order to build the test suite run script under user choosen:

    ./build_conf_tests.sh

Note: during the build you'll be prompted to enter the root password.
This is needed to set appropriate privileges for the agent in order
to test ncurses and process functionality. If you have already got root
privileges password prompt will be answered automatically.

It takes approximately 25 minutes to complete the build on Athlon XP 2700+.

After successful build you should prepare additional data for test
scenarios. Run:

	./install.sh

Additional cleaning could be performed. Run script:
	
	./clean_tests.sh

---------------------------------------------------------------------
3. Running the Test Suite
---------------------------------------------------------------------
Prerequisites

To run OLVER and obtain execution reports one need to have
the following components installed in the system:

1. LSB runtime loader: /lib/ld-lsb.so.3. If it is not found, you should
   install LSB 4.1 Software Development Kit (SDK). You may find it on
       http://www.linuxfoundation.org/collaborate/workgroups/lsb/download
2. Perl language interpreter.

In order to generate reports you should additionally supply either

3a. Java Runtime Environment version 5 or later (IBM or Sun preferred)
   and the 'java' command needs to be in your path.

(preferred) or 

3b. any fast XML::SAX based Perl module


In order to execute the test suite run script:

    ./bin/olver_run_tests [ testplan [ timeout ] ]

	tesplan is a list of test scenarios to be run.
	./bin/testplan is used by default.
	To decide which scenarios must not be included in the test run, please
	open testplan and comment out lines with scenario names.

		#str_scenario           (will not be executed)
		integer_scenario        (will be executed)

	timeout is a scenario max lifetime. 30 minites by default.

You could choose you own directory to store the results: 

	export OLVER_REPORT_DIR=...

By default it is /var/opt/lsb/test/olver-core/<timestamp>


---------------------------------------------------------------------
4. Report Generation
---------------------------------------------------------------------
You could generate reports separately by runnung:

	./bin/olver_report

You should point out the directory with test traces as:

	export REPORT_DIR=...

You also can choose XML::SAX based parser you want.

	export OLVER_XML_PARSER=<XML::SAX based perl module>

---------------------------------------------------------------------
5. Viewing Test Results
---------------------------------------------------------------------
Both raw traces and reports generated are stored in 
/var/opt/lsb/test/olver-core/<timestamp> directory.
You can use any web browser you like to see following reports:

Test Execution Summary Reports:
    1. In OLVER format: /var/opt/lsb/test/olver-core/<timestamp>/summary.htm
    2. In TET format: /var/opt/lsb/test/olver-core/<timestamp>/nice_tet.log

Test Execution Report:
    /var/opt/lsb/test/olver-core/<timestamp>/report/index.html.

Requirements Coverage Report:
    /var/opt/lsb/test/olver-core/<timestamp>/result.htm.

For detailed information about test reports see
http://linuxtesting.org/downloads/olver-reports.pdf

---------------------------------------------------------------------
6. Troubleshooting
---------------------------------------------------------------------
1. SuSE. Fail to build the agent. Add symbolic links:
    a) libz
        su
        cd /lib
        ln -s libz.so.1 libz.so
    b) libpam
        su
        cd /lib
        ln -s libpam.so.0 libpam.so
    c) libncurses
        su
        cd /lib
        ln -s libncurses.so.5 libncurses.so

2. SuSE. Can not build olverterm because can not find
   tic (terminfo compiler). Install ncurses-devel package.

3. Mandriva. Fail to build the agent. Add symbolic links:
    a) libpam
        su
        cd /lib
        ln -s libpam.so.0 libpam.so

4. If you experience problems with building model of the test suite
   make sure that your java environment is IBM or Sun Java Runtime Environment 
   version 5 or later. OpenJDK 1.6 seems to work.

5. On some systems a problem may occur with agent build - the clog10() 
   function is incorrectly defined as
	extern double complex clog10(void);
   in LSB headers on these systems. This is a problem with older versions of
   the LSB Software Development Kit (SDK). Correct by installing a current
   version or manually correct the above definition in 
   /opt/lsb/include/complex.h to:
	extern double complex clog10(double complex)