Skip to content

azzmosphere/azzmos

Repository files navigation

AZZMOS PROJECT
==============

VERSION 0.1

Azzmos is WIP project that attempts to build a multi-threaded web crawler that can use different
index algoritms.  The aim of the project is to provide three programs on completion:

These are:
   azzmos  - web crawler.
   azidx   - indexer
   azctl   - controller function.

The program requires a running and multi-threaded version of postgresQL database.

COMPILING POSTGRESQL
====================
postgresql will need to be compiled in the following way:

./configure --enable-thread-safety
make 
make install

In the postgresql.conf file the following should be added:

client_encoding = iso-8859-1

INSTALLATION INSTRUCTIONS
=========================
Ensure that psotgresql is installed and running.

Modify the ddl/install.sh script.

Create a build directory 'mkdir build'

Change directory to build,  'cd build'

Use the following commands:

../configure
make 
make check
make install



PREREQUISITS
============
To install azzmos the following is required:
    PostgresQL compiled to run in multi-threaded mode (v 8.2.16)
    cURL development headers
    PCRE library and headers



About

general purpose web crawler for UNIX systems using PosgresQL backemd

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published