general purpose web crawler for UNIX systems using PosgresQL backemd
License
azzmosphere/azzmos
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
AZZMOS PROJECT ============== VERSION 0.1 Azzmos is WIP project that attempts to build a multi-threaded web crawler that can use different index algoritms. The aim of the project is to provide three programs on completion: These are: azzmos - web crawler. azidx - indexer azctl - controller function. The program requires a running and multi-threaded version of postgresQL database. COMPILING POSTGRESQL ==================== postgresql will need to be compiled in the following way: ./configure --enable-thread-safety make make install In the postgresql.conf file the following should be added: client_encoding = iso-8859-1 INSTALLATION INSTRUCTIONS ========================= Ensure that psotgresql is installed and running. Modify the ddl/install.sh script. Create a build directory 'mkdir build' Change directory to build, 'cd build' Use the following commands: ../configure make make check make install PREREQUISITS ============ To install azzmos the following is required: PostgresQL compiled to run in multi-threaded mode (v 8.2.16) cURL development headers PCRE library and headers
About
general purpose web crawler for UNIX systems using PosgresQL backemd
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published