Library for optimal compression using entropy/arithmetic/range coding.

While a binary coder is simpler and allows more optimization, it can only encode a binary choice at a time. At best it will produce one bit at a time.

A 64 bit coder is slower to encode a symbol (though this implementation aims to be fast). Yet it can handle a very large alphabet, and if the model permits, encode up to 63 bits at a time.

If we encode symbols with probability $0.5$ then the outcome should be exactly the bits we feed in plus a final one.

If we encode a stream of $n$ identical symbols with probability $0$ or $1$ then the outcome should be the $n$th finitely odd number. This effectively encodes only the length of the stream.

If we encode symbols with probabilities that are multiples of $2⁻ⁿ$ for some $n$ then we output should be equivalent to that of some Huffman code, followed by a trailing $1$.

Configuring your system:

apt-get install make git clang libunittest++-dev

Make sure clang supports -std=c++14. The following should not produce errors:

echo "int main(){return 0;}" | clang++ -std=c++14 -x c++ - -o /dev/null

Obtaining the project source code:

git clone git@github.com:Recmo/EntropyCoder.git
cd EntropyCoder

Building the project source code:

make build

Testing the project build:

make run-tests

Submitting changes:

To be written

• Data Compression Explained (Matt Mahoney)

cd EntropyCoder
docker run -t -i -v (pwd)/.git:/git:ro ubuntu:wily /bin/bash

Then in the docker instance:

apt-get update
apt-get install -y git make clang pkg-config libunittest++-dev

git clone /git /build
cd /build
make