This is an implementation of the iSHAKE incremental hashing algorithm, based on the relatively new SHA3 standard and particularly on the shake extendable-output functions included in NIST FIPS 202.
This library is under development, and therefore should be considered unstable and buggy. Please do not use it unless you know what you are doing.
This project depends on the
KeccakCodePackage. We will try
to download and build it for you, optimizing it by default for generic 64-bit
platforms. If you are running on a different platform, take a look at the
different targets provided by the KeccakCodePackage Makefile.build
, and set
the KECCAK_TARGET
environment variable to the one that fits the most. Make
sure to use a target to build the library, not the KeccakSum
utility or the
tests. That means only the */libkeccak.a
targets can be used.
We use cmake
and make
to build. Just run cmake
in the root
directory of the project to generate a Makefile, and then run make
to build.
% cmake .
% make
A couple of binaries are provided when building:
-
sha3sum
is the equivalent to the UNIX utility shasum for SHA3. It allows the computations of both SHA3 and SHAKE digest, with the latter allowing extendable output. The following parameters are supported:--shake128
to use the SHAKE128 XOF.--shake256
to use the SHAKE256 XOF.--sha3-224
to use SHA3 with 224 bits of output.--sha3-256
to use SHA3 with 256 bits of output.--sha3-384
to use SHA3 with 384 bits of output.--sha3-512
to use SHA3 with 512 bits of output.--bytes
to specify the amount of bytes desired in the output (only for the two XOFs).--hex
to indicate that the input is hex-encoded.--quiet
to indicate that the output should only consist of the hash.- Additionally, a file can be specified as the source of the data to hash. Input can also be passed into the utility by using a UNIX pipe.
-
ishakesum
is the equivalent to the UNIX utility shasum for iSHAKE. It has two different variants, iSHAKE128 and iSHAKE256, both allowing extendable output. The following parameters are supported:--128
to select the 128-bit equivalent version of the algorithm. Output length defaults to 2688 bits.--256
to select the 256-bit equivalent version of the algorithm. Output lenght defaults to 6528 bits.--bits
to specify the amount of bits in the output. Must be a multiple of 64. The ranges 2688 - 4160 for 128-bit equivalent and 6528 - 16512 for 256-bit equivalent are allowed.--block-size
to specify the amount of bytes of input that should be used per block.--hex
to indicate that the input is hex-encoded.--quiet
to indicate that the output should only consist of the hash.- Additionally, a file can be specified as the source of the data to hash. Input can also be passed into the utility by using a UNIX pipe.
-
ishakesumd
is the equivalent to ishakesum for directories. It takes a directory as a parameter, and searches for files in there, applying the algorithm over each file as a different block. Files should be numbered (starting with 1) with their corresponding block number. Its parameters are the same as for ishakesum, with two main differences:--hex
is not available. Input cannot be hex-encoded, nor piped into the program.--rehash
allows recomputing the hash, based on a previous hashed passed as a parameter immediately after this option.
The library can also be used directly. Just include ishake.h
and use the
interface. Make sure to call ishake_init()
before other functions of the
interface, and ishake_final()
when you've finished feeding data into the
algorithm, in order to retrieve the final hash. ishake_hash()
acts as a
convenience wrapper around the rest of the functions.
Refer directly to the ishake.h header for details on how to use the API.
The library can be used directly by including the ishake.h
header file and
using the types and functions defined there. There are two ways to use the API:
- Call the
ishake_hash()
function if you just want to compute the digest corresponding to a given input. - Use the rest of the interface to compute the digest over dynamic data or modify an existing digest.
While the former is very simple and convenient, it is also quite limited as it will not allow you to use the incremental functionality of the iSHAKE algorithm. In order to do that, you need to use the rest of the functions in the API, and this is how you use them:
- First, you will need to allocate memory for a
ishake_t
structure that you will pass around as the first parameter to every function. Initialize the structure by calling theishake_init()
function. The structure must be initialized always. - If you have a precomputed hash that you want to recompute based on a
set of changes to the input, add that hash to the
ishake_t
structure. Bear in mind that thehash
field of theishake_t
algorithm is declared as a buffer ofuint64_t
integers, and as such, you need to prepare the hash before. If your hash is in hexadecimal form, you will need to convert it to binary form first (use thehex2bin()
function in definedutils.h
), and then convert the result touint64_t *
(again, a helper function calleduint8_t2uint64_t()
is provided in theutils.h
header). - Depending on the mode you use, you will be able to perform some operations or not. If you are using the APPEND_ONLY mode, you can only append new data or modify new blocks. On the other hand, the FULL_RW mode will allow you to insert, update and delete blocks at any given position.
- In order to use most functions, you will need to pass a block (or several) containing the data to be hashed, plus a header describing the position of the block. In APPEND_ONLY mode, the header is a simple 64-bit incremental index, while in FULL_RW mode the header consist of a random, unique, 64-bit nonce identifying the block, plus the nonce of the next block in the chain.
- Once you are done modifying the input by appending, updating, inserting or
deleting blocks, you need to call the
ishake_final()
function to obtain the final digest corresponding to the input given. - After obtaining the final digest, remember to call the
ishake_cleanup()
function to ensure that all internal structures and memory used by iSHAKE are freed.
-
ishake_nonce
represents the header of an iSHAKE block in FULL_RW mode. It contains the nonce for a given block, plus the nonce of the next block in the chain, as 64-bit unsigned integers. -
ishake_header
represents the header of an iSHAKE block, where that could be a 64-bit unsigned integer index in case of APPEND_ONLY mode, or anishake_nonce
structure in FULL_RW mode. -
ishake_block_t
represents an iSHAKE block, containing the input data plus the correspondingishake_header
structure. -
ishake_t
is the internal structure used by iSHAKE to pass state information around to the different functions of the API.
Here is a more detailed description of the functions provided by the iSHAKE API:
-
ishake_init()
: initializes anishake_t
structure. Accepts as parameters the block size to use, the length in bits of the resulting hash, the mode of operation and the number of threads to use. -
ishake_append()
: appends data to the existing input. It will split the data in chunks of the size of an iSHAKE block automatically, and keep the excess until more data arrives and can be appended to form a new block. It accepts as parameters the data to append and its length. -
ishake_insert()
: inserts an iSHAKE block at a given position, right after another block given. It accepts anishake_block_t
structure with the block after which the data must be inserted, and anotherishake_block_t
structure containing the data to insert itself. Note that it is your responsibility to build the inserted block correctly. This means if there was a block already after the previous block (that is, we are not appending the block), then thenext
pointer of the inserted block must be the same as thenext
pointer of the previous block. You don't need to modify the previous block yourself, iSHAKE will take care of that for you though. Note also that if you want to insert a block at the beginning of the chain, you then need to passNULL
as the previous block. -
ishake_delete()
: deletes an iSHAKE block at a given position, right after another block given. The behaviour of this function is analogous to theishake_insert()
function. -
ishake_update()
: updates the contents of a given iSHAKE block and its corresponding hash. Its parameters are the original block and the modified one, where the difference between both must be only regarding the data. The function will not prevent you from updating a block by changing its headers too, but this only makes sense under certain circumstances (like inserting or deleting blocks) and you should not do that unless you know what you are doing. -
ishake_final()
: computes the final digest corresponding to the input given. It needs as a parameter a buffer ofuint8_t
integers previously allocated to store as many bits as specified when initializing the algorithm when callingishake_init()
. -
ishake_cleanup()
: cleans and frees all internal structures used by the algorithm. Make sure to call this function always. -
ishake_hash()
: computes the iSHAKE hash corresponding a given input. As parameters, it needs a string with the data to hash plus its length, a buffer ofuint8_t
integers where to store the result and its corresponding length in bits.
iSHAKE allows you to process the blocks in parallel to boost performance. This
allows you to leverage the computing power of CPUs with multiple cores and
pipelines to reduce the total processing time to the minimum possible. In
order to use this feature, just pass a positive number of threads to use to the
ishake_init()
function when initializing the ishake_t
structure, and
iSHAKE will take care of the rest for you.
There is no magic bullet to determine what amount of threads is best for your setup. Check the amount of cores you have available and test different configurations in order to find out the optimal number of threads to use.
iSHAKE was proposed by Hristina Mihajloska, Danilo Gligoroski and Simona Samardjiska, and implemented by Jaime Pérez Crespo.
The tiny SHA3 implementation used in the library by David Leon Gil, licensed under CC0.