forked from BodyTrack/datastore
-
Notifications
You must be signed in to change notification settings - Fork 0
TannerTaphorn/datastore
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
TODO is in TODO.txt ----- Installing on OS X: Please install boost using macports: sudo port install boost Build and install make make test make install-local Installing on Ubuntu: Please install boost using apt-get sudo apt-get install libboost-dev libboost-doc (from memory, may be incorrect) sudo apt-get install libboost-thread (from memory, may be incorrect) Build and install make make test make install-local ----------- If only one tile, pick: infinity tile until needing to split? or default level tile? problem with default level tile is that we might want to propagate it upwards should infinity tile be hacked to cover all time? go up in levels until the tile is too large separate positive and negative roots DONE add tests for lock, unlock. use boost threads BodyTrack datastore design: https://docs.google.com/document/d/1Hhpw2jGVkDOo5fZjMAAkJ0cFW_eWMrTHIjcq1UJvibg/edit?hl=en_US Dealing with variable sampling rates Approaches: put everything that fits into a bin Problems: - need to filter before display - need to filter before calculating - need to filter differently depending on display or calculating So, the DB interface: get me this range of values, with at least this resolution (but not "at most this resolution") Make a bin be ~1MB (32K samples x 32 bytes = 1MB) Lowest level tile has no summarizations time,value. 64K samples x 16 bytes = 1MB Lowest level 8bit samples tile has time, values. 1M samples x 1 bytes = 1MB 1g samples/ Building jsoncpp: cd jsoncpp-src-0.5.0/ python scons.py platform=linux-gcc check 2TB = $100 + $1/yr 1%/yr=power,for drive 1GB = $0.05, plus 24*30 = 1KWH = 0.10 1GB = ------------------------- New approach: full-res all original samples subsampling filter is a function applied -------------------- If the amount of storage required to store N samples varies by a factor of 32, how do we cope with this? 1) does subsampling necessarily inflate 1x to 32/2x = 16x? yuck or does subsampling filter and create something with even sampling? maybe float val + float stddev = 8x instead of 32x 1Gsample/day x 1B = 1GB/day = 0.05 / day 1Gsample/day x 32B = 32GB/day = 1.50 / day ------------------------- To install with optional FFT support via the NFFTv3 library, which is under the GPLv2 license: Download and install NFFT, version 3 (as of the time of this writing, NFFT 3.2.2 was the most recent version, and is also the version which was used for testing during development. You might try a newer version if it's available): $ wget http://www-user.tu-chemnitz.de/~potts/nfft/download/nfft-3.2.2.tar.gz $ tar xvzf nfft-3.2.2.tar.gz $ cd nfft-3.2.2 $ ./configure $ make $ sudo make install $ cd .. $ rm -r nfft-3.2.2 nfft-3.2.2.tar.gz This should install the NFFT library on your system. Then, to opt in to NFFT support, add FFT_SUPPORT=1 to the make command line (the Makefile compiles with FFT support whenever FFT_SUPPORT is set to a non-empty string). In other words, replace "make target" with "make target FFT_SUPPORT=1" for whichever choice of target you would have used normally. For example, instead of "make test", execute $ make test FFT_SUPPORT=1 You will know that your freshly built binaries support the Fourier transform by running $ ./gettile If the help message contains a sentence mentioning the DFT or Fourier transform, such as "If the string '.DFT' is appended to the channel name, the discrete Fourier transform of the data is returned instead," then of course your binaries have been compiled with FFT support.
About
BodyTrack datastore. C++ project for handling and serving data streams.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published