==============================================
Spike sorting for ms11d45.dat and related data
==============================================

While this tool was created for the purpose of analyzing a particular
dataset, it is part of a larger project to disseminate spike sorting
techniques developed at the Simons Center for Data Analysis.

== BASIC SETUP ==

1. Put the raw data into the raw directory.
For example: raw/ms11d45.dat and raw/ms11.prb
If you run the below processing script without the .dat file present
there will be some instructions about how to obtain this file.

2. Run do_processing.m
This will create files in output/detect
There are some configuration strings inside the script
The default options should work

3. Run do_view.m
Manipulate the script to view the results for different channels.
On each view window, there will be a button to launch spikespy.
See below for information on running spikespy.

== ADVANCED SETUP ==

To view the events/labels in the context of the raw dataset you
will want to compile "spikespy", a Qt GUI, using the instructions below.
Once compiled, you can launch the program by clicking on the button
at the bottom of one of the output windows generated by do_view.m.
There is also a command for launching spikespy directly from the MATLAB
console.

== COMPILING SPIKESPY ==

See spikespy/README.txt

You will need to install Qt4 or Qt5 development environment. This software
was developed on Ubuntu/Linux. I want it to be cross-platform, and I
especially want spikespy to run on a mac. If you have Mac OS, I will work
with you to compile and run this program.

== NOTES AND TO-DO ==

The raw data have been "pre-whitened". That's why you will notice some
of the spikes are upside-down. Pre-whitening helps a lot with detection and
sorting. If you want to view the original data after sorting, that can
be arranged.

So far, I've only implemented sorting through the clustering stage.
The remaining step is the fitting stage, where we will handle overlapping
spikes and more thoroughly and accurately detect spikes. Therefore,
the output at this stage will identify only a subset of the spikes. For
more information see the section below.

Information about the .mda file format can be found here:
http://magland.github.io//articles/mda-format/
All .mda files can be read using util/readmda.m (see also writemda.m)

Right now the sorting is done on a neighborhood-by-neighborhood basis. The
adjacency matrix is assembled in raw/adjacency.mda based on the locations
of the electrodes (raw/locations.mda). Channels are considered one at a
time and the adjacent electrode channels are included in the clustering.

We still need to consolodate spikes between these local clusterings runs
because there are certainly redundancies (firings usually affect more than
one channel).

Removal of outliers -- some clusters clearly have outliers. They should be
automatically removed.

Modeling noise for each cluster / spike type. This will help in the final
fitting stage.

Iteratively re-sort the dataset after subtracting out the fit spikes.
This will allow detection of smaller / more subtle spike types.

Move all processing / visualization to command-line. Do not require MATLAB.

Investigate alternatives to greedy fitting.

Explore impact of upsampling (for better time alignment)

Apply validation scheme, to give reliability score to individual neurons.

The firetrack software may be used to visualize the spatial layout of the
neurons. I will work toward integrating that view into this project.
See https://github.com/magland/firetrack.git

Compare the results with the results of other sorting software.

Create a more general package that may be used by additional laboratories.

For details on our methodology see the accompanying .tex/.pdf file