Spike Sorting Evaluation

Welcome to the Spike Sorting Project - Algorithm Evaluation

Extracellular recordings are a standard procedure to analyzing the activity of neurons. A problem with this kind of recording is the simultaneous recording of not a single neuron but a small local population of neurons. It is not straight-forward to reconstruct the single neuronal activities. To estimate the single neuron activity (the so called spike train) from this multi neuron activity, spike sorting is applied. Many different algorithms for spike sorting were proposed. However, despite many efforts to tackle this problem, it is still difficult to tell under which circumstances which spike sorting algorithm is the best. For a review please see

Einevoll GT, Franke F, Hagen E, Pouzat C, Harris KD (2011) Towards reliable spike-train recordings from thousands of neurons with multielectrodes. Curr Opin Neurobiol. 2011 Oct 22. [Epub ahead of print]

In May 2011, a workshop on Validation of Automatic Spike Sorting Methods took place in Ski, Norway.

The Problem - No Benchmark

In our oppinion the lack of systematic evaluations of different spike sorting algorithms derives from the fact that there is no common benchmark dataset used in the spike sorting literature, rendering comparisons of the results of different publications virtually impossible.
However, a good benchmark dataset is difficult to establish - since it is currently impossible to record the true activity of a larger number of neurons simultaneously with extracellular recordings.

Aim of this project

We want to establish a platform for spike sorting with two different goals: First, we want to provide a sophisticated benchmark dataset which could be used for future publications on this topic. Second, we want to give researchers using spike sorting software the possibility to evaluate their software under different conditions and compare their results to other spike sorters.

How this website works

This website provides extracellular data of various types for download. Common to this data is, that we know the ground truth i.e. we know how many neurons are present in the data and when which neuron fired an action potential. The datasets vary in their underlying questions.

If you have decided which dataset you are interested in, go to the download page and download the datafiles in the format which is most convinient to you.

 You can now apply the spike sorter of your choice to the dataset. Keep in mind that we do not publish the ground-truth for the datafiles.

The result of your sorting can then be uploaded via the website. This data will be treated confidential, we will not publish it, unless you say so. You will retrieve a unique key for your upload.

 The website now automatically calculates an evaluation of your sorting using the hidden ground truth information. This may take a while. Once finished you can access the results under the MyResults page using the key provided under 3.

 If you are comfortable with the results you can help us by allowing us to publish your results on the Summary Page for this dataset. You will have to provide some additional information about the algorithm you used. Furthermore, only complete results (for all files of the benchmark dataset) can be published. Published results can be seen by everybody and are compared to the published results of other users.

What this website is not

This website does not provide spike sorting or code for spike sorting implementations in any way.