I´m trying to find a global measure of similarity for spikes trains over time. The signals look like in the picture (in this example I have 17 neurons).
Can I use windowed cross correlations? If yes, what should I do with the output matrices? I use MATLAB by the way.
Yes, you can use binned spikes or binned bursts to compute the cross correlations. You can then threshold the output matrix to see what electrodes have the most similarity.
If you have the spikes in text format or have exported your spike train to an HDF5 file from MC_Rack, you can use MEAnalyzer to compute and export all the comparisons: https://github.com/RDastgh1/MEAnalyzer (but please cite it if you use it).
Related
I have a lot of mixed files, and from those files I would like to keep those that correspond to EEG recordings.
The problem is that some EEG files are supposed to be recorded with 2, 4, 5 and 6 channels, and that there also is other data (of unknown type) that also have the same amount of channels (I already filtered those that don't have these amount of channels). And of course, there are thousands of files, so manual inspection isn't really an option.
So, is there some kind of metric or algorithm that helps me to differentiate non-EEG signal from EEG ones? With MATLAB if possible.
You might first try comparing statistics such as mean, std and kurtosis of each channel relative to known good channels. Since you have lots of files, you would probably want to the sample possible matches in a few places (e.g., compare seconds 1-3 and say 7-9) and then get a probability of match. Interesting question!
This is not really a MATLAB question, but...
What is (potentially) in the other recordings? Perhaps it is easier to filter them out than it is to identify EEG specifically.
I don't know of any metric or algorithm that can identify EEG recordings. One problem is that EEG recordings often contain artefacts to varying degrees such as muscle activity, line noise, and any other electromagnetic interference in the room that make the recording nonspecific to cortical activity.
The frequency spectrum may be one meaningful measure. Cortical activity usually tapers off in power towards, say, 40 Hz, and tends to have some peaks before. For example, depending on the placement of EEG electrodes and the task that was being performed, a peak in the alpha band (around 10 Hz) may be prominent. This is assuming there is little interference from artefacts.
Also the amplitude of the signal may be something to look at.
Perhaps you could take a number of measures and statistical properties (e.g. powers in different frequency bands, variance, drift/slope, amplitude, etc.), cluster them using e.g. t-SNE, and, assuming you get clearly separable clusters, identify a few samples from each cluster by hand to figure out which is the EEG cluster.
Review for removing periodicsI have a dataset that contains hourly wind speed data for 7 seven. I am trying to implement a forecasting model to the data and the review paper states that trimming of diurnal, weekly, monthly, and annual patterns in data significantly enhances estimation accuracy. They then follow along by using the fourier series to remove the periodic components as seen in the image. Any ideas on how i model this in matlab?
I am afraid this topic is not explained "urgently". What you need is a filter for the respective frequencies and a certain number of their harmonics. You can implement such a filter with an fft or directly with a IIR/FIR-formula.
FFT is faster than a IIR/FIR-implementation, but requires some care with respect to window function. Even if you do a "continuous" DFT, you will have a window function (like exponential or gaussian). The window function determines the bandwidth. The wider the window, the smaller the bandwidth. With an IIR/FIR-filter the bandwidth is encoded in the recursive parameters.
For suppressing single frequencies (like the 24hr weather signal) you need a notch-filter. This also requires you to specify a bandwidth, as you can see in the linked article. The smaller the bandwidth, the longer it will take (in time) until the filter has evolved to the frequency to suppress it. If you want the filter to recognize the amplitude of the 24hr-signal fast, then you need a wider bandwidth. But then however you are going to suppress also more frequencies slightly lower and slightly higher than 1/24hrs. It's a tradeoff.
If you also want to suppress several harmonics (like described in the paper) you have to combine several notch-filters in series. If you want to do it with FFT, you have to model the desired transfer function in the frequency space and since you can do this for all frequencies at once, so it's more efficient.
An easy but approximate way to get something similar to a notch-filter including all harmonics is with a Comb-filter. But it's an approximation, you have no control over the details of the transfer function. You could do that in Matlab by adding to the original a signal that is shifted by 12hrs. This is because a sinusoidal signal will cancel with one that is shifted by pi.
So you see, there's lots of possibilities for what you want.
I'm a neuroscientist, and not a very good one. My colleague has kindly provided me with a noisy voltage measurements of the PY neuron of the Stomatogastric Ganglion of the lobster.
The activity of this neuron is characterised by a slow depolarised plateaux with fast spikes on top (a burst).
Both idealised and noisy versions are presented here for you to peruse at your leisure.
It's my job to extract the spike times from the noisy signal but this is so far beyond my experience level I have no idea where to begin. Fortunately, I am a total ninja at Matlab.
Could someone kindly provide me with the name of the procedure, filter or smoothing function which is best suited for this task. Or even the appropriate forum to ask such an asinine question.
Presumably, it needs to increase the signal to noise ratio? The problem here seems to be determining the difference between noise and a bona fide spike as the margin between the two is quite small.
UPDATE: 02/07/2013
I have tried the following filters in Matlab with mixed results. It's still very hard to say what is noise and what is a spike.
Lowpass Butterworth filter,
median filter,
gaussian,
moving weighted window,
moving average filter,
smooth,
sgolay filter.
This may not be an adequate response for stackoverflow - but one way of increasing a signal to noise ratio in your case is to average parts of the signal.
low pass your signal to remove noise (and spikes), and find the minima of the filtered signal (from your image, one minimum every 600 data points). Keep the indexes of each minimum,
on the noisy signal, for each minimum index, select the consecutive 700 data points. If you have 50 minima, you should have a 50 by 700 matrix,
average your matrix. You should have a 1 by 700 vector.
By averaging parts of the signal (minimum-locked potentials), you will take advantage of two properties: noise is zero-mean (well, it should be), and the signal of interest is repetitive. The first will therefore decrease as you pile up potentials, and the second will increase. With this process however, you will lose the spike times for each slow wave figure, but at least have them for blocks of 50 minima.
This technique is known in neuroscience as event-related potential (http://en.wikipedia.org/wiki/Event-related_potential). It may not fit perfectly your signal, or the result may not give nice spikes, but you may extract the spike times for some periods of interest (given the nature of your signal, I would say that you would need 5 or 10 potentials to see an emerging mean activity).
There are some toolboxes that do part of the job (but I would program it myself given the complexity of the task). These are eeglab or fieldtrip. They have a bunch of filter/decomposition options too, as well as some statistical features.
I have some dynamic light scattering data. The machine pumps out the autocorrelation function, and a count-rate.
I can do a simple fit to the ACF
ACF = exp(-D*q^2*t)
and obtain the diffusion coefficient.
I want to obtain the same D from the power spectrum. I have been able to create a power spectrum in two ways -- from the Fourier transform of the ACF, and from the count rate. Both agree, but the power spectrum does not look like in the one in the books, so I'm not sure how to use it to work out the line width.
Attached is an image from a PDF that shows what you should get, and what I get from MATLAB. Can anyone make sense of whats going on?
I have used the code of answer #3 on this question. The resulting autocorrelation comes out exactly the same as
the machine gives me and
using MATLAB's autocorr command on the photoncount data.
Thank you for your time.
When you compute the Fourier transform from short sequences of data it often looks very noisy. There are a number of reasons for this. One reason is that the statistics of individual Fourier components are not Gaussian, and so averaging the spectra across multiple samples of data will only slowly improve the quality of the estimate.
Another causes of "noisiness" in empirical spectra behavior is that you are applying (to a finite data sample) a transform which involves a pathological sinc function and which assumes an infinite length signal. To diminish this problem, it helps to apply a "windowing-function" to your data before computing the Fourier transform. One of the more complicated but also more powerful windowing approaches is the use of so-called 'Slepian tapers'.
MATLAB conveniently implements well-known windows in functions such as hamming and hann.
I am trying to plot a 2 GB matrix using MATLAB hist on a computer with 4 GB RAM. The operation is taking hours. Are there ways to increase the performance of the computation, by pre-sorting the data, pre-determining bin sizes, breaking the data into smaller groups, deleting the raw data as the data is added to bins, etc?
Also, after the data is plotted, I need to adjust the binning to ensure the curve is smooth. This requires starting over and re-binning the raw data. I assume the strategy involving the least computation would be to first bin the data using very small bins and then manipulate the bin size of the output, rather than re-binning the raw data. What is the best way to adjust bin sizes post-binning (assuming the bin sizes can only grow and not shrink)?
I don't like answers to StackOverflow Questions of the form "well even though you asked how to do X, you don't really want to do X, you really want to do Y, so here's a solution to Y"
But that's what i am going to do here. I think such an answer is justified in this rare instance becuase the answer below is in accord with sound practices in statistical analysis and because it avoids the current problem in front of you which is crunching 4 GB of datda.
If you want to represent the distribution of a population using a non-parametric density estimator, and you wwish to avoid poor computational performance, a kernel density estimator (KDE) will do the job far better than a histogram.
To begin with, there's a clear preference for KDEs versus histograms among the majority of academic and practicing statisticians. Among the numerous texts on this topic, ne that i think is particularly good is An introduction to kernel density estimation )
Reasons why KDE is preferred to histogram
the shape of a histogram is strongly influenced by the choice of
total number of bins; yet there is no authoritative technique for
calculating or even estimating a suitable value. (Any doubts about this, just plot a histogram from some data, then watch the entire shape of the histogram change as you adjust the number of bins.)
the shape of the histogram is strongly influenced by the choice of
location of the bin edges.
a histogram gives a density estimate that is not smooth.
KDE eliminates completely histogram properties 2 and 3. Although KDE doesn't produce a density estimate with discrete bins, an analogous parameter, "bandwidth" must still be supplied.
To calculate and plot a KDE, you need to pass in two parameter values along with your data:
kernel function: the most common options (all available in the MATLAB kde function) are: uniform, triangular, biweight, triweight, Epanechnikov, and normal. Among these, gaussian (normal) is probably most often used.
bandwith: the choice of value for bandwith will almost certainly have a huge effect on the quality of your KDE. Therefore, sophisticated computation platforms like MATLAB, R, etc. include utility functions (e.g., rusk function or MISE) to estimate bandwith given oother parameters.
KDE in MATLAB
kde.m is the function in MATLAB that implementes KDE:
[h, fhat, xgrid] = kde(x, 401);
Notice that bandwith and kernel are not supplied when calling kde.m. For bandwitdh: kde.m wraps a function for bandwidth selection; and for the kernel function, gaussian is used.
But will using KDE in place of a histogram solve or substantially eliminate the very slow performance given your 2 GB dataset?
It certainly should.
In your Question, you stated that the lagging performance occurred during plotting. A KDE does not require mapping of thousands (missions?) of data points a symbol, color, and specific location on a canvas--instead it plots a single smooth line. And because the entire data set doesn't need to be rendered one point at a time on the canvas, they don't need to be stored (in memory!) while the plot is created and rendered.