I have a very huge set of data which needs to be imported from different files and need to plot in one figure(at most 5 plots). Right now I am doing this in a for loop which is really slowing up the process. My idea is to make two matrices one contains 'x' values and another with 'y' values and finally plot. The data size is not the same. What is the best way to achieve this? Can this be done using arrayfun or vectorization?
Related
So I am plotting two years worth of data to see the change in distribution of values, but with one of the histograms, one of the years is heavily dependent upon the first column, this is because there are many zeros in the data set. Would you recommend creating a bar strictly for zeros? How would I create this index in matlabd? Or how can I better manipulate the histogram to reflect the actual data set and make it clear that zeros are accounting for the sharp initial rise?
Thanks.
This is more of a statistical question. I f you have good reason to ignore the zeros, for example one of your data aquisition system produced them because of malfunction. you can get simply get rid of them by
hist(data(data~=0))
but you would not need to look at the histograms anyways you can use the variance or even standard deviation to see how much your data shifted.
Furthermore to compare data populations boxplots are much better and easier to handle.
doc boxplot
If on the other hand your zeros are genuine to your data you have to keep them! I am sorry but also here the boxplot function might help you because the zeros might be outliers (shown as little red crosses) or the box is just starting at the zero line.
I am searching for a method to eliminate freak values out of given dataset. For example:
All these peaks should be eliminated. I've tried different filters like medfilt, but the peaks are still there. I've also tried a lowpass filter, but it didn't work either. I am a beginner in filtering signals, so I probably did it wrong.
You can download data sets for the x array here and y array here.
I could also imagine a loop to compare the values next to each other, but I am sure there has to be a built-in function?
Here is the result using medfilt1(input,15):
The peaks are vanishing, but the then I get these ugly steps, which I don't want.
just use median filter! medfilt1(data,3) will do if this is a 1 pixel "cosmic" spike. If the peaks remain, increase the window size to 5 or more...
EDIT:
so this is how op's data looks like:
So we see that the data is not exactly uniform or ordered, and there are a lot of data points in the spikes unlike what one first understand from the question (guys please plot your data correctly!) The question is now, is the data in the spikes or on in the baseline?
I have a huge ~9Gb .bin file.
Reading data with fread(), getting 2D array A ~ 10^9 points.
Trying to display with imagesc() as simple as:
figure(1)
imagesc(x,y,A)
It takes ~ 800 seconds for me and I can't see anything on the figure.
I am sure that I read the file right. Checked with smaller ones.
So I wonder is there a way to display such a huge data with less effort for my PC?
Perhaps use some kind of downsampling on A. To do it right you'd have to apply a low-pass filter followed by decimation, but the low-pass filter may take very long in your case. So, even if it's subject to possible aliasing, you can try to just take a sample out of n and plot that:
n = 10; %// choose as suits you best
imagesc(x(1:n:end), y(1:n:end), A(1:n:end,1:n:end))
It is quite hard to answer your question without knowing the nature of the data.
Here are some ideas:
If your data is an image, you should downscale it using on of the known methods, or crop it.
If you know that your data is smooth, you can sample it without introducing aliasing.
Show some kind of statistics on your data, instead of showing the data itself.
How to save a cell matrix of size 500K x 110 in Matlab in an efficient and quickest way (if possible).
It has a combination of strings and numerical data.
Traditional Matlab "save" method is failing most likely because the matrix is too big.
I tried hdf5write but this does not work when the data types are different within the same matrix data.
I also tried the suggestion here but this method slows over time
Cannot save really big matrix in Matlab
I also tried these but over time they slow too
http://www.mathworks.co.uk/matlabcentral/fileexchange/24483-fast-function-to-save-a-matrix
I'm currently trying to solve this; I have an idea of how to do this, but I would really appreciate any extra input I could get.
I need to come up with a Binning function that takes in raw thickness data from a trace file, that is read in by another function. I need to put this data in 5mm bins. The tricky part will be to smooth fit the curves at both ends, and also perform data validation of the input data.
In summary: I need to take data values and "bin"/average them in 5mm intervals.
Thanks guys.
It sounds like you want the hist() function, which creates a histogram by binning the input data.