I have some data which when plotted looks like the one shown in left on the picture attached.
It has some kinks which I wish to delete and smooth-en to get a nice curve [shown on right].
Presently I manually delete the kinks and interpolate the deleted part by polynomial of high order [say 9]. Then I repopulate the deleted fragment and re-draw the curve.
This takes a long time and I have quite a number of files to process.
Could you folks suggest an efficient way to do this ?
[in MATLAB or some other way]
Thanks a lot !!
P.S.: Added one more plot for clarification above
I think this is a simple low pass filtering problem like #thewaywealk suggests. Removing the kinks corresponds removing certain high frequencies in your signal. This can be achieved in matlab by the filter operation. Demonstrations are shown here on denoising a sinusoidal wave here.
Related
I have certain movement data acquired from motion capture system which I want to automatically choose which 5 signals are more alike.
Picture shows example of the particular data, all normalized to 100 samples due to the difference in speed.
Data set for knee flexion/extension
What I am looking for is some idea to actually compare the shapes of the curves.
The easiest solution is just to substract the "raw" curves and check which one has the smallest RMSE.
But looking at your data (which are smooth curves), another option is to use PLS or GMM to describe them. Then you can use RMSE to compute the error between your input curve and your database of curves and take the one with lowest error.
I am generation some data whose plots are as shown below
In all the plots i get some outliers at the beginning and at the end. Currently i am truncating the first and the last 10 values. Is there a better way to handle this?
I am basically trying to automatically identify the two points shown below.
This is a fairly general problem with lots of approaches, usually you will use some a priori knowledge of the underlying system to make it tractable.
So for instance if you expect to see the pattern above - a fast drop, a linear section (up or down) and a fast rise - you could try taking the derivative of the curve and looking for large values and/or sign reversals. Perhaps it would help to bin the data first.
If your pattern is not so easy to define but you are expecting a linear trend you might fit the data to an appropriate class of curve using fit and then detect outliers as those whose error from the fit exceeds a given threshold.
In either case you still have to choose thresholds - mean, variance and higher order moments can help here but you would probably have to analyse existing data (your training set) to determine the values empirically.
And perhaps, after all that, as Shai points out, you may find that lopping off the first and last ten points gives the best results for the time you spent (cf. Pareto principle).
I have a set of 3D points which numbers up around 1 million points. I am looking to visualise these with matlab.
I have tried the following functions:
plot3
scatter3
But they are both very sluggish. Is there a more efficient way to visualise this level of points in matlab? Maybe a way to mesh the points?
If not can anyone suggest a plug-in or even a different program for visualising 3D points?
You're going to run into efficiency issues no matter what plugin/program you use if you want all million+ points to show up in a plot. My suggestion would be to downsample. Use the plot3 or scatter3 function on every other point, or every nth point, until you get a figure that is not sluggish. As long as the variance in your data isn't astronomical, downsampling a little bit shouldn't affect the overall distribution of points (given that you have a million+ points). And any software that is able to display that much data without being sluggish is most likely downsampling/binning or using some interpolation technique to do so (so you might as well have control over it).
fscatter3 from the file exchange, does what you like.
Is there a specific reason to actually have it display that many points?
I Googled around a bit and found some people who have had similar issues (someone suggested Avizo as an alternate program but I've never used it):
http://www.mathworks.com/matlabcentral/newsreader/view_thread/308948
mathworks.com/matlabcentral/newsreader/view_thread/134022 (not clickable because I don't have enough rep to post more than two links)
An alternate solution would be to generate a histogram if you're more interested in the density of the data:
http://blogs.mathworks.com/videos/2010/01/22/advanced-making-a-2d-or-3d-histogram-to-visualize-data-density/
I you know beforehand roughly the coordinates of the feature you are looking for, try passing the cloud through a simple pass-through-filter, which essentially crops your point cloud. I.e. if you know that the feature is at a x-coordinate > 5, remove all points with x-coordinate < 5.
This filter could for the first coordinated be realized as
data = data(data(1,:) > 5,:);
Provided that your 3d data is stored in an n by 3 matrix.
This, together with downsampling, could help you out. If you still find the performance lagging, consider using something like the PCD viewer in PointCloudLibrary, check half way down the page at
http://pointclouds.org/documentation/overview/visualization.php
It is a stand alone app you could launch from matlab. I find it's performance far better than the sluggish matlab plotting tools.
For anyone who is interested I ended up finding a Point cloud visualiser called Cloud Compare. It is extremely fast and allows selection and segmentation as well as filtering on point clouds.
I'm fairly new to MATLAB, but have acquainted myself with Simulink and Computer Vision over the past few days. My problem statement involves taking a traffic/highway video input and detecting if an accident has occurred.
I plan to do this by extracting the values of centroid to plot trajectory, velocity difference (between frames) and distance between two vehicles. I can successfully track the centroids, and aim to derive the rest of the features.
What I don't know is how to map these to ANN. I mean, every image has more than one vehicle blobs, which means, there are multiple centroids in a single frame/image. So, how does NN act on multiple inputs (the extracted features per vehicle) simultaneously? I am obviously missing the link. Help me figure it out please.
Also, am I looking at time series data?
I am not exactly sure about your question. The problem can be both time series data and not. You might be able to transform the time series version of the problem, such that it can be solved using ANN, but it is sort of a Maslow's hammer :). Also, Could you rephrase the problem.
As you said, you could give it features from two or three frames and then use the classifier to detect accident or not, but it might be difficult to train such a classifier. The problem is really difficult and the so you might need tons of training samples to get it right, esp really good negative samples (for examples cars travelling close to each other) etc.
There are multiple ways you can try to solve this problem of accident detection. For example : Build a classifier (ANN/SVM etc) to detect accidents without time series data. In which case your input would be accident images and non accident images or some sort of positive and negative samples for training and later images for test. In this specific case, you are not looking at the time series data. But here you might need lots of features to detect the same (this in some sense a single frame version of the problem).
The second method would be to use time series data, in which case you will have to detect the features, track the features (say using Lucas Kanade/Horn and Schunck) and then use the information about velocity and centroid to detect the accident. You might even be able to formulate it for HMMs.
i need to smooth better this kind of plot, I've already used a moving average (10 points) to get this plot but it's not yet perfect. I want to remove all these little peaks dued by noise, I need to consider only the bigger ones because I'm counting the num of beats from a sensor.
(ie.: in the first 30 seconds I should have just one peak instead of several successive little peaks)
I thought to use a cubic spline but isn't simple to implement in C and it's going to take almost 1-2 weeks of work.
Is there a simpler method / algorithm to use for this achievement? I'm working on this project for iOS (iPhone) environment.
a busy cat http://img15.imageshack.us/img15/1929/schermata022455973alle1o.png
The answer to your question depends a lot on the underlying data. Is the jaggedness of the data really 'noise' or is it really jagged data.
Strategies you could try:
windowing the data and take the median/mean in each window -- so each window is 50 (from your x axis)
sample the data
Nonlinear least squares curve fit (you'd probably have to use a C++ library for that, here is an open source version you could port http://www.ics.forth.gr/~lourakis/levmar/)
some sort of naive bezier smoothing should be pretty easy.
All of these methods have ramifications and none are without problems. Good luck.