In MATLAB, how could I combine two matrices of data measured at different frequencies such that the result is indexed at the higher frequency? Since the data measured at the lower frequency will have many unknown values in the result, I would like to replace them with the last known value in the matrix. There is a lot of data so a vectorized solution would be preferred. I've added some sample data below.
Given:
index1 data1 index2 data2
1 2.1 2 30.5
2 3.3 6 32.0
3 3.5 9 35.0
4 3.9 13 35.5
5 4.5 17 34.5
6 5.0 20 37.0
7 5.2 ... ...
8 5.7
9 6.8
10 7.9
... ...
Result:
index1 data1 data2
1 2.1 NaN
2 3.3 30.5
3 3.5 30.5
4 3.9 30.5
5 4.5 30.5
6 5.0 32.0
7 5.2 32.0
8 5.7 32.0
9 6.8 35.0
10 7.9 35.0
... ... ...
EDIT:
I think the following post is close to what I need, but I'm not sure how to transform the solution to fit my problem.
http://www.mathworks.com/matlabcentral/newsreader/view_thread/260139
EDIT (Several Months Later):
I've recently come across this excellent little function that I think may be of use to anyone who lands on this post:
function yi = interpLast(x,y,xi)
%INTERPLAST Interpolates the input data to the last known value.
% Note the index data should be input in ASCENDING order.
inds = arrayfun(#findinds, xi);
yi = y(inds);
function ind = findinds(val)
ind = find(x<=val,1,'last');
if isempty(ind)
ind = 1;
end
end
end
Credit goes here: http://www.mathworks.com/support/solutions/en/data/1-48KETY/index.html?product=SL&solution=1-48KETY
The problem is one of run length decoding. See section 15.5.2 of Matlab array manipulation tips and tricks (which is an eye-opening read for any Matlab enthusiast).
Here's using the method with your example (I'm using octave but the code is identical for Matlab):
octave:33> a=[2,30.5;6,32;9,35;13,35.5;17,34.5;20,37]
a =
2.0000 30.5000
6.0000 32.0000
9.0000 35.0000
13.0000 35.5000
17.0000 34.5000
20.0000 37.0000
octave:34> i=a(:,1)-1
i =
1
5
8
12
16
19
octave:35> j=zeros(1,i(end))
j =
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
octave:36> j(i(1:end-1)+1)=1
j =
0 1 0 0 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0
octave:37> j(1)=1
j =
1 1 0 0 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0
octave:38> val=a(:,2)
val =
30.500
32.000
35.000
35.500
34.500
37.000
octave:39> x=val(cumsum(j))
x =
30.500
32.000
32.000
32.000
32.000
35.000
35.000
35.000
35.500
35.500
35.500
35.500
34.500
34.500
34.500
34.500
37.000
37.000
37.000
And pad the beginning with NaN as needed.
I recently had the same problem as you: I had data, measured by different systems, which had to be synchronized and processed.
My solution consisted of putting the measurement data and time information (frequency, time at start of measurements) in a class object. Then I implemented a multiplication, addition, etc. method for that class that automatically took care of all the necessary things, being:
upsampling the lower frequency signal (with linear interpolation (interp1)
shifting one of the signals, so the data lines up in time
cutting off the non-overlapping data set at beginning and end (with two different systems you never start or stop measuring at the same time, so there is some excess data)
actually performing the multiplication
returning the result as a new class object
Next to that there were other functions of which you can guess what they do: plot, lpf, mean, getTimeAtIndex, getIndexAtTime, ...
This allowed me to simply do
signalsLabview = importLabViewSignals(LabViewData);
signalsMatlab = importMatlabSignals(MatlabData, 100); %hz
hydrPower = signalsLabview.flow * signalsMatlab.pressure;
plot(hydrPower);
or things like that. If you have a lot of these signals on which you have to do some math, this really helps and results in clear code. Otherwise you have a lot of general code just for doing the syncing, shifting, trimming around each operation. Also for quickly checking things it's easy.
If you have to do this things a lot, I think it's definitely worth investing some time in it to build a proper framework.
Unfortunately I don't think I can disclose this code (IP and such), but it wasn't rocket science.
Related
I'm analyzing an induction motor, varying the frequency and absolute value of the stator current. Since the FEM-Tool only works with a current input, I need to vary the current over the frequency to obtain current-values of constant torque for each frequency.
To generate a mesh, I use 2 for-loops:
The outer loop sets the current.
The inner loop varies the frequency with said current, gets the machine's torque and finally, the matrices are appended adding the current stator-current, frequency and torque each in separate matrices. Plotted it looks like this:
Example of the plot using the raw data
For the plot I used smaller, more imprecise matrices and rather arbitrary values:
I_S = [ 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 ];
fre = [ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 ];
tor = [ 0 0.1 0.3 0.5 0.7 1 1.5 2 2.6 3.3 0 1.1 1.3 1.5 1.7 2 2.5 3 3.6 4.3 0 2.1 2.3 2.5 2.7 3 3.5 4 4.6 5.3 ];
While tor is shown as the colormap in the plot. Each matrix has a length of 30.
One simulation needs about 20-30 seconds. Thus, to get a precise mesh, the FEM-tool needs several hours to generate.
I would like to interpolate the spaces in between the known ones.
It seems that either the way of creating the matrices is the problem or the interp*...-functions of Octave/MATLAB simply don't work for this kind of interpolation.
Is there a way to achieve a mesh/grid-like interpolation from this type of matrices? I found many examples with x,y as variables and z as a math-function but rarely 3 linear/non-linear matrices.
Your data need to be in a meshgrid form, that is 2D:
// Known data
current = [0:2];
frequency = [0:9];
[current2D, frequency2D] = meshgrid(current,frequency);
torque2D = [ 0 0.1 0.3; 0.5 0.7 1; 1.5 2 2.6; 3.3 0 1.1; 1.3 1.5 1.7; 2 2.5 3; 3.6 4.3 0; 2.1 2.3 2.5; 2.7 3 3.5; 4 4.6 5.3 ];
// Interpolated data
currentToInterpolate = [0.5 1.5];
frequncyToInterpolate = [0.5 : 8.5];
[currentToInterpolate2D, frequencyToInterpolate2D] = meshgrid(currentToInterpolate,frequncyToInterpolate);
interpolatedTorque2D = interp2(current2D,frequency2D,torque2D,currentToInterpolate2D,frequencyToInterpolate2D);
I need your expertise for a problem. I have the following matrix with different columns.
Time subsamp raw filt_BG filter
230.5382060 1 1 1 0
230.5382176 2 1 1 0
230.5382292 1 4 0 1
230.5382407 2 4 0 1
230.5382523 1 3 0 1
230.5382639 2 3 0 1
230.5382755 1 2 1 0
230.5382870 2 2 1 0
230.5382986 1 2 0 1
230.5383102 2 2 0 1
230.5383218 1 1 0 1
230.5383333 2 1 0 1
230.5383449 1 3 1 0
230.5383565 2 3 1 0
It contains two different type of raw data (column 3). Each type of raw data has same no. of sub sample (column 2). The 'filt_BG' and 'filter' seprate the type of data. I try to explain the problem and required solution with the following diagram.
In the above diagram (leftside), raw data is plotted as a function of time. The colors are representing different windows(subsample) 'sig' and 'BG'based on 'filter' and 'filt_BG', repectively. I want to do interpolation from the window 'BG1' to the window 'BG2'(subsample to subsample), so that interpolated data can be subtracted (subsample to subsample) from 'sig1' and 'sig2'. Similarly from 'BG2' to 'BG3' interpolation and subtraction from 'sig3' and 'sig4'. This gives me 'solution1' (upper right). Now, if 'BG3' do not exsist then I want to use only 'BG2' for subtraction from 'sig3' and 'sig4'(solution2, lower right). The solution matrix would be something like below.!!! Thank you for your ideas/solution.!!! Please keep in mind that I shall have millions of data points not as simple data as described here. Also, I do not have any toolbox but only the matlab available. Any solution based on toolbox is also fine.
Time subsamp solution1 solution2
230.5382060 1 NaN NaN
230.5382176 2 NaN NaN
230.5382292 1 2.5 2.5
230.5382407 2 2.5 2.5
230.5382523 1 1.5 1.5
230.5382639 2 1.5 1.5
230.5382755 1 NaN NaN
230.5382870 2 NaN NaN
230.5382986 1 -0.5 0
230.5383102 2 -0.5 0
230.5383218 1 -1.5 -1
230.5383333 2 -1.5 -1
230.5383449 1 NaN NaN
230.5383565 2 NaN NaN
Best Regards
You can use cumsum to compute sample and query points for the interpolation and the data can be interpolated using interp1. In the case of missing BG data (raw data contain NaN) it is required to add the first and the last valid raw data to its beginning and its end so interp1 can generate the desired extrapolations.
idxsig = data(:,5)==1; % logical index to sig
idxBG = (data(:,4)==1) & (~isnan(data(:,3))); % logical index to BG
f1 = find(idxBG,1,'first');
f2 = find(idxBG,1,'last');
% Add the first and the last-
% valid data to the beginning-
% and the end of raw data to-
% get valid extrapolation
sig = data(idxsig,3);
BG = [data(f1,3);data(idxBG,3);data(f2,3)];
idxBG = [true ;idxBG; true];
idxsig = [false; idxsig; false];
% preparing sample and query-
% points for interpolation
idx_sum = cumsum(idxBG);
idx_sig = idx_sum(idxsig)+0.5; % query points
idx_BG = idx_sum(idxBG); % sample points
intr = sig - interp1(idx_BG, BG, idx_sig);
solution = NaN(size(idxBG));
solution(idxsig) = intr % reformat to the original size
solution = solution(2:end-1);
solution =
NaN
NaN
2.50000
2.50000
1.50000
1.50000
NaN
NaN
-0.50000
-0.50000
-1.50000
-1.50000
NaN
NaN
I am asking an upgraded and more focused version of my query as I think I remained unable to explain properly in my last query.
I want to down sample my signal based on new time array.
I have time array and sample array.
t = [0 2 3 7 8 9 10 11 12 17 18 19 20];
A = [0 0 1 2 3 5.2 0 -1.4 0 2 2.7 2 2.3];
New time array is:
Tx = 1:4:25;
I am using interpolation proposed by Andrei Davydov on stackoverflow but I think I am at fault some point.
Can someone point me out where I am at fault? Thanks a lot in advance.
If the value of A is same at Tx as w.r.t (t)then use that value and if no value is found interpolate value and assign that value to new time .
Example Code:
t = [0 2 3 7 8 9 10 11 12 17 18 19 20 ];
A = [0 0 1 2 3 5.2 0 -1.4 0 2 2.7 2 2.3 ];
plot(t,A)
Tx = 1:4:25;
B = interp1(t,A,Tx); %re-make example data to have decimal points on the x-axis
y = resample(B, 1, 2);
T = 0.05;
Ty = T / (1 / 2);
ty = (0:length(y)-1)*Ty;
figure
plot(Tx,B,'b')
hold on
plot(ty,y,'r')
plot(t,A,'g')
hold off
Link to my previous question is attached here.
Resampling of time signal in MATLAB
Note :
This is now exactly what i want so this is more clear and mature.
I am using MATLAB version 2012b so please provide me solution w.r.t that as there are few Matlab build in command which don't work in 2012b.
The main Your problem - you try to extrapolate, as original time ends at 20, but Yours ends at 25. Try this code:
clc
t = [0 2 3 7 8 9 10 11 12 17 18 19 20 ];
A = [0 0 1 2 3 5.2 0 -1.4 0 2 2.7 2 2.3 ];
Tx = 1:4:25; % Are you sure you want to extrapolate?
% max(Tx1)>max(t)
% This variant is WITHOUT extrapolation
B = interp1(t,A,Tx);
% This variant is WITH extrapolation and WHOLE time-series interpolated
% cubically
extrapBcub=interp1(t,A,Tx,'pcchip');
% If you want to have linear (default) interpolation, but cubic
% extrapolation then
extrapBlin=[B(~isnan(B)), extrapBcub(isnan(B))];
It gives the following figure:
f=figure('Position',[50 50 1500 800])
h1=subplot(1,2,1);
hold all
h(1)=plot(t,A,'-ok','LineWidth',3)
h(2)=plot(Tx,B,'-ob','LineWidth',9)
h(3)=plot(Tx,extrapBcub,':or','LineWidth',7)
h(4)=plot(Tx,extrapBlin,'-og','LineWidth',3)
xlabel('time')
ylabel('signal')
set(gca,'Box','off','Color','none','FontSize',14,'LineWidth',2)
legend({'Original data','No extrapolation','Cubic all',...
'Linear interpolation+cubic extrapolation'},'Location','SouthOutside',...
'FontSize',22)
legend boxoff
h2=subplot(1,2,2);
hold all
h3 = copyobj(h(end:-1:1), h2) % copy plots just for scaling
ylim([-2 6])
xlabel('time')
ylabel('signal')
set(gca,'Box','off','Color','none','FontSize',14,'LineWidth',2)
I was wondering if there is a function like the one (smaller) below, but instead extracts values that are unequal to zero?
a = [67 0 8 25 0 20 0 90 7 2 9];
smaller10 = a(a < 10)
smaller10 =
0 8 0 0 7 2 9
Either use
a(a~=0)
or
nonzeros(a).'
You can find the documentation for nonzeros here.
I have the following two matrices
c=[1 0 1.05
1 3 2.05
1 6 2.52
1 9 0.88
2 0 2.58
2 3 0.53
2 6 3.69
2 9 0.18
3 0 3.22
3 3 1.88
3 6 3.98]
f=[1 6 3.9
1 9 9.1
1 12 9
2 0 0.3
2 3 0.9
2 6 1.2
2 9 2.5
3 0 2.7]
And the final matrix should be
n=[1 6 2.52 3.9
1 9 0.88 9.1
2 0 2.58 0.3
2 3 0.53 0.9
2 6 3.69 1.2
2 9 0.18 2.5
3 0 3.22 2.7]
The code I used gives as a result only the last row of the previous matrix [n].
for j=1
for i=1:rs1
for k=1
for l=1:rs2
if f(i,j)==c(l,k) && f(i,j+1)==c(l,k+1)
n=[f(i,j),f(i,j+1),f(i,j+2), c(l,k+2)];
end
end
end
end
end
Can anyone help me on this?
Is there something more simple?
Thanks in advance
You should learn to use set operations and avoid loops wherever possible. Here intersect could be extremely useful:
[u, idx_c, idx_f] = intersect(c(:, 1:2) , f(:, 1:2), 'rows');
n = [c(idx_c, :), f(idx_f, end)];
Explanation: by specifying the 'rows' flag, intersect finds the common rows in c and f, and their indices are given in idx_c and idx_f respectively. Use vector subscripting to extract matrix n.
Example
Let's use the example from your question:
c = [1 0 1.05;
1 3 2.05
1 6 2.52
1 9 0.88
2 0 2.58
2 3 0.53
2 6 3.69
2 9 0.18
3 0 3.22
3 3 1.88
3 6 3.98];
f = [1 6 3.9
1 9 9.1
1 12 9
2 0 0.3
2 3 0.9
2 6 1.2
2 9 2.5
3 0 2.7];
[u, idx_c, idx_f] = intersect(c(:, 1:2) , f(:, 1:2), 'rows');
n = [c(idx_c, :), f(idx_f, end)];
This should yield the desired result:
n =
1.0000 6.0000 2.5200 3.9000
1.0000 9.0000 0.8800 9.1000
2.0000 0 2.5800 0.3000
2.0000 3.0000 0.5300 0.9000
2.0000 6.0000 3.6900 1.2000
2.0000 9.0000 0.1800 2.5000
3.0000 0 3.2200 2.7000
According to this answer on Mathworks support you can use join from the statistics toolbox, specifically in your case, an inner join.
Unfortunately I don't have access to my computer with matlab on it, but give it a try and let us know how/if it works.
You can reduce the number of loops by comparing both the first and second columns of at once, then using the "all" function to only collapse the values if they both match. The following snippet replicates the "n" array you had provided.
n = [];
for r1 = 1:size(c, 1)
for r2 = 1:size(f,1)
if all(c(r1, [1 2]) == f(r2, [1 2]))
n(end+1, 1:4) = [c(r1,:) f(r2,3)];
end
end
end
If you insist on doing this in a loop you need to give n the proper dimension according
to the loop counter you are using, or concatenate it to itself of each iteration (this can be very slow for big matrices). For example, writing:
for j=1
for i=1:rs1
for k=1
for l=1:rs2
m=m+1;
if f(i,j)==c(l,k) && f(i,j+1)==c(l,k+1)
n(m,:)=[f(i,j),f(i,j+1),f(i,j+2), c(l,k+2)];
end
end
end
end
end
will save into the m-th row the for numbers when the loop reaches a counter value of m.
However, just be aware that this can be done also without a nested loop and an if condition, in a vectorized way. For example, instead of the condition if f(i,j)==c(l,k)... you can use ismember etc...
How about without any for loops at all (besides in native code)
mf = size(f,1);
mc = size(c,1);
a = repmat(c(:,1:2),1,mf);
b = repmat(reshape((f(:,1:2))',1,[]),mc,1);
match = a == b;
match = match(:, 1 : 2 : 2*mf) & match(:, 2 : 2 : 2*mf);
crows = nonzeros(diag(1:mc) * match);
frows = nonzeros(match * diag(1:mf));
n = [c(crows,:),f(frows,3)]