I need your expertise for a problem. I have the following matrix with different columns.
Time subsamp raw filt_BG filter
230.5382060 1 1 1 0
230.5382176 2 1 1 0
230.5382292 1 4 0 1
230.5382407 2 4 0 1
230.5382523 1 3 0 1
230.5382639 2 3 0 1
230.5382755 1 2 1 0
230.5382870 2 2 1 0
230.5382986 1 2 0 1
230.5383102 2 2 0 1
230.5383218 1 1 0 1
230.5383333 2 1 0 1
230.5383449 1 3 1 0
230.5383565 2 3 1 0
It contains two different type of raw data (column 3). Each type of raw data has same no. of sub sample (column 2). The 'filt_BG' and 'filter' seprate the type of data. I try to explain the problem and required solution with the following diagram.
In the above diagram (leftside), raw data is plotted as a function of time. The colors are representing different windows(subsample) 'sig' and 'BG'based on 'filter' and 'filt_BG', repectively. I want to do interpolation from the window 'BG1' to the window 'BG2'(subsample to subsample), so that interpolated data can be subtracted (subsample to subsample) from 'sig1' and 'sig2'. Similarly from 'BG2' to 'BG3' interpolation and subtraction from 'sig3' and 'sig4'. This gives me 'solution1' (upper right). Now, if 'BG3' do not exsist then I want to use only 'BG2' for subtraction from 'sig3' and 'sig4'(solution2, lower right). The solution matrix would be something like below.!!! Thank you for your ideas/solution.!!! Please keep in mind that I shall have millions of data points not as simple data as described here. Also, I do not have any toolbox but only the matlab available. Any solution based on toolbox is also fine.
Time subsamp solution1 solution2
230.5382060 1 NaN NaN
230.5382176 2 NaN NaN
230.5382292 1 2.5 2.5
230.5382407 2 2.5 2.5
230.5382523 1 1.5 1.5
230.5382639 2 1.5 1.5
230.5382755 1 NaN NaN
230.5382870 2 NaN NaN
230.5382986 1 -0.5 0
230.5383102 2 -0.5 0
230.5383218 1 -1.5 -1
230.5383333 2 -1.5 -1
230.5383449 1 NaN NaN
230.5383565 2 NaN NaN
Best Regards
You can use cumsum to compute sample and query points for the interpolation and the data can be interpolated using interp1. In the case of missing BG data (raw data contain NaN) it is required to add the first and the last valid raw data to its beginning and its end so interp1 can generate the desired extrapolations.
idxsig = data(:,5)==1; % logical index to sig
idxBG = (data(:,4)==1) & (~isnan(data(:,3))); % logical index to BG
f1 = find(idxBG,1,'first');
f2 = find(idxBG,1,'last');
% Add the first and the last-
% valid data to the beginning-
% and the end of raw data to-
% get valid extrapolation
sig = data(idxsig,3);
BG = [data(f1,3);data(idxBG,3);data(f2,3)];
idxBG = [true ;idxBG; true];
idxsig = [false; idxsig; false];
% preparing sample and query-
% points for interpolation
idx_sum = cumsum(idxBG);
idx_sig = idx_sum(idxsig)+0.5; % query points
idx_BG = idx_sum(idxBG); % sample points
intr = sig - interp1(idx_BG, BG, idx_sig);
solution = NaN(size(idxBG));
solution(idxsig) = intr % reformat to the original size
solution = solution(2:end-1);
solution =
NaN
NaN
2.50000
2.50000
1.50000
1.50000
NaN
NaN
-0.50000
-0.50000
-1.50000
-1.50000
NaN
NaN
Related
I have been having problem with identifying two maximum values' position in 3D matrix (MATLAB). Say I have matrix A output as follows:
A(:,:,1) =
5 3 5
0 1 0
A(:,:,2) =
0 2 0
8 0 8
A(:,:,3) =
3 0 0
0 7 7
A(:,:,4) =
6 6 0
4 0 0
For the first A(:,:,1), I want to identify that the first row have the highest value (A=5). But I need the two index position, which in this case, 1 and 3. And this is the same as the other A(:,:,:).
I have searched through SO but since I am bad in MATLAB, I couldn't find way to work this through.
Please do help me on this. It would be better if I don't need to use for loop to get the desired output.
Shot #1 Finding the indices for maximum values across each 3D slice -
%// Reshape A into a 2D matrix
A_2d = reshape(A,[],size(A,3))
%// Find linear indices of maximum numbers for each 3D slice
idx = find(reshape(bsxfun(#eq,A_2d,max(A_2d,[],1)),size(A)))
%// Convert those linear indices to dim1, dim2,dim3 indices and
%// present the final output as a Nx3 array
[dim1_idx,dim2_idx,dim3_idx] = ind2sub(size(A),idx)
out_idx_triplet = [dim1_idx dim2_idx dim3_idx]
Sample run -
>> A
A(:,:,1) =
5 3 5
0 1 0
A(:,:,2) =
0 2 0
8 0 8
A(:,:,3) =
3 0 0
0 7 7
A(:,:,4) =
6 6 0
4 0 0
out_idx_triplet =
1 1 1
1 3 1
2 1 2
2 3 2
2 2 3
2 3 3
1 1 4
1 2 4
out_idx_triplet(:,2) is what you are looking for!
Shot #2 Finding the indices for highest two numbers across each 3D slice -
%// Get size of A
[m,n,r] = size(A)
%// Reshape A into a 2D matrix
A_2d = reshape(A,[],r)
%// Find linear indices of highest two numbers for each 3D slice
[~,sorted_idx] = sort(A_2d,1,'descend')
idx = bsxfun(#plus,sorted_idx(1:2,:),[0:r-1]*m*n)
%// Convert those linear indices to dim1, dim2,dim3 indices
[dim1_idx,dim2_idx,dim3_idx] = ind2sub(size(A),idx(:))
%// Present the final output as a Nx3 array
out_idx_triplet = [dim1_idx dim2_idx dim3_idx]
out_idx_triplet(:,2) is what you are looking for!
The following code gives you the column and row of the respective maximum.
The first step will obtain the maximum of each sub-matrix containing the first and second dimension. Since max works per default with the first dimension, the matrix is reshaped to combine the original first and second dimension.
max_vals = max(reshape(A,size(A,1)*size(A,2),size(A,3)));
max_vals =
5 8 7 6
In the second step, the index of elements equal to the respective max_vals of each sub-matrix is obtained using arrayfun over the third dimension. Since the output of arrayfun are cells, cell2mat is used to transform the output into a matrix. As a last step, the linear index from find is transformed into sub-indices by ind2sub.
[i,j] = ind2sub(size(A(:,:,1)),cell2mat(arrayfun(#(i)find(A(:,:,i)==max_vals(i)),1:size(A,3),'UniformOutput',false)))
i =
1 2 2 1
1 2 2 1
j =
1 1 2 1
3 3 3 2
Hence, the values in j are the ones you want to have.
I have a Nx3-matrix in matlab, where I have a degree value from 0 to 360 in the first column, a radius value from 0 to 0.5 in the second and an integer in the third. Every combination out of (M(n,1),M(n,2)) is unique with M the matrix and n a random number between 1 and N, but it is possible that there is a value in M(:,1) or M(:,2) more than once. M is sorted, first after the first row, then after the second.
My target is now to reshape this matrix into a 360xV-matrix, with V the amount of unique values in M(:,2). If there is a value in M(:,3) at the position M(o,1) and M(p,2) with 1 <= o, p <= N, it should be placed at the position (o,p), if there is no value, then there should a NaN-value placed instead.
Is there a simple way to do this, or do I have to write my own function for that?
Edit:
Desired input:
0 0.25 1
0 0.43 4
1 0.25 2
2 0.03 5
2 0.43 2
Desired output:
NaN 1 4
NaN 2 NaN
5 NaN 2
You can use an approach of finding unique indices for the first and second columns from the input arrays and then using those to set elements in an appropriately (discussed in more detail inside the code as comments) sized output array with the elements from the third column. Here's the implementation -
%// Input array
A = [
0 0.25 1
0 0.43 4
1 0.25 2
2 0.03 5
2 0.43 2 ]
%// Find unique indices for col-1,2
[~,~,idx1] = unique(A(:,1)) %// these would form the row indices of output
[~,~,idx2] = unique(A(:,2)) %// these would form the col indices of output
%// Decide the size of output array based on the "extents" of those indices
nrows = max(idx1)
ncols = max(idx2)
%// Initialize output array with NaNs
out = NaN(nrows,ncols)
%// Set the elements in output indexed by those indices to values from
%// col-3 of input array
out((idx2-1)*nrows + idx1) = A(:,3)
Code run -
out =
NaN 1 4
NaN 2 NaN
5 NaN 2
Is there a simple way to do this, or do I have to write my own function for that?
You'll need to write a method, seeing that what you've described is utterly specific to your problem. There's methods to find unique values, so this will help you when designing your for loop.
I have been through a bunch of questions about the Repeat function in MatLab, but I can't figure out how this process work.
I am trying to translate it into R, but my problem is that I do not know how the function manipulates the data.
The code is part of a process to make a pairs trading strategy, where the code takes in a vector of FALSE/TRUE expressions.
The code is:
% initialize positions array
positions=NaN(length(tday), 2);
% long entries
positions(shorts, :)=repmat([-1 1], [length(find(shorts)) 1]);
where shorts is the vector of TRUE/FALSE expressions.
Hope you can help.
repmat repeats the matrix you give him [dim1 dim2 dim3,...] times. What your code does is:
1.-length(find(shorts)): gets the amount of "trues" in shorts.
e.g:
shorts=[1 0 0 0 1 0]
length(find(shorts))
ans = 2
2.-repmat([-1 1], [length(find(shorts)) 1]); repeats the [-1 1] [length(find(shorts)) 1] times.
continuation of e.g.:
repmat([-1 1], [length(find(shorts)) 1]);
ans=[-1 1
-1 1];
3.- positions(shorts, :)= saves the given matrix in the given indexes. (NOTE!: only works if shorts is of type logical).
continuation of e.g.:
At this point, if you haven't omit anything, positions should be a 6x2 NaN matrix. the indexing will fill the true positions of shorts with the [-1 1] matrix. so after this, positions will be:
positions=[-1 1
NaN NaN
NaN NaN
NaN NaN
-1 1
NaN NaN]
Hope it helps
The MATLAB repmat function replicates and tiles the array. The syntax is
B = repmat(A,n)
where A is the input array and n specifies how to tile the array. If n is a vector [n1,n2] - as in your case - then A is replicated n1 times in rows and n2 times in columns. E.g.
A = [ 1 2 ; 3 4]
B = repmat(A,[2,3])
B = | |
1 2 1 2 1 2
3 4 3 4 3 4 __
1 2 1 2 1 2
3 4 3 4 3 4
(the lines are only to illustrate how A gets tiled)
In your case, repmat replicates the vector [-1, 1] for each non-zero element of shorts. You thus set each row of positions, whos corresponding entry in shorts is not zero, to [-1,1]. All other rows will stay NaN.
For example if
shorts = [1; 0; 1; 1; 0];
then your code will create
positions =
-1 1
NaN NaN
-1 1
-1 1
NaN NaN
I hope this helps you to clarify the effect of repmat. If not, feel free to ask.
I have a 3x3 matrix, populated with NaN and values of a variable:
NaN 7 NaN
5 NaN 0
NaN NaN 4
matrix = [NaN 7 NaN; 5 NaN 0; NaN NaN 4]
I would like to get the row and column numbers of non-NaN cells and put them in a matrix together with the value of the variable. That is, I would like to obtain the following matrix:
row col value
1 2 7
2 1 5
2 3 0
3 3 4
want = [1 2 7; 2 1 5; 2 3 0; 3 3 4]
Any help would be highly appreciated.
This can be done without loops:
[jj, ii, kk] = find((~isnan(matrix).*(reshape(1:numel(matrix), size(matrix)))).');
result = [ii jj matrix(kk)];
The trick is to multiply ~isnan(matrix) by a matrix of indices so that the third output of find gives the linear index of non-NaN entries. The transpose is needed to have the same order as in the question.
The following should work!
[p,q]=find(~isnan(matrix)) % Loops through matrix to find indices
want = zeros(numel(p),3) % three columns you need with same number of rows as p
for i=1:numel(p)
want[i,:] = [p(i) q(i) matrix(p(i), matrix(i))]
end
Should give you the correct result which is:
2 1 5
1 2 7
2 3 0
3 3 4
If you don't mind the ordering of the rows, you can use a simplified version of Luis Mendo's answer:
[row, col] = find(~isnan(matrix));
result = [row(:), col(:), matrix(~isnan(matrix))];
Which will result in:
2 1 5
1 2 7
2 3 0
3 3 4
I would like to get the minimum nonzero values per row in a sparse matrix. Solutions I found for dense matrices suggested masking out the zero values by setting them to NaN or Inf. However, this obviously doesn't work for sparse matrices.
Ideally, I should get a column vector of all the row-wise minima, as I would get with
minValues = min( A, [], 2);
Except, obviously, using min leaves me with an all-zeros column vector due to the sparsity. Is there a solution using find?
This is perfect for accumarray. Consider the following sparse matrix,
vals = [3 1 1 9 7 4 10 1]; % got this from randi(10,1,8)
S = sparse([1 3 4 4 5 5 7 9],[2 2 3 6 7 8 8 11],vals);
Get the minimum value for each row, assuming 0 for empty elements:
[ii,jj] = find(S);
rowMinVals = accumarray(ii,nonzeros(S),[],#min)
Note that rows 4 and 5 of rowMinVals, which are the only two rows of S with multiple nonzero values are equal to the min of the row:
rowMinVals =
3
0
1
1 % min([1 9]
4 % min([7 4]
0
10
0
1
If the last row(s) of your sparse matrix do not contain any non-zeros, but you want your min row value output to reflect that you have numRows, for example, change theaccumarray command as follows,
rowMinVals = accumarray(ii,nonzeros(S),[numRows 1],#min).
Also, perhaps you also want to avoid including the default 0 in the output. One way to handle that is to set the fillval input argument to NaN:
rowMinVals = accumarray(ii,nonzeros(S),[numRows 1],#min,NaN)
rowMinVals =
3
NaN
1
1
4
NaN
10
NaN
1
NaN
NaN
NaN
Or you can keep using a sparse matrix with the fifth input argument, issparse:
>> rowMinVals = accumarray(ii,nonzeros(S),[],#min,[],true)
rowMinVals =
(1,1) 3
(3,1) 1
(4,1) 1
(5,1) 4
(7,1) 10
(9,1) 1