I have collected two types of data. One is a struct Outputs with 3 fields: Outputs.time, Outputs.signals and an unimportant one. Outputs.time is a columnvector containing all the time values (where the data is sampled), Outputs.signals has 15 rows, on each row the values and properties of a signal (so there are 15 signals in total). Consequently Outputs.signals(i).values has the same number of rows as Outputs.time.
Now i have another table with 4 columns: LabData.time, LabData.NdBoiler, LabData.NdOutput and an unimportant one. Outputs.time contains all the computer sampled data, LabData.time only some measurements taken by hand. Ergo, Outputs.time is way larger than LabData.time, but at certain times (where Outputs.time = LabData.time) there are values for both Outputs.signals and the other columns of LabData.
The goal is to put the values of LabData.NdBoiler and LabData.NdOutput in Outputs.signals(16) and Outputs.signals(17) for the time-samples where the value is known. For the other values, Outputs.signals(16) = NaN and Outputs.signals(17) = NaN. But i dont know how to do that, could you help me?
Example:
Outputs.time = [1; 2; 3; 4; 5];
Outputs.signals(1).values = [1111; 2222; 3333; 4444; 5555]; %and so on for the other signals
LabData.time = [2; 4];
LabData.NdBoiler = [1.23; 1.32];
%% Now the final result should be
Outputs.signals(16).values = [NaN; 1.23; NaN; 1.32; NaN]
The idea is to first create the vector of NaNs whereafter you match the timepoints using ismember to substitute the values you know in.
Outputs.signals(16).values = nan(1,length(Outputs.time)); %Vector of nans
Lia = ismember(Outputs.time,LabData.time); %Where does the times match?
Outputs.signals(16).values(Lia) = LabData.NdBoiler; %substitute
Related
I am working with a n x 1 matrix, A, that has repeating values inside it:
A = [0;1;2;3;4; 0;1;2;3;4; 0;1;2;3;4; 0;1;2;3;4]
which correspond to an n x 1 matrix of B values:
B = [2;4;6;8;10; 3;5;7;9;11; 4;6;8;10;12; 5;7;9;11;13]
I am attempting to produce a generalised code to place each repetition into a separate column and store it into Aa and Bb, e.g.:
Aa = [0 0 0 0 Bb = [2 3 4 5
1 1 1 1 4 5 6 7
2 2 2 2 6 7 8 9
3 3 3 3 8 9 10 11
4 4 4 4] 10 11 12 13]
Essentially, each repetition from A and B needs to be copied into the next column and then deleted from the first column
So far I have managed to identify how many repetitions there are and copy the entire column over to the next column and then the next for the amount of repetitions there are but my method doesn't shift the matrix rows to columns as such.
clc;clf;close all
A = [0;1;2;3;4;0;1;2;3;4;0;1;2;3;4;0;1;2;3;4];
B = [2;4;6;8;10;3;5;7;9;11;4;6;8;10;12;5;7;9;11;13];
desiredCol = 1; %next column to go to
destinationCol = 0; %column to start on
n = length(A);
for i = 2:1:n-1
if A == 0;
A = [ A(:, 1:destinationCol)...
A(:, desiredCol+1:destinationCol)...
A(:, desiredCol)...
A(:, destinationCol+1:end) ];
end
end
A = [...] retrieved from Move a set of N-rows to another column in MATLAB
Any hints would be much appreciated. If you need further explanation, let me know!
Thanks!
Given our discussion in the comments, all you need is to use reshape which converts a matrix of known dimensions into an output matrix with specified dimensions provided that the number of elements match. You wish to transform a vector which has a set amount of repeating patterns into a matrix where each column has one of these repeating instances. reshape creates a matrix in column-major order where values are sampled column-wise and the matrix is populated this way. This is perfect for your situation.
Assuming that you already know how many "repeats" you're expecting, we call this An, you simply need to reshape your vector so that it has T = n / An rows where n is the length of the vector. Something like this will work.
n = numel(A); T = n / An;
Aa = reshape(A, T, []);
Bb = reshape(B, T, []);
The third parameter has empty braces and this tells MATLAB to infer how many columns there will be given that there are T rows. Technically, this would simply be An columns but it's nice to show you how flexible MATLAB can be.
If you say you already know the repeated subvector, and the number of times it repeats then it is relatively straight forward:
First make your new A matrix with the repmat function.
Then remap your B vector to the same size as you new A matrix
% Given that you already have the repeated subvector Asub, and the number
% of times it repeats; An:
Asub = [0;1;2;3;4];
An = 4;
lengthAsub = length(Asub);
Anew = repmat(Asub, [1,An]);
% If you can assume that the number of elements in B is equal to the number
% of elements in A:
numberColumns = size(Anew, 2);
newB = zeros(size(Anew));
for i = 1:numberColumns
indexStart = (i-1) * lengthAsub + 1;
indexEnd = indexStart + An;
newB(:,i) = B(indexStart:indexEnd);
end
If you don't know what is in your original A vector, but you do know it is repetitive, if you assume that the pattern has no repeats you can use the find function to find when the first element is repeated:
lengthAsub = find(A(2:end) == A(1), 1);
Asub = A(1:lengthAsub);
An = length(A) / lengthAsub
Hopefully this fits in with your data: the only reason it would not is if your subvector within A is a pattern which does not have unique numbers, such as:
A = [0;1;2;3;2;1;0; 0;1;2;3;2;1;0; 0;1;2;3;2;1;0; 0;1;2;3;2;1;0;]
It is worth noting that from the above intuitively you would have lengthAsub = find(A(2:end) == A(1), 1) - 1;, But this is not necessary because you are already effectively taking the one off by only looking in the matrix A(2:end).
I have a matrix with constant consecutive values randomly distributed throughout the matrix. I want the indices of the consecutive values, and further, I want a matrix of the same size as the original matrix, where the number of consecutive values are stored in the indices of the consecutive values. For Example
original_matrix = [1 1 1;2 2 3; 1 2 3];
output_matrix = [3 3 3;2 2 0;0 0 0];
I have struggled mightily to find a solution to this problem. It has relevance for meteorological data quality control. For example, if I have a matrix of temperature data from a number of sensors, and I want to know what days had constant consecutive values, and how many days were constant, so I can then flag the data as possibly faulty.
temperature matrix is number of days x number of stations and I want an output matrix that is also number of days x number of stations, where the consecutive values are flagged as described above.
If you have a solution to that, please provide! Thank you.
For this kind of problems, I made my own utility function runlength:
function RL = runlength(M)
% calculates length of runs of consecutive equal items along columns of M
% work along columns, so that you can use linear indexing
% find locations where items change along column
jumps = diff(M) ~= 0;
% add implicit jumps at start and end
ncol = size(jumps, 2);
jumps = [true(1, ncol); jumps; true(1, ncol)];
% find linear indices of starts and stops of runs
ijump = find(jumps);
nrow = size(jumps, 1);
istart = ijump(rem(ijump, nrow) ~= 0); % remove fake starts in last row
istop = ijump(rem(ijump, nrow) ~= 1); % remove fake stops in first row
rl = istop - istart;
assert(sum(rl) == numel(M))
% make matrix of 'derivative' of runlength
% don't need last row, but needs same size as jumps for indices to be valid
dRL = zeros(size(jumps));
dRL(istart) = rl;
dRL(istop) = dRL(istop) - rl;
% remove last row and 'integrate' to get runlength
RL = cumsum(dRL(1:end-1,:));
It only works along columns since it uses linear indexing. Since you want do something similar along rows, you need to transpose back and forth, so you could use it for your case like so:
>> original = [1 1 1;2 2 3; 1 2 3];
>> original = original.'; % transpose, since runlength works along columns
>> output = runlength(original);
>> output = output.'; % transpose back
>> output(output == 1) = 0; % see hitzg's comment
>> output
output =
3 3 3
2 2 0
0 0 0
I have a data matrix A, the size of which is 4*20 (4 rows, 20 columns). The matrix A is generated by A = randn (4, 20).
In the first iteration, I delete columns [2,3] of matrix A. Then matrix A becomes matrix A1, the size of which is 4*18.
In the second iteration, I delete columns [4 8 10] of matrix A1. Then matrix A1 becomes matrix A2, the size of which is 4*15.
In the third iteration, I delete columns [1 3 6 9 10] of matrix A2. Then matrix A2 becomes matrix A3, the size of which is 4*10.
The deleted elements are put into a matrix B. My question is how to figure out the x and y coordinates in original matrix A of every deleted element in B. Anyone can give me a help? Thank you so much!
I would personally keep another vector that ranges between 1 to 20 and put your removal of columns within a loop. Let's call this vector column_choice. At each iteration, use randperm to randomly select from column_choice those columns you want to remove, then append these to your matrix B. Once you select these columns, remove these elements from column_choice and continue with your code. Also, those columns from column_choice we will add to another vector... call it, final_columns. This vector will tell you which vectors you ultimately removed in the end, and you can reference these columns in the original matrix.
To make things efficient, create an array where each element contains the total number of columns you want to remove at each iteration. Therefore, do something like:
cols_to_remove = [2 3 5];
The first element means you want to remove 2 columns in the first iteration, the second element means you want to remove 3 columns in the second iteration, and 5 columns in the last iteration. Because you're looping, it's a good idea to pre-allocate your matrix. In total, you're going to have 10 columns removed and populated in B, and since your random matrix has 4 rows, you should do this:
B = zeros(4,sum(cols_to_remove));
We are summing over cols_to_remove as this tells us how many columns we will ultimately be removing all together. One thing I'd like to mention is that you should make a copy of A before we start removing columns. That way, you're able to reference back into the original matrix.
Finally, without further ado, here's the code that I would write to tackle this problem:
column_choice = 1 : 20;
cols_to_remove = [2 3 5];
B = zeros(4,sum(cols_to_remove));
final_columns = zeros(1,sum(cols_to_remove));
A = randn(4,20); %// From your post
Acopy = A; %// Make a copy of the matrix
%// Keep track of which column we need to start populating
%// B at
counter = 1;
%// For each amount of columns we want to remove...
for cols = cols_to_remove
%// Randomly choose the columns we want to remove
to_remove = randperm(numel(column_choice), cols);
%// Remove from the A matrix and store into B
removed_cols = Acopy(:,to_remove);
Acopy(:,to_remove) = [];
B(:,counter : counter + cols - 1) = removed_cols;
%// Also add columns removed to final_columns
final_columns(counter : counter + cols - 1) = column_choice(to_remove);
%// Increment counter for the next spot to place columns
counter = counter + cols;
%// Also remove from column_choice
column_choice(to_remove) = [];
%// Continue your code here to process A and/or B
%//...
%//...
end
%// Remove copy to save memory
clear Acopy;
Therefore, final_columns will give you which columns were removed from the original matrix, and you can refer back to A to locate where these are. B will contain those removed columns from A and are all concatenated together.
Edit
As per your comments, you want to remove certain rows from each intermediate result. As such, you would specify which columns you want to remove in the second dimension of each matrix, then set it equal to []. Make sure you copy over each result into a new matrix before removing the columns. Also, you'll need to keep track of which indices from the original matrix you removed, so make that column_choice and final_columns vector again and repeat the saving logic that we have talked about before.
Therefore:
column_choice = 1:20;
final_columns = zeros(1,10);
A1 = A;
A1(:,[2 3]) = [];
final_columns(1:2) = column_choice([2 3]);
column_choice([2 3]) = [];
A2 = A1;
A2(:,[4 9 11]) = [];
final_columns(3:5) = column_choice([4 9 11]);
column_choice([4 9 11]) = [];
A3 = A2;
A3(:,[1 2 5 8 12]);
final_columns(6:10) = column_choice([1 2 5 8 12]);
column_choice([1 2 5 8 12]) = [];
I have a quite big (107 x n) matrix X. Within these n columns, each three columns belong to each other. So, the first three columns of matrix X build a block, then columns 4,5,6 and so on.
Within each block, the first 100 row elements of the first column are important X(1:100,1:3:end). Whenever in this first column the number of zeros or NaNs is greater or equal 20, it should delete the whole block.
Is there a way to do this without a loop?
Thanks for any advice!
Assuming the number of columns of the input to be a multiple of 3, there could be two approaches here.
Approach #1
%// parameters
rl = 100; %// row limit
cl = 20; %// count limit
X1 = X(1:rl,1:3:end) %// Important elements from input
match_mat = isnan(X1) | X1==0 %// binary array of matches
match_blk_id = find(sum(match_mat)>=cl) %// blocks that satisfy requirements
match_colstart = (match_blk_id-1).*3+1 %// start column indices that satisfy
all_col_ind = bsxfun(#plus,match_colstart,[0:2]') %//'columns indices to be removed
X(:,all_col_ind)=[] %// final output after removing to be removed columns
Or if you prefer "compact" codes -
X1 = X(1:rl,1:3:end);
X(:,bsxfun(#plus,(find(sum(isnan(X1) | X1==0)>=cl)-1).*3+1,[0:2]'))=[];
Approach #2
X1 = X(1:rl,1:3:end)
match_mat = isnan(X1) | X1==0 %// binary array of matches
X(:,repmat(sum(match_mat)>=cl,[3 1]))=[] %// Find matching blocks, replicate to
%// next two columns and remove them from X
Note: If X is not a multiple of 3, use this before using the codes - X = [X zeros(size(X,1) ,3 - mod(size(X,2),3))].
This post follows a previous question regarding the restructuring of a matrix:
re-formatting a matrix in matlab
An additional problem I face is demonstrated by the following example:
depth = [0:1:20]';
data = rand(1,length(depth))';
d = [depth,data];
d = [d;d(1:20,:);d];
Here I would like to alter this matrix so that each column represents a specific depth and each row represents time, so eventually I will have 3 rows (i.e. days) and 21 columns (i.e. measurement at each depth). However, we cannot reshape this because the number of measurements for a given day are not the same i.e. some are missing. This is known by:
dd = sortrows(d,1);
for i = 1:length(depth);
e(i) = length(dd(dd(:,1)==depth(i),:));
end
From 'e' we find that the number of depth is different for different days. How could I insert a nan into the matrix so that each day has the same depth values? I could find the unique depths first by:
unique(d(:,1))
From this, if a depth (from unique) is missing for a given day I would like to insert the depth to the correct position and insert a nan into the respective location in the column of data. How can this be achieved?
You were thinking correctly that unique may come in handy here. You also need the third output argument, which maps the unique depths onto the positions in the original d vector. have a look at this code - comments explain what I do
% find unique depths and their mapping onto the d array
[depths, ~, j] = unique(d(:,1));
% find the start of every day of measurements
% the assumption here is that the depths for each day are in increasing order
days_data = [1; diff(d(:,1))<0];
% count the number of days
ndays = sum(days_data);
% map every entry in d to the correct day
days_data = cumsum(days_data);
% construct the output array full of nans
dd = nan(numel(depths), ndays);
% assing the existing measurements using linear indices
% Where data does not exist, NaN will remain
dd(sub2ind(size(dd), j, days_data)) = d(:,2)
dd =
0.5115 0.5115 0.5115
0.8194 0.8194 0.8194
0.5803 0.5803 0.5803
0.9404 0.9404 0.9404
0.3269 0.3269 0.3269
0.8546 0.8546 0.8546
0.7854 0.7854 0.7854
0.8086 0.8086 0.8086
0.5485 0.5485 0.5485
0.0663 0.0663 0.0663
0.8422 0.8422 0.8422
0.7958 0.7958 0.7958
0.1347 0.1347 0.1347
0.8326 0.8326 0.8326
0.3549 0.3549 0.3549
0.9585 0.9585 0.9585
0.1125 0.1125 0.1125
0.8541 0.8541 0.8541
0.9872 0.9872 0.9872
0.2892 0.2892 0.2892
0.4692 NaN 0.4692
You may want to transpose the matrix.
It's not entirely clear from your question what your data looks like exactly, but the following might help you towards an answer.
Suppose you have a column vector
day1 = 1:21';
and, initially, all the values are NaN
day1(:) = NaN
Suppose next that you have a 2d array of measurements, in which the first column represents depths, and the second the measurements at those depths. For example
msrmnts = [1,2;2,3;4,5;6,7] % etc
then the assignment
day1(msrmnts(:,1)) = msrmnts(:,2)
will set values in only those rows of day1 whose indices are found in the first column of msrmnts. This second statement uses Matlab's capabilities for using one array as a set of indices into another array, for example
d([9 7 8 12 4]) = 1:5
would set elements [9 7 8 12 4] of d to the values 1:5. Note that the indices of the elements do not need to be in order. You could even insert the same value several times into the index array, eg [4 4 5 6 3 4] though it's not terribly useful.