Overview
I am currently working with a series of .txt files I am importing into MATLAB. For simplicity, I'll show my problem conceptually. Obligatory, I'm new to MATLAB (or programming in general).
These .txt files contain data from tracking a ROI in a video (frame-by-frame) with time ('t') in the first column and velocity ('v') in the second as shown below;
T1 = T2 = etc.
t v t v
0 NaN 0 NaN
0.1 100 0.1 200
0.2 200 0.2 500
0.3 400 0.3 NaN
0.4 150
0.5 NaN
Problem
Files differ in their size, the columns remain fixed but the rows vary from trial to trial as shown in T1 and T2.
The time column is the same for each of these files so I wanted to organise data in a table as follows;
time v1 v2 etc.
0 NaN NaN
0.1 100 200
0.2 200 500
0.3 400 NaN
0.4 150 0
0.5 NaN 0
Note that I want to add 0s (or NaN) to end of shorter trials to fix the issue of size differences.
Edit
Both solutions worked well for my dataset. I appreciate all the help!
You could import each file into a table using readtable and then use outerjoin to combine the tables in the way that you would expect. This will work if all data starts at t = 0 or not.
To create a table from a file:
T1 = readtable('filename1.dat');
T2 = readtable('filename2.dat');
Then to perform the outerjoin (pseudo data created for demonstration purposes).
t1 = table((1:4)', (5:8)', 'VariableNames', {'t', 'v'});
%// t v
%// _ _
%// 1 5
%// 2 6
%// 3 7
%// 4 8
% t2 is missing row 2
t2 = table([1;3;4], [1;3;4], 'VariableNames', {'t', 'v'});
%// t v
%// _ _
%// 1 1
%// 3 3
%// 4 4
%// Now perform an outer join and merge the key column
t3 = outerjoin(t1, t2, 'Keys', 't', 'MergeKeys', true)
%// t v_t1 v_t2
%// _ ____ ____
%// 1 5 1
%// 2 6 NaN
%// 3 7 3
%// 4 8 4
I would suggest the use of the padarray and horzcat functions. They respectively :
Pad a matrix or vector with extra data, effectively adding extra 0's or any specified value (NaNs work too).
Concatenate matrices or vectors horizontally.
First, try to obtain the length of the longest vector you have to concatenate. Let's call this value max_len. Once you have that, you can then pad each vector by doing :
v1 = padarray(v1, max_len - length(v1), 0, 'post');
% You can replace the '0' by any value you want !
Finally, once you have vectors of the same size, you can concatenate them using horzcat :
big_table = horzcat(v1, v2, ... , vn);
Related
From a binary matrix, I want to calculate a kind of adjacency/joint probability density matrix (not quite sure how to label it as so please feel free to rename).
For example, I start with this matrix:
A = [1 1 0 1 1
1 0 0 1 1
0 0 0 1 0]
I want to produce this output:
Output = [1 4/5 1/5
4/5 1 1/5
1/5 1/5 1]
Basically, for each row, I want to calculate the proportion of times where they agreed (1 and 1 or 0 and 0). A will always agree with itself and thus have it as 1 along the diagonal. No matter how many different js are added it will still result in a 3x3, but an extra i variable will result in a 4x4.
I like to think of the inputs along i in the A matrix as the person and Js as the question and so the final output is a 3x3 (number of persons) matrix.
I am having some trouble with this on matlab. If you could please help point me in the right direction that would be fabulous.
So, you can do this in two parts.
bothOnes = A*A';
gives you a matrix showing how many 1s each pair of rows share, and
bothZeros = (1-A)*(1-A)';
gives you a matrix showing how many 0s each pair of rows share.
If you just add them up, you get how many elements they share of either type:
bothSame = A*A' + (1-A)*(1-A)';
Then just divide by the row length to get the desired fractional representation:
output = (A*A' + (1-A)*(1-A)') / size(A, 2);
That should get you there.
Note that this only works if A contains only 1's and 0's, but it can be adapted for other cases.
Here are some alternatives, assuming A can only contain 0 and 1:
If you have the Statistics Toolbox:
result = 1-squareform(pdist(A, 'hamming'));
Manual approach with implicit expansion:
result = mean(permute(A, [1 3 2])==permute(A, [3 1 2]), 3);
Using bitwise operations. This is a more esoteric approach, and is only valid if A has at most 53 columns, due to floating-point limitations:
t = bin2dec(char(A+'0')); % convert each row from binary to decimal
u = bitxor(t, t.'); % bitwise xor
v = mean(dec2bin(u)-'0', 2); % compute desired values
result = 1 - reshape(v, size(A,1), []); % reshape to obtain result
I have 3 vectors, v1, v2, v3. What I want to get is the difference between every possible pair of them, that is, v1-v2, v1-v3, v2-v3. How can I do this without looping in matlab?
Thank you.
Just use nchoosek to generate the combinations first and then use them to index into your array of row-vectors:
Test case:
numVectors = 3;
dim = 5;
Vs = rand(numVectors, dim);
Actual computation:
combs = nchoosek(1:size(Vs,1), 2);
differences = Vs(combs(:,1),:) - Vs(combs(:,2),:);
The above creates 3 random row vectors of dimension 5. So in your case, you may want to replace the creation of the random matrix with Vs = [v1; v2; v3]; if your vectors are row vectors; or transpose the vectors using Vs = [v1, v2, v3].'; if your data are column vectors.
Using bsxfun:
clear
clc
%// Sample vectors.
v1 = [1 2];
v2 = [10 20];
v3 = [0 0];
Out = bsxfun(#minus,[v1 v2 v3], [v1 v2 v3].')
Out =
0 1 9 19 -1 -1
-1 0 8 18 -2 -2
-9 -8 0 10 -10 -10
-19 -18 -10 0 -20 -20
1 2 10 20 0 0
1 2 10 20 0 0
Reasoning: Each difference is computed starting from the 1st element of the 1st vector until the 2nd element of the last vector.
The 1st column contains all the differences for the 1st element of the 1st vector, i.e. (1 -1), (1-2), (1-10), (1 - 20), (1 - 0), (1 - 0).
Then 2nd column, same thing but this time with the 2: (2 - 1), (2 - 2), (2 - 10), and so on.
Sorry if my explanations are unclear haha I don't know the right terms in english. Please ask for more details.
Code
%// Concatenate all vectors to form a 2D array
V = cat(2,v1(:),v2(:),v3(:),v4(:),v5(:))
N = size(V,2) %// number of vectors
%// Find all IDs of all combinations as x,y
[y,x] = find(bsxfun(#gt,[1:N]',[1:N])) %//'
%// OR [y,x] = find(tril(true(size(V,2)),-1))
%// Use matrix indxeing to collect vector data for all combinations with those
%// x-y IDs from V. Then, perform subtractions across them for final output
diff_array = V(:,x) - V(:,y)
Few points about the code
bsxfun with find gets us the IDs for forming pairwise combinations.
We use those IDs to index into the 2D concatenated array and perform subtractions between them to get the final output.
Bonus Stuff
If you look closely into the part where it finds the IDs of all combinations, that is basically nchoosek(1:..,2).
So, basically one can have alternatives to nchoosek(1:N,2) as:
[Y,X] = find(bsxfun(#gt,[1:N]',[1:N]))
[y,x] = find(tril(true(N),-1))
with [X Y] forming those pairwise combinations and might be interesting to benchmark them!
I'm trying to represent a simple matrix m*n (let's assume it has only one row!) such that m1n1 = m1n1^1, m1n2 = m1n1^2, m1n3 = m1n1^3, m1n3 = m1n1^4, ... m1ni = m1n1^i.
In other words, I am trying to iterate over a matrix columns n times to add a new vector(column) at the end such that each of the indices has the same value as the the first vector but raised to the power of its column number n.
This is the original vector:
v =
1.2421
2.3348
0.1326
2.3470
6.7389
and this is v after the third iteration:
v =
1.2421 1.5429 1.9165
2.3348 5.4513 12.7277
0.1326 0.0176 0.0023
2.3470 5.5084 12.9282
6.7389 45.4128 306.0329
now given that I'm a total noob in Matlab, I really underestimated the difficulty of such a seemingly easy task, that took my almost a day of debugging and surfing the web to find any clue. Here's what I have come up with:
rows = 5;
columns = 3;
v = x(1:rows,1);
k = v;
Ncol = ones(rows,1);
extraK = ones(rows,1);
disp(v)
for c = 1:columns
Ncol = k(:,length(k(1,:))).^c; % a verbose way of selecting the last column only.
extraK = cat(2,extraK,Ncol);
end
k = cat(2,k,extraK);
disp(extraK(:,2:columns+1)) % to cut off the first column
now this code (for some weird reason) work only if rows = 6 or less, and columns = 3 or less.
when rows = 7, this is the output:
v = 1.0e+03 *
0.0012 0.0015 0.0019
0.0023 0.0055 0.0127
0.0001 0.0000 0.0000
0.0023 0.0055 0.0129
0.0067 0.0454 0.3060
0.0037 0.0138 0.0510
0.0119 0.1405 1.6654
How could I get it to run on any number of rows and columns?
Thanks!
I have found a couple of things wrong with your code:
I'm not sure as to why you are defining d = 3;. This is just nitpicking, but you can remove that from your code safely.
You are not doing the power operation properly. Specifically, look at this statement:
Ncol = k(:,length(k(1,:))).^c; % a verbose way of selecting the last column only.
You are selectively choosing the last column, which is great, but you are not applying the power operation properly. If I understand your statement, you wish to take the original vector, and perform a power operation to the power of n, where n is the current iteration. Therefore, you really just need to do this:
Ncol = k.^c;
Once you replace Ncol with the above line, the code should now work. I also noticed that you crop out the first column of your result. The reason why you are getting duplicate columns is because your for loop starts from c = 1. Since you have already computed v.^1 = v, you can just start your loop at c = 2. Change your loop starting point to c = 2, and you can get rid of the removal of the first column.
However, I'm going to do this in an alternative way in one line of code. Before we do this, let's go through the theory of what you're trying to do.
Given a vector v that is m elements long stored in a m x 1 vector, what you want is to have a matrix of size m x n, where n is the desired number of columns, and for each column starting from left to right, you wish to take v to the nth power.
Therefore, given your example from your third "iteration", the first column represents v, the second column represents v.^2, and the third column represents v.^3.
I'm going to introduce you to the power of bsxfun. bsxfun stands for Binary Singleton EXpansion function. What bsxfun does is that if you have two inputs where either or both inputs has a singleton dimension, or if either of both inputs has only one dimension which has value of 1, each input is replicated in their singleton dimensions to match the size of the other input, and then an element-wise operation is applied to these inputs together to produce your output.
For example, if we had two vectors like so:
A = [1 2 3]
B = [1
2
3]
Note that one of them is a row vector, and the other is a column vector. bsxfun would see that A and B both have singleton dimensions, where A has a singleton dimension being the number of rows being 1, and B having a singleton dimension which is the number of columns being 1. Therefore, we would duplicate B as many columns as there are in A and duplicate A for as many rows as there are in B, and we actually get:
A = [1 2 3
1 2 3
1 2 3]
B = [1 1 1
2 2 2
3 3 3]
Once we have these two matrices, you can apply any element wise operations to these matrices to get your output. For example, you could add, subtract, take the power or do an element wise multiplication or division.
Now, how this scenario applies to your problem is the following. What you are doing is you have a vector v, and you will have a matrix of powers like so:
M = [1 2 3 ... n
1 2 3 ... n
...........
...........
1 2 3 ... n]
Essentially, we will have a column of 1s, followed by a column of 2s, up to as many columns as you want n. We would apply bsxfun on the vector v which is a column vector, and another vector that is only a single row of values from 1 up to n. You would apply the power operation to achieve your result. Therefore, you can conveniently calculate your output by doing:
columns = 3;
out = bsxfun(#power, v, 1:columns);
Let's try a few examples given your vector v:
>> v = [1.2421; 2.3348; 0.1326; 2.3470; 6.7389];
>> columns = 3;
>> out = bsxfun(#power, v, 1:columns)
out =
1.2421 1.5428 1.9163
2.3348 5.4513 12.7277
0.1326 0.0176 0.0023
2.3470 5.5084 12.9282
6.7389 45.4128 306.0321
>> columns = 7;
>> format bank
>> out = bsxfun(#power, v, 1:columns)
out =
Columns 1 through 5
1.24 1.54 1.92 2.38 2.96
2.33 5.45 12.73 29.72 69.38
0.13 0.02 0.00 0.00 0.00
2.35 5.51 12.93 30.34 71.21
6.74 45.41 306.03 2062.32 13897.77
Columns 6 through 7
3.67 4.56
161.99 378.22
0.00 0.00
167.14 392.28
93655.67 631136.19
Note that for setting the columns to 3, we get what we see in your post. For pushing the columns up to 7, I had to change the way the numbers were presented so you can see the numbers clearly. Not doing this would put this into exponential form, and there were a lot of zeroes that followed the significant digits.
Good luck!
When computing cumulative powers you can reuse previous results: for scalar x and n, x.^n is equal to x * x.^(n-1), where x.^(n-1) has been already obtained. This may be more efficient than computing each power independently, because multiplication is faster than power.
Let N be the maximum exponent desired. To use the described approach, the column vector v is repeated N times horizontally (repmat), and then a cumulative product is applied along each row (cumprod):
v =[1.2421; 2.3348; 0.1326; 2.3470; 6.7389]; %// data. Column vector
N = 3; %// maximum exponent
result = cumprod(repmat(v, 1, N), 2);
Can anybody help me to find out the method to calculate the elements of different sized matrix in Matlab ?
Let say that I have 2 matrices with numbers.
Example:
A=[1 2 3;
4 5 6;
7 8 9]
B=[10 20 30;
40 50 60]
At first,we need to find maximum number in each column.
In this case, Ans=[40 50 60].
And then,we need to find ****coefficient** (k).
Coefficient(k) is equal to 1 divided by quantity of column of matrix A.
In this case, **coefficient (k)=1/3=0.33.
I wanna create matrix C filling with calculation.
Example in MS Excel.
H4 = ABS((C2-C6)/C9)*0.33+ABS((D2-D6)/D9)*0.33+ABS((E2-E6)/E9)*0.33
I4 = ABS((C3-C6)/C9)*0.33+ABS((D3-D6)/D9)*0.33+ABS((E3-E6)/E9)*0.33
J4 = ABS((C4-C6)/C9)*0.33+ABS((D4-D6)/D9)*0.33+ABS((E4-E6)/E9)*0.33
And then (Like above)
H5 = ABS((C2-C7)/C9)*0.33+ABS((D2-D7)/D9)*0.33+ABS((E2-E7)/E9)*0.33
I5 = ABS((C3-C7)/C9)*0.33+ABS((D3-D7)/D9)*0.33+ABS((E3-E7)/E9)*0.33
J5 = ABS((C4-C7)/C9)*0.33+ABS((D4-D7)/D9)*0.33+ABS((E4-E7)/E9)*0.33
C =
0.34 =|(1-10)|/40*0.33+|(2-20)|/50*0.33+|(3-30)|/60*0.33
0.28 =|(4-10)|/40*0.33+|(5-20)|/50*0.33+|(6-30)|/60*0.33
0.22 =|(7-10)|/40*0.33+|(8-20)|/50*0.33+|(9-30)|/60*0.33
0.95 =|(1-40)|/40*0.33+|(2-50)|/50*0.33+|(3-60)|/60*0.33
0.89 =|(4-40)|/40*0.33+|(5-50)|/50*0.33+|(6-60)|/60*0.33
0.83 =|(7-40)|/40*0.33+|(8-50)|/50*0.33+|(9-60)|/60*0.33
Actually A is a 15x4 matrix and B is a 5x4 matrix.
Perhaps,the matrices dimensions are more than this matrices (variables).
How can i write this in Matlab?
Thanks you!
You can do it like so. Let's assume that A and B are defined as you did before:
A = vec2mat(1:9, 3)
B = vec2mat(10:10:60, 3)
A =
1 2 3
4 5 6
7 8 9
B =
10 20 30
40 50 60
vec2mat will transform a vector into a matrix. You simply specify how many columns you want, and it will automatically determine the right amount of rows to transform the vector into a correctly shaped matrix (thanks #LuisMendo!). Let's also define more things based on your post:
maxCol = max(B); %// Finds maximum of each column in B
coefK = 1 / size(A,2); %// 1 divided by number of columns in A
I am going to assuming that coefK is multiplied by every element in A. You would thus compute your desired matrix as so:
cellMat = arrayfun(#(x) sum(coefK*(bsxfun(#rdivide, ...
abs(bsxfun(#minus, A, B(x,:))), maxCol)), 2), 1:size(B,1), ...
'UniformOutput', false);
outputMatrix = cell2mat(cellMat).'
You thus get:
outputMatrix =
0.3450 0.2833 0.2217
0.9617 0.9000 0.8383
Seems like a bit much to chew right? Let's go through this slowly.
Let's start with the bsxfun(#minus, A, B(x,:)) call. What we are doing is taking the A matrix and subtracting with a particular row in B called x. In our case, x is either 1 or 2. This is equal to the number of rows we have in B. What is cool about bsxfun is that this will subtract every row in A by this row called by B(x,:).
Next, what we need to do is divide every single number in this result by the corresponding columns found in our maximum column, defined as maxCol. As such, we will call another bsxfun that will divide every element in the matrix outputted in the first step by their corresponding column elements in maxCol.
Once we do this, we weight all of the values of each row by coefK (or actually every value in the matrix). In our case, this is 1/3.
After, we then sum over all of the columns to give us our corresponding elements for each column of the output matrix for row x.
As we wish to do this for all of the rows, going from 1, 2, 3, ... up to as many rows as we have in B, we apply arrayfun that will substitute values of x going from 1, 2, 3... up to as many rows in B. For each value of x, we will get a numCol x 1 vector where numCol is the total number of columns shared by A and B. This code will only work if A and B share the same number of columns. I have not placed any error checking here. In this case, we have 3 columns shared between both matrices. We need to use UniformOutput and we set this to false because the output of arrayfun is not a single number, but a vector.
After we do this, this returns each row of the output matrix in a cell array. We need to use cell2mat to transform these cell array elements into a single matrix.
You'll notice that this is the result we want, but it is transposed due to summing along the columns in the second step. As such, simply transpose the result and we get our final answer.
Good luck!
Dedication
This post is dedicated to Luis Mendo and Divakar - The bsxfun masters.
Assuming by maximum number in each column, you mean columnwise maximum after vertically concatenating A and B, you can try this one-liner -
sum(abs(bsxfun(#rdivide,bsxfun(#minus,permute(A,[3 1 2]),permute(B,[1 3 2])),permute(max(vertcat(A,B)),[1 3 2]))),3)./size(A,2)
Output -
ans =
0.3450 0.2833 0.2217
0.9617 0.9000 0.8383
If by maximum number in each column, you mean columnwise maximum of B, you can try -
sum(abs(bsxfun(#rdivide,bsxfun(#minus,permute(A,[3 1 2]),permute(B,[1 3 2])),permute(max(B),[1 3 2]))),3)./size(A,2)
The output for this case stays the same as the previous case, owing to the values of A and B.
I was trying to get the min values of a matrix before the max values of the matrix occurred. I have two matrices: matrix data and matrix a. Matrix a is a subset of matrix data and is composed of the max values of matrix data. I have the following code but obviously doing something wrong.
edit:
Matrix a are the max values of matrix data. I derived it from:
for x=1:size(data,1)
a(x)=max(data(x,:));
end
a=a'
clear x
matrix b code:
for x=1:size(data,1)
b(x)=min(data(x,(x<data==a)));
end
b=b'
clear x
matrix data matrix a matrix b
1 2 3 4 4 1
6 5 4 7 7 4
9 6 12 5 12 6
I need all the min values that occurred before to matrix a occurred in matrix data
Short and simple:
[a,idxmax] = max(data,[],2);
b = arrayfun(#(ii) min(data(ii,1:idxmax(ii))), 1:size(data,1));
which is the same as
b=NaN(1,size(data,1)); % preallocation!
for ii=1:size(data,1)
b(ii) = min(data(ii,1:idxmax(ii)));
end
Ignore maximum itself
If you want minimum of everything really before (and not including the maximum), it's possible that the maximum is the first number, and you try taking minimum of an empty matrix. Solution then is to use cell output, which can be empty:
b = arrayfun(#(ii) min(data(ii,1:idxmax(ii)-1)), 1:size(data,1),'uni',false);
Replace empty cells with NaN
If you want to replace empty cells to Nan and then back to a matrix use this:
b(cellfun(#isempty,b))={NaN};
b=cell2mat(b);
or simply use the earlier version and replace b(ii) with NaN when it is equal to a(ii) same outcome:
b = arrayfun(#(ii) min(data(ii,1:idxmax(ii))), 1:size(data,1));
b(b'==a) = NaN
Example:
data=magic(4)
16 2 3 13
5 11 10 8
9 7 6 12
4 14 15 1
outputs:
a' = 16 11 12 15
b =
16 5 6 4
and
b =[1x0 double] [5] [6] [4]
for the 2nd solution using cell output and ignoring the maximum itself also.
And btw:
for x=1:size(data,1)
a(x)=max(data(x,:));
end
a=a'
clear x
can be replaced with
a=max(data,[],2);
It's not pretty but this is the only way I found so far of doing this kind of thing without a loop.
If loops are ok I would recommend Gunther Struyf answer as the most compact use of matlab's in-built array looping function, arrayfun.
Some of the transposition etc may be superfluous if you're wanting column mins instead of row...
[mx, imx] = max(data');
inds = repmat(1:size(data,2), [size(data,1),1]);
imx2 = repmat(imx', [1, size(data,2)]);
data2 = data;
data2(inds >= imx2) = inf;
min(data2');
NOTE: if data is not needed we can remove the additional data2 variable, and reduce the line count.
So to demonstrate what this does, (and see if I understood the question correctly):
for input
>> data = [1,3,-1; 5,2,1]
I get minima:
>> min(data2')
ans = [1, inf]
I.e. it only found the min values before the max values for each row, and anything else was set to inf.
In words:
For each row get index of maximum
Generate matrix of column indices
Use repmat to generate a matrix, same size as data where each row is index of maximum
Set data to infinity where column index > max_index matrix
find min as usual.