I have two tables (or actually I have several) that should be merged using innerjoin. It works as expected; except that it sort on the "Keys", which destroys my data since it actually must be arranged in the original row order.
From the help section:
C is sorted by the values in the key variables
t1 = Case Val
3 1
1 1
2 1
t2 = Val2 Case Subset
2 2 2
1 2 2
2 3 1
tnew = innerjoin(t1,t2)
tnew = Case Val Val2 Subset
2 ...
% will start with "2" since its a lower value than "3", but in t1 "3" was in lower row than "2", it is rearranged
How should I avoid the sorting? Hopeless to use innerjoin?
In addition to the resulting table, innerjoin returns two extra outputs: The row indices of the first table and the row indices of the second table that correspond to the rows in the output.
You can simply use the second output to determine the rows in t1 that were used and you could sort these. Then use the sort order to change the ordering of the rows in the result of the join.
%// Setup the data
t1 = table([3;1;2], [1;1;1], 'VariableNames', {'Case', 'Val'});
t2 = table([2;1;2],[2;2;3],[2;2;1], 'VariableNames', {'Val2', 'Case', 'Subset'});
%// Perform the inner join and keep track of where the rows were in t1
[tnew, rows_in_t1] = innerjoin(t1, t2);
%// Sort them in order to maintain the order in t1
[~, sortinds] = sort(rows_in_t1);
%// Apply this sort order to the new table
tnew = tnew(sortinds,:);
%// Case Val Val2 Subset
%// ____ ___ ____ ______
%// 3 1 2 1
%// 2 1 2 2
%// 2 1 1 2
Related
I have the following MATLAB table
item_a item_b score
a b 1
a b 1
b c 3
d e 2
d e 1
d e 0
I want to average the redundant rows. The desired result is as follows:
item_a item_b score
a b (1+1)/2
b c 3
d e (2+1+0)/3
This is a classic scenario for the findgroups, split-apply workflow.
Given your table named t:
% Find mean values.
G = findgroups(t.item_a);
meanValues = splitapply(#mean,t.score,G);
% Create new table.
[~,i] = unique(G);
newTable = t(i,:);
newTable.score = meanValues
newTable contains the desired table.
See this documentation page for more examples.
This is what I got. You can tweak with the final results. There is a similar example on MATLAB documentation. Here are two key functions, accumarray and unique. Note that this solution works only for array inputs not cell data types. By manipulating data types, you can also find the solution for table and cell data types. Otherwise, I think for loop will be necessary.
items = ['a' 'b'
'a' 'b'
'b' 'c'
'd' 'e'
'd' 'e'
'd' 'e' ];
scores = [1 1 3 2 1 0]';
[items_unique,ia,ic] = unique(items,'rows');
score_mean = accumarray(ic,scores, [], #mean);
result = {items_unique score_mean};
I have two tables with same variables. One table contain one row while other table contain more than one rows.
a=[1 2; 2 3],b=[2 3; 1 2]
S1=table(a,b)
a=[1 1],b=[1 1]
S2=table(a,b)
if all(S2{:,:}<S1{:,:}) & any(S2{:,:}<=S1{:,:})
S1=[S1;S2]
end
Where is the mistake in specifying table or cell? Even the conversion table2cell, table2struct, table2array did not work (getvar error was shown).
Table values are fixed. There is no addition, no replacement, but the appending only when the condition is satisfied. Final output is table with values as shown.
S1 = 3×2 table
a b
______ ______
1 2 2 3
2 3 1 2
1 1 1 1
The error is due to the fact that you are trying to compare two set of data (S1 and S2) that have different size.
S2{:,:}
1 1 1 1
S1{:,:}
1 2 2 3
2 3 1 2
If you want to compare each row of S1 against S2 you can use the function bsxfun:
to check S2 < S1
bsxfun(#lt,S2{:,:},S1{:,:})
to check S2 <= S1
bsxfun(#le,S2{:,:},S1{:,:})
This will lead to:
if all(bsxfun(#lt,S2{:,:},S1{:,:})) & any(bsxfun(#le,S2{:,:},S1{:,:}))
S1=[S1;S2]
end
In the following toy example, tables t1 and t2 have shapes (3 x 0) and (3 x 1), respectively. Furthermore, both tables have the same row names.
>> t1 = table('RowNames', {'a', 'b', 'c'});
>> t2 = table([3 ; 2 ; 1], ...
'RowNames', {'c', 'a', 'b'}, 'VariableNames', {'x'});
Then a copy of t2's single column is added to t1 as a new column, with the same variable name.
>> t1.('x') = t2.('x');
The resulting table t1, however, differs from t2 in the association between row names and the values in the x-column:
>> t1({'a', 'b', 'c'}, :)
ans =
x
_
a 3
b 2
c 1
>> t2({'a', 'b', 'c'}, :)
ans =
x
_
a 2
b 1
c 3
What's the simplest way to assign t2.('x') to t1.('x') "respecting rownames"? By this last condition I mean that the final t1 should look just like t2; e.g.:
>> t1({'a', 'b', 'c'}, :)
ans =
x
_
a 2
b 1
c 3
You can index the table using row names so if you extract the list of rownames from t1 you can use that as the ordering for t2:
order = t1.Properties.RowNames % cell array
intermediate = t2(order, :);
or just do it all in one go:
t2(t1.Properties.RowNames, :);
Since t1 doesn't have the x column you can concatenate t1 with column x of t2
>> t1=[t1, t2(:,'x')]
t1 =
x
_
a 2
b 1
c 3
It will automatically take care of matching rows.
OK, this is the OP here.
I found a (potential) answer to my question: instead of
t1.('x') = t2.('x');
use
t1.('x') = t2{t1.Properties.RowNames, 'x'};
I say that this is a "potential" answer because with MATLAB I never know when something that works for a particular type, or under particular circumstances, will generalize. E.g., at this point I don't know if the above will work if column x holds non-numeric values.
If someone knows of a better way, or can point to documentation in support of my lucky guess here, please post it. I'll be glad to accept it as the answer.
I have the following table
name = ['A' 'A' 'A' 'B' 'B' 'C' 'C' 'C' 'C' 'D' 'D' 'E' 'E' 'E']';
value = randn(14, 1);
T = table(name, value);
i,e.
T =
name value
____ _________
A 0.0015678
A -0.76226
A 0.98404
B -1.0942
B 0.71249
C 1.688
C 1.4001
C -0.9278
C -1.3725
D 0.11563
D 0.076776
E 1.0568
E 1.1972
E 0.29037
I want to transform it in the following way: take the first two cells in value corresponding to different values in name and put it in the 5x2 matrix. This matrix would have rows corresponding to different names A,B,C,D,E and columns corresponding to values, e.g. the first two rows are
0.0015678 -0.76226
-1.0942 0.71249
This can be done with accumarray using a custom function. The first step is to convert the name column of T into a numeric vector; and then accumarray can be applied.
This approach requires T being sorted according to column 1, because only in this case is accumarray guaranteed to preserve order (as indicated in its documentation). So if T may not be sorted (although it is in your example), sort it first using sortrows.
T = sortrows(T, 1); %// you can remove this line if T is guaranteed to be sorted
[~, ~, names] = unique(T(:,1)); %// names as a numeric vector
result = cell2mat(accumarray(names, T.value, [], #(x) {x([1 2]).'}));
First figure out where each name has values located in the table, then cycle through each name and place the first two values encountered for each name into individual cell arrays. Once you're done, reshape the matrix to 5 x 2 as you have said. As such, do something like this:
names = unique(T.name); %// 1
ind = arrayfun(#(x) find(T.name == x), names, 'uni', 0); %// 2
vals = cellfun(#(x) T.value(x(1:2)), ind, 'uni', 0); %// 3
m = [vals{:}].'; %// 4
Let's go through each line of code slowly.
Line #1
The first line finds all unique names through unique and we store them into names.
Line #2
The next line goes through all of the unique names and finds those locations / rows in the table that share that particular name. I use arrayfun and go through each name in names, find those rows that share the same name as one we are looking for, and place those row locations into individual cells; these are stored into ind. To find the locations of each valid name in our table, I use find and the locations are placed into a column vector. As such, we will have five column vectors where each column vector is placed into an individual cell. These column vectors will tell us which rows match a particular name located in your table.
Line #3
The next line uses cellfun to go through each of the cells in ind and extracts the first two row locations that share a particular name, indexes into the value field for your table to pull those two values, and these are placed as two-element vectors into individual cells for each name.
Line #4
The last line of code simply unrolls each two-element vector. The first two elements of each name get stored into columns. To get them into rows, I simply transpose the unrolling. The output matrix is stored into m.
If you want to see what the output looks like, this is what I get when I run the above code with your example table:
m =
0.0016 -0.7623
-1.0942 0.7125
1.6880 1.4001
0.1156 0.0768
1.0568 1.1972
Be advised that I only showed the first 5 digits of precision so there is some round-off at the end. However, this is only for display purposes and so what I got is equivalent to what your expect for the output.
Hope this helps!
If you want use the tables, you could try something like this:
count = 1;
U = unique(table2array(T(:,1)));
for ii = 1:size(U,1)
A = find(table2array(T(:,1)) == U(ii));
A = A(1:2);
B(count,1:2) = table2array(T(A,2));
count = count + 1;
end
Personally, I would find this simpler to do with your name and value arrays and forget about the table. If it is a requirement then I understand, however I will provide my solution still. It may provide some insight either way.
count = 1;
U = unique(name);
for ii = 1:size(U,1)
A = find(name == U(ii));
A = A(1:2);
B(count,1:2) = value(A);
count = count + 1;
end
Quick and dirty, but hopefully it's good enough. Good luck.
Another solution that is more manageable and easily scalable exists. Since MATLAB R2013b you can use a specialized function for pivoting a table (which is what you want to do): unstack.
In order to get exactly what you wanted, you need to add an extra variable to your table that will indicate replications:
name = ['A' 'A' 'A' 'B' 'B' 'C' 'C' 'C' 'C' 'D' 'D' 'E' 'E' 'E']';
value = randn(14, 1);
rep = [1, 2, 3, 1, 2, 1, 2, 3, 4, 1, 2, 1, 2, 3];
T = table(name, value, rep);
T =
name value rep
____ _________ ___
A 0.53767 1
A 1.8339 2
A -2.2588 3
B 0.86217 1
B 0.31877 2
C -1.3077 1
C -0.43359 2
C 0.34262 3
C 3.5784 4
D 2.7694 1
D -1.3499 2
E 3.0349 1
E 0.7254 2
E -0.063055 3
Then you just use unstack like this:
pivotTable = unstack(T, 'value','name')
pivotTable =
rep A B C D E
___ _______ _______ ________ _______ _________
1 0.53767 0.86217 -1.3077 2.7694 3.0349
2 1.8339 0.31877 -0.43359 -1.3499 0.7254
3 -2.2588 NaN 0.34262 NaN -0.063055
4 NaN NaN 3.5784 NaN NaN
Afterwards, it's a matter of re-arranging the table if you still want to.
The easiest way is to first convert the table into a matrix form and then reshape it by using the "reshape" function in Matlab.
matrix = t{:,:};% t-- your table variable
reshape_matrix = reshape(matrix ,[2,3]) % [2,3]--> the size of the matrix you desire
These two steps can be done by one line of code
reshape_matrix = reshape(t{:,:},[2,3]);
I have more 8+ tables in Matlab of different lengths. They all include dates in the their first column. I would like to get the intersection of all these tables on the date columns. The following small example with 3 tables shows what I want:
Date1=datenum(2011,6,7,0:24:240,0,0).';
Date2=datenum(2011,6,8,0:24:240,0,0).';
Date3=datenum(2011,6,5,0:24:240,0,0).';
T1 = table(Date1,ones(11,1),'VariableNames',{'Date','Var1'})
T2 = table(Date2,ones(11,1)*2,'VariableNames',{'Date','Var2'})
T3 = table(Date3,ones(11,1)*3,'VariableNames',{'Date','Var3'})
Thus, I would like the following output:
Date Var1 Var2 Var3
______ ____ ____ ____
734662 1 2 3
734663 1 2 3
734664 1 2 3
734665 1 2 3
734666 1 2 3
734667 1 2 3
734668 1 2 3
734669 1 2 3
Is there a function in Matlab that can do this?
The intersect function may be of use for your case.
I am not sure how you use the tables, but the code below should let you find the indices of the intersection for each DateN vector. Once you have the indices, you can rebuild a global table which only incorporate the common indices between all tables.
[C,ia,ib ] = intersect(Date1,Date2) ; %// get indices of intersection of first 2 vectors (Date1&2)
[D,ja,ix3] = intersect( C ,Date3) ; %// get indices of intersection of last result (Date1&2) with last vector (Date 3)
ix1 = ia(ja) ; %// take only the common indices of the 2 intersection operations (For Date1)
ix2 = ib(ja) ; %// take only the common indices of the 2 intersection operations (For Date2)
%// Build the "common" table
intersectionTable = [ Date1(ix1) Var1(ix1) Var2(ix2) Var3(ix3) ] ;
I wrote a very closely related code for another question and I am putting down a polished function code here that finds the intersecting elements and the corresponding indices for several arrays at the same time without resorting to any kind of looping for the computation part. Here's the code -
function [out_val,out_idx] = intersect_arrays(varargin)
%// Concatenate all vector arrays into a 2D array
if isrow(varargin{1})
M = vertcat(varargin{:});
else
M = horzcat(varargin{:}).'; %//'
end
%// Find unique values for all elements in all arrays
unqvals = unique(M(:),'stable')'; %//'
%// Find unqiue elements common across all arrays (intersecting elements)
out_val = unqvals(all(any(bsxfun(#eq,M,permute(unqvals,[1 3 2])),2),1));
%// Find first indices across all arrays holding the intersecting elements
[~,idx] = max(bsxfun(#eq,M,permute(out_val,[1 3 2])),[],2);
out_idx = squeeze(idx).'; %//'
return
Now, to solve your case, we can use the function code like so -
num_arrays = 3; %// Number of arrays to be used
%// Find intersecting elements and their corresppinding indices in each array
[int_ele,int_idx] = intersect_arrays(T1.Date,T2.Date,T3.Date) %// Add inputs here
%// Create an array of all Var data
all_idx = cat(2,T1.Var1,T2.Var2,T3.Var3) %// Add inputs here
%// Select Var data based on the intersecting indices
select_idx = all_idx(bsxfun(#plus,int_idx,[0:num_arrays-1]*size(T1.Var1,1)))
%// Output results as a numeric array and table
out_array = [int_ele(:) select_idx]
out_table = cell2table(num2cell(out_array),'VariableNames',...
{'Date','Var1','Var2','Var3'})
Output -
out_table =
Date Var1 Var2 Var3
______ ____ ____ ____
734662 1 2 3
734663 1 2 3
734664 1 2 3
734665 1 2 3
734666 1 2 3
734667 1 2 3
734668 1 2 3
734669 1 2 3