As an example, I have a matrix [1,2,3,4,5]'. This matrix contains one column and 5 rows, and I have to generate a pair of points like (1,2),(1,3)(1,4)(1,5),(2,3)(2,4)(2,5),(3,4)(3,5)(4,5).
I have to store these values in 2 columns in a matrix. I have the following code, but it isn't quite giving me the right answer.
for s = 1:5;
for tb = (s+1):5;
if tb>s
in = sub2ind(size(pairpoints),(tb-1),1);
pairpoints(in) = s;
in = sub2ind(size(pairpoints),(tb-1),2);
pairpoints(in) = tb;
end
end
end
With this code, I got (1,2),(2,3),(3,4),(4,5). What should I do, and what is the general formula for the number of pairs?
One way, though is limited depending upon how many different elements there are to choose from, is to use nchoosek as follows
pairpoints = nchoosek([1:5],2)
pairpoints =
1 2
1 3
1 4
1 5
2 3
2 4
2 5
3 4
3 5
4 5
See the limitations of this function in the provided link.
An alternative is to just iterate over each element and combine it with the remaining elements in the list (assumes that all are distinct)
pairpoints = [];
data = [1:5]';
len = length(data);
for k=1:len
pairpoints = [pairpoints ; [repmat(data(k),len-k,1) data(k+1:end)]];
end
This method just concatenates each element in data with the remaining elements in the list to get the desired pairs.
Try either of the above and see what happens!
Another suggestion I can add to the mix if you don't want to rely on nchoosek is to generate an upper triangular matrix full of ones, disregarding the diagonal, and use find to generate the rows and columns of where the matrix is equal to 1. You can then concatenate both of these into a single matrix. By generating an upper triangular matrix this way, the locations of the matrix where they're equal to 1 exactly correspond to the row and column pairs that you are seeking. As such:
%// Highest value in your data
N = 5;
[rows,cols] = find(triu(ones(N),1));
pairpoints = [rows,cols]
pairPoints =
1 2
1 3
2 3
1 4
2 4
3 4
1 5
2 5
3 5
4 5
Bear in mind that this will be unsorted (i.e. not in the order that you specified in your question). If order matters to you, then use the sortrows command in MATLAB so that we can get this into the proper order that you're expecting:
pairPoints = sortrows(pairPoints)
pairPoints =
1 2
1 3
1 4
1 5
2 3
2 4
2 5
3 4
3 5
4 5
Take note that I specified an additional parameter to triu which denotes how much of an offset you want away from the diagonal. The default offset is 0, which includes the diagonal when you extract the upper triangular matrix. I specified 1 as the second parameter because I want to move away from the diagonal towards the right by 1 unit so I don't want to include the diagonal as part of the upper triangular decomposition.
for loop approach
If you truly desire the for loop approach, going with your model, you'll need two for loops and you need to keep track of the previous row we are at so that we can just skip over to the next column until the end using this. You can also use #GeoffHayes approach in using just a single for loop to generate your indices, but when you're new to a language, one key advice I will always give is to code for readability and not for efficiency. Once you get it working, if you have some way of measuring performance, you can then try and make the code faster and more efficient. This kind of programming is also endorsed by Jon Skeet, the resident StackOverflow ninja, and I got that from this post here.
As such, you can try this:
pairPoints = []; %// Initialize
N = 5; %// Highest value in your data
for row = 1 : N
for col = row + 1 : N
pairPoints = [pairPoints; [row col]]; %// Add row-column pair to matrix
end
end
We get the equivalent output:
pairPoints =
1 2
1 3
1 4
1 5
2 3
2 4
2 5
3 4
3 5
4 5
Small caveat
This method will only work if your data is enumerated from 1 to N.
Edit - August 20th, 2014
You wish to generalize this to any array of values. You also want to stick with the for loop approach. You can still keep the original for loop code there. You would simply have to add a couple more lines to index your new array. As such, supposing your data array was:
dat = [12, 45, 56, 44, 62];
You would use the pairPoints matrix and use each column to subset the data array to access your values. Also, you need to make sure your data is a column vector, or this won't work. If we didn't, we would be creating a 1D array and concatenating rows and that's not obviously what we're looking for. In other words:
dat = [12, 45, 56, 44, 62];
dat = dat(:); %// Make column vector - Important!
N = numel(dat); %// Total number of elements in your data array
pairPoints = []; %// Initialize
%// Skip if the array is empty
if (N ~= 0)
for row = 1 : N
for col = row + 1 : N
pairPoints = [pairPoints; [row col]]; %// Add row-column pair to matrix
end
end
vals = [dat(pairPoints(:,1)) dat(pairPoints(:,2))];
else
vals = [];
Take note that I have made a provision where if the array is empty, don't even bother doing any calculations. Just output an empty matrix.
We thus get:
vals =
12 45
12 56
12 44
12 62
45 56
45 44
45 62
56 44
56 62
44 62
Related
In matrix A, every column represents an output variable and every row represents a reading (6 rows in total). Every output has a certain subgroup size (groups of 3 rows). I need A's elements to be sorted in the vertical direction within every subgroup.
A = [ 1 7 4; 4 9 3; 8 5 7; 2 9 1; 7 4 4; 8 1 3];
% consecutive 3 rows is one subgroup, within which sorting is required.
B = [1 5 3; 4 7 4; 8 9 7; 2 1 1; 7 4 3; 8 9 4]; % the expected result.
I was considering something along the lines of B = splitapply(#sort,A,2), but splitapply cannot be called like that. How can I get the desired result?
Please note, the actual matrix contains 8 columns and 300 rows. An example is demonstrated above.
The easiest solution would be to reshape your data, sort, then permute it:
rps = 3; % rows per subgroup
B = permute(sort(reshape(A.',rps,size(A,2),[]),2),[2 1 3]);
The above results in a 3x3x2 arrays, which in my opinion are easier to work with, but if you want the output as in the example, you can do the following:
B = reshape(permute(sort(reshape(A.',rps,size(A,2),[]),2),[2 3 1]),size(A));
Alternatively, you are correct to think that splitapply can be useful here, but it requires a bit more work.
This command works on the sample data and should also work on your full dataset:
b = cell2mat( splitapply( #(x){sort(x,2).'}, A.', repelem( 1:size(A,1)/rps, rps ) ).' );
I'll explain what this does:
repelem( 1:size(A,1)/rps, rps ) returns a row vector of groups. The amount of groups is the total amount of rows divided by the group size. (For good measure, there should be an assertion that this is divisible with no remainder).
splitapply( #(x){sort(x,2).'}, ... since splitapply has to return a scalar object per group, it needs to be told that the output is a cell so that it can return a matrix. (This might not be the best explanation, but if you attempt to run it w/o a cell output, you will get the following error:
The function 'sort' returned a non-scalar value when applied to the 1st group of data.
To compute nonscalar values for each group, create an anonymous function to return each value in a scalar cell:
#(x1){sort(x1)}
I perform several transpose operations since this is what splitapply expects.
I used cell2mat to convert the output cells back to a numeric array.
This question is motivated by very specific combinatorial optimization problem, where search space is defined as a space of permuted subsets of vector unsorted set of discrete values with multiplicities.
I am looking for effective (fast enough, vectorized or any other more clever solution) function which is able to find indices of subsets in the following manner:
t = [1 1 3 2 2 2 3 ]
is unsorted vector of all possible values, including its multiplicities.
item = [2 3 1; 2 1 2; 3 1 1; 1 3 3]
is a list of permuted subsets of vector t.
I need to find list of corresponding indices of subsets item which corresponds to the vector t. So, for above mentioned example we have:
item =
2 3 1
2 1 2
3 1 1
1 3 3
t =
1 1 3 2 2 2 3
ind = item2ind(item,t)
ind =
4 3 1
4 1 5
3 1 2
1 3 7
So, for item = [2 3 1] we get ind = [4 3 1], which means, that:
first value "2" at item corresponds to the first value "2" at t on position "4",
second value "3" at item corresponds to the first value "3" at t on position "3" and
third value "1" at item corresponds to the first value "1" at t on position "1".
In a case item =[ 2 1 2] we get ind = [4 1 5], which means, that:
first value "2" at item corresponds to the first value "2" at t on position "4",
second value "1" at item corresponds to the first value "1" at t on position "1", and
third value "2" at item corresponds to the second(!!!) value "1" at t on position "5".
For
item = [1 1 1]
does not exist any solution, because vector t contains only two "1".
My current version of function "item2ind" is very trivial serial code, which is possible simple parallelized by changing of "for" to "parfor" loop:
function ind = item2ind(item,t)
[nlp,N] = size(item);
ind = zeros(nlp,N);
for i = 1:nlp
auxitem = item(i,:);
auxt = t;
for j = 1:N
I = find(auxitem(j) == auxt,1,'first');
if ~isempty(I)
auxt(I) = 0;
ind(i,j) = I;
else
error('Incompatible content of item and t.');
end
end
end
end
But I need something definitely more clever ... and faster:)
Test case for larger input data:
t = 1:10; % 10 unique values at vector t
t = repmat(t,1,5); % unsorted vector t with multiplicity of all unique values 5
nlp = 100000; % number of item rows
[~,p] = sort(rand(nlp,length(t)),2); % 100000 random permutations
item = t(p); % transform permutations to items
item = item(:,1:30); % transform item to shorter subset
tic;ind = item2ind(item,t);toc % runing and timing of the original function
tic;ind_ = item2ind_new(item,t);toc % runing and timing of the new function
isequal(ind,ind_) % comparison of solutions
To achieve vectorizing the code, I have assumed that the error case won't be present. It should be discarded first, with a simple procedure I will present below.
Method First, let's compute the indexes of all elements in t:
t = t(:);
mct = max(accumarray(t,1));
G = accumarray(t,1:length(t),[],#(x) {sort(x)});
G = cellfun(#(x) padarray(x.',[0 mct-length(x)],0,'post'), G, 'UniformOutput', false);
G = vertcat(G{:});
Explanation: after putting input in column vector shape, we compute the max number of occurences of each possible value in t using accumarray. Now, we form array of all indexes of all numbers. It forms a cell array as there may be not the same number of occurences for each value. In order to form a matrix, we pad each array independently to the max length (naming mct). Then we can transform the cell array into a matrix. At this step, we have:
G =
1 11 21 31 41
2 12 22 32 42
3 13 23 33 43
4 14 24 34 44
5 15 25 35 45
6 16 26 36 46
7 17 27 37 47
8 18 28 38 48
9 19 29 39 49
10 20 30 40 50
Now, we process item. For that, let's figure out how to create the cumulative sum of occurences of values inside a vector. For example, if I have:
A = [1 1 3 2 2 2 3];
then I want to get:
B = [1 2 1 1 2 3 2];
Thanks to implicit expansion, we can have it in one line:
B = diag(cumsum(A==A'));
As easy as this. The syntax A==A' expands into a matrix where each element is A(i)==A(j). Making the cumulative sum in only one dimension and taking the diagonal gives us the good result: each column in the cumulative sum of occurences over one value.
To use this trick with item which 2-D, we should use a 3D array. Let's call m=size(item,1) and n=size(item,2). So:
C = cumsum(reshape(item,m,1,n)==item,3);
is a (big) 3D matrix of all cumulatives occurences. Last thing is to select the columns that are on the diagonal along dimension 2 and 3:
ia = C(sub2ind(size(C),repelem((1:m).',1,n),repelem(1:n,m,1),repelem(1:n,m,1)));
Now, with all these matrices, indexing is easy:
ind = G(sub2ind(size(G),item,ia));
Finally, let's recap the code of the function:
function ind = item2ind_new(item,t)
t = t(:);
[m,n] = size(item);
mct = max(accumarray(t,1));
G = accumarray(t,1:length(t),[],#(x) {sort(x)});
G = cellfun(#(x) padarray(x.',[0 mct-length(x)],0,'post'), G, 'UniformOutput', false);
G = vertcat(G{:});
C = cumsum(reshape(item,m,1,n)==item,3);
ia = C(sub2ind(size(C),repelem((1:m).',1,n),repelem(1:n,m,1),repelem(1:n,m,1)));
ind = G(sub2ind(size(G),item,ia));
Results Running the provided script on an old 4-core, I get:
Elapsed time is 4.317914 seconds.
Elapsed time is 0.556803 seconds.
ans =
logical
1
Speed up is substential (more than 8x), along with memory consumption (with matrix C). I guess some improvements can be done with this part to save more memory.
EDIT For generating ia, this procedure can cost a lost of memory. A way to save memory is to use a for-loop to generate directly this array:
ia = zeros(size(item));
for i=unique(t(:)).'
ia = ia+cumsum(item==i, 2).*(item==i);
end
In all cases, when you have ia, it's easy to test if there is an error in item compared to t:
any(ind(:)==0)
A simple solution to get items in error (as a mask) is then
min(ind,[],2)==0
I have a matrix like this:
fd =
x y z
2 5 10
2 6 10
3 5 11
3 9 11
4 3 11
4 9 12
5 4 12
5 7 13
6 1 13
6 5 13
I have two parts of my problem:
1) I want to calculate the difference of each two elements in a column.
So I tried the following code:
for i= 1:10
n=10-i;
for j=1:n
sdiff1 = diff([fd(i,1); fd(i+j,1)],1,1);
sdiff2 = diff([fd(i,2); fd(i+j,2)],1,1);
sdiff3 = diff([fd(i,3); fd(i+j,3)],1,1);
end
end
I want all the differences such as:
x1-x2, x1-x3, x1-x4....x1-x10
x2-x3, x2-x4.....x2-x10
.
.
.
.
.
x9-x10
same for y and z value differences
Then all the values should stored in sdiff1, sdiff2 and sdiff3
2) what I want next is for same z values, I want to keep the original data points. For different z values, I want to merge those points which are close to each other. By close I mean,
if abs(sdiff3)== 0
keep the original data
for abs(sdiff3) > 1
if abs(sdiff1) < 2 & abs(sdiff2) < 2
then I need mean x, mean y and mean z of the points.
So I tried the whole programme as:
for i= 1:10
n=10-i;
for j=1:n
sdiff1 = diff([fd(i,1); fd(i+j,1)],1,1);
sdiff2 = diff([fd(i,2); fd(i+j,2)],1,1);
sdiff3 = diff([fd(i,3); fd(i+j,3)],1,1);
if (abs(sdiff3(:,1)))> 1
continue
mask1 = (abs(sdiff1(:,1)) < 2) & (abs(sdiff2(:,1)) < 2) & (abs(sdiff3:,1)) > 1);
subs1 = cumsum(~mask1);
xmean1 = accumarray(subs1,fd(:,1),[],#mean);
ymean1 = accumarray(subs1,fd(:,2),[],#mean);
zmean1 = accumarray(subs1,fd(:,3),[],#mean);
fd = [xmean1(subs1) ymean1(subs1) zmean1(subs1)];
end
end
end
My final output should be:
2.5 5 10.5
3.5 9 11.5
5 4 12
5 7 13
6 1 13
where, (1,2,3),(4,6),(5,7,10) points are merged to their mean position (according to the threshold difference <2) whereas 8 and 9th point has their original data.
I am stuck in finding the differences for each two elements of a column and storing them. My code is not giving me the desired output.
Can somebody please help?
Thanks in advance.
This can be greatly simplified using vectorised notation. You can do for instance
fd(:,1) - fd(:,2)
to get the difference between columns 1 and 2 (or equivalently diff(fd(:,[1 2]), 1, 2)). You can make this more elegant/harder to read and debug with pdist but if you only have three columns it's probably more trouble than it's worth.
I suspect your first problem is with the third argument to diff. If you use diff(X, 1, 1) it will do the first order diff in direction 1, which is to say between adjacent rows (downwards). diff(X, 1, 2) will do it between adjacent columns (rightwards), which is what you want. Matlab uses the opposite convention to spreadsheets in that it indexes rows first then columns.
Once you have your diffs you can then test the elements:
thesame = find(sdiff3 < 2); % for example
this will yield a vector of the row indices of sdiff3 where the value is less than 2. Then you can use
fd(thesame,:)
to select the elements of fd at those indexes. To remove matching rows you would do the opposite test
notthesame = find(sdiff > 2);
to find the ones to keep, then extract those into a new array
keepers = fd(notthesame,:);
These won't give you the exact solution but it'll get you on the right track. For the syntax of these commands and lots of examples you can run e.g. doc diff in the command window.
I have a <206x193> matrix A. It contains the values of a parameter at 206 different locations at 193 time steps. I am interested in the maximum value at each location over all times as well as the corresponding indices. I have another matrix B with the same dimensions of A and I'm interested in values for each location at the time that A's value at that location was maximal.
I've tried [max_val pos] = max(A,[],2), which gives the right maximum values, but A(pos) does not equal max_val.
How exactly does this function work?
I tried a smaller example as well. Still I don't understand the meaning of the indices....
>> H
H(:,:,1) =
1 2
3 4
H(:,:,2) =
5 6
7 8
>> [val pos] = max(H,[],2)
val(:,:,1) =
2
4
val(:,:,2) =
6
8
pos(:,:,1) =
2
2
pos(:,:,2) =
2
2
The indices in idx represent the index of the max value in the corresponding row. You can use sub2ind to create a linear index if you want to test if A(pos)=max_val
A=rand(206, 193);
[max_val, idx]=max(A, [], 2);
A_max=A(sub2ind(size(A), (1:size(A,1))', idx));
Similarly, you can access the values of B with:
B_Amax=B(sub2ind(size(A), (1:size(A,1))', idx));
From your example:
H(:,:,2) =
5 6
7 8
[val pos] = max(H,[],2)
val(:,:,2) =
6
8
pos(:,:,2) =
2
2
The reason why pos(:,:,2) is [2; 2] is because the maximum is at position 2 for both rows.
max is a primarily intended for use with vectors. In normal mode, even the multi-dimensional arrays are treated as a series of vectors along which the max function is applied.
So, to get the values in B at each location at the time where A is maximum, you should
// find the maximum values and positions in A
[c,i] = max(A, [], 2);
// iterate along the first dimension, to retrieve the corresponding values in B
C = [];
for k=1:size(A,1)
C(k) = B(k,i(k));
end
You can refer to #Jigg's answer for a more concise way of creating matrix C
How can I construct a scrambled matrix with 128 rows and 32 columns in vb.net or Matlab?
Entries of the matrix are numbers between 1 and 32 with the condition that each row mustn't contain duplicate elements and rows mustn't be duplicates.
This is similar to #thewaywewalk's answer, but makes sure that the matrix has no repeated rows by testing if it does and in that case generating a new matrix:
done = 0;
while ~done
[~, matrix] = sort(rand(128,32),2);
%// generate each row as a random permutation, independently of other rows.
%// This line was inspired by randperm code
done = size(unique(matrix,'rows'),1) == 128;
%// in the event that there are repeated rows: generate matrix again
end
If my computations are correct, the probability that the matrix has repteated rows (and thus has to be generated again) is less than
>> 128*127/factorial(32)
ans =
6.1779e-032
Hey, it's more likely that a cosmic ray will spoil a given run of the program! So I guess you can safely remove the while loop :-)
With randperm you can generate one row:
row = randperm(32)
if this vector wouldn't be that long you could just use perms to find all permutations:
B = perms(randperm(32))
but it's memory-wise too much! ( 32! = 2.6313e+35 rows )
so you can use a little loop:
N = 200;
A = zeros(N,32);
for ii = 1:N
A(ii,:) = randperm(32);
end
B = unique(A, 'rows');
B = B(1:128,:);
For my tests it was sufficient to use N = 128 directly and skip the last two lines, because with 2.6313e+35 possibly permutations the probability that you get a correct matrix with the first try is very high. But to be sure that there are no row-duplicates choose a higher number and select the first 128 rows finally. In case the input vector is relatively short and the number of desired rows close to the total number of possible permutations use the proposed perms(randperm( n )).
small example for intergers from 1 to 4 and a selection of 10 out of 24 possible permutations:
N = 20;
A = zeros(N,4);
for ii = 1:N
A(ii,:) = randperm(4);
end
B = unique(A, 'rows');
B = B(1:10,:);
returns:
B =
1 2 3 4
1 2 4 3
1 3 4 2
2 3 1 4
2 3 4 1
2 4 1 3
2 4 3 1
3 1 2 4
3 1 4 2
3 2 1 4
some additional remarks for the choice of N:
I made some test runs, where I used the loop above to find all permutations like perms does. For vector lengths of n=4 to n=7 and in each case N = factorial(n): 60-80% of the rows are unique.
So for small n I would recommend to choose N as follows to be absolutely on the safe side:
N = min( [Q factorial(n)] )*2;
where Q is the number of permutations you want. For bigger n you either run out of memory while searching for all permutations, or the desired subset is so small compared to the number of all possible permutations that repetition is very unlikely! (Cosmic Ray theory linked by Luis Mendo)
Your requirements are very loose and allow many different possibilities. The most efficient solution I can think off that meets these requirements is as follows:
p = perms(1:6);
[p(1:128,:) repmat(7:32,128,1)]