Here is the thing
I have a large matrix and I need to sort that based on some columns:
B is the large matrix
A=sortrows(B,[1 4 9 10]);%Just a sample input vector
Its OK so far.But then in next step (I'm iterating this function) I will need this:
A=sortrows(B,[1 4 9 10 16]);%Notice I just add a new column to input vector
And same story in next iterations
So my question is can I use previous sort results in each iteration?
Edit------------------------------------
for those who are familiar the main problem is a feature selection task.I have a matrix lets name it IS .You can find one in here .In IS rows are objects and columns are features.The task is evaluate a subset of features based on a fitness function .I'm not going in to fitness function because it is not related but as a black-box the input and output are:
function fitnessValue=fitness(inputMatrix,featureSubset)
end
inputMatrix must be sorted based on featureSubset but being sorted based on any permutation of feature set would do the thing
Now main algorithm
featureSubset=[];
iter=1;
while iter<iterMax and fitnessValue<1
j=chooseFeature(IS,featureSubset) %select a feature that has not been selected yet based on some parameters
featureSubset=union(featureSubset,j);%Add j to selected features
IS=sortrows(IS,featureSubset);%Here is the problem
fitnessValue=fitness(IS,featureSubset)
iter=iter+1;
end
That's it at each iteration I am adding features one by one and evaluating new feature subset but in order to do that I need to sort the IS matrix based on feature subset .As i said in previous iteration I also sort the entire matrix based on featureSubset(1:end-1) and I belive I can use previous sort result and somehow just sort IS based on current selected feature and get the same result .
P.S if someone can explain the algorithm of sortrows i would be really grateful
This is the function that I came up with might be useful for someone in future :
function [ndx]=sort2(x,cols,oldIndex)
x=x(:,cols);
[m,n] = size(x);
if nargin<3
ndx = (1:m)';
else
ndx=oldIndex;
end
for k = n:-1:1
[ignore,ind] = sort(x(ndx,k));
ndx = ndx(ind);
end
result--------
a=sort2(IS,[1 4 9 10]);
a=sort2(IS,16,a);
b=sort2(IS,[16 1 4 9 10]);
isequal(a,b)=1
Related
I have two for loops in MATLAB.
One of the for loops leads to different variables being inserted into the model, which are 43 and then I have 5 horizons.
So I estimate the model 215 times.
My problem is I want to store this in 215x5 matrix, the reason I have x5 is that I am estimating 5 variables, 4 are fixed and the other one comes from the for loop.
I have tried to do this in two ways,
Firstly, I create a variable called out,
out=zeros(215,5);
The first for loop is,
for i=[1,2,3,4,5];
The second for loop is,
for ii=18:60;
The 18:60 is how I define my variables using XLS read, e.g. they are inserted into the model as (data:,ii).
I have tried to store the data in two ways, I want to store OLS which contains the five estimates
First,
out(i,:)=OLS;
This method creates a 5 x 5 matrix, with the estimates for one of the (18:60), at each of the horizons.
Second,
out(ii,:)=OLS;
This stores the variables for each of the variables (18:60), at just one horizon.
I want to have a matrix which stores all of the estimates OLS, at each of the horizons, for each of my (18:60).
Minimal example
clear;
for i=[1,2,3,4,5];
K=i;
for ii=18:60
x=[1,2,3,i,ii];
out(i,:)=x;
end
end
So the variable out will store 1 2 3 5 60
I want the variable out to store all of the combinations
i.e.
1 2 3 1 1
1 2 3 1 2
...
1 2 3 5 60
Thanks
The simplest solution is to use a 3D matrix:
for jj=[1,2,3,4,5];
K=jj;
for ii=18:60
x=[1,2,3,jj,ii];
out(ii-17,jj,:)=x;
end
end
If you now reshape the out matrix you get the same result as the first block in etmuse's answer:
out = reshape(out,[],size(out,3));
(Note I replaced i by jj. i and ii are too similar to use both, it leads to confusion. It is better to use different letters for loop indices. Also, i is OK to use, but it also is the built-in imaginary number sqrt(-1). So I prefer to use ii over i.)
As you've discovered, using just one of your loop variables to index the output results in most of the results being overwritten, leaving only the results from the final iteration of the relevant loop.
There are 2 ways to create an indexing variable.
1- You can use an independent variable, initialised before the loops and incremented at the end of the internal loop.
kk=1;
for i=1:5
for ii=18:60
%calculate OLC
out(kk,:)=OLC;
kk = kk+1;
end
end
2- Use a calculation of i and ii
kk = i + 5*(ii-18)
(Use in loop as before, without increment)
I'm currently working on implementing a gradient check function in which it requires to get certain index values from the result matrix. Could someone tell me how to get a group of values from the matrix?
To be specific, for a result matrx res with size M x N, I'll need to get element res(3,1), res(4,2), res(1,3), res(2,4)...
In my case, M is dimension and N is batch size and there's a label array whose size is 1xbatch_size, [3 4 1 2...]. So the desired values are res(label(:),1:batch_size). Since I'm trying to practice vectorization programming and it's better not using loop. Could someone tell me how to get a group of value without a iteration?
Cheers.
--------------------------UPDATE----------------------------------------------
The only idea I found is firstly building a 'mask matrix' then use the original result matrix to do element wise multiplication (technically called 'Hadamard product', see in wiki). After that just get non-zero element out and do the sum operation, the code in matlab should look like:
temp=Mask.*res;
desired_res=temp(temp~=0); %Note: the temp(temp~=0) extract non-zero elements in a 'column' fashion: it searches temp matrix column by column then put the non-zero number into container 'desired_res'.
In my case, what I wanna do next is simply sum(desired_res) so I don't need to consider the order of those non-zero elements in 'desired_res'.
Based on this idea above, creating mask matrix is the key aim. There are two methods to do this job.
Codes are shown below. In my case, use accumarray function to add '1' in certain location (which are stored in matrix 'subs') and add '0' to other space. This will give you a mask matrix size [rwo column]. The usage of full(sparse()) is similar. I made some comparisons on those two methods (repeat around 10 times), turns out full(sparse) is faster and their time costs magnitude is 10^-4. So small difference but in a large scale experiments, this matters. One benefit of using accumarray is that it could define the matrix size while full(sparse()) cannot. The full(sparse(subs, 1)) would create matrix with size [max(subs(:,1)), max(subs(:,2))]. Since in my case, this is sufficient for my requirement and I only know few of their usage. If you find out more, please share with us. Thanks.
The detailed description of those two functions could be found on matlab's official website. accumarray and full, sparse.
% assume we have a label vector
test_labels=ones(10000,1);
% method one, accumarray(subs,1,[row column])
tic
subs=zeros(10000,2);
subs(:,1)=test_labels;
subs(:,2)=1:10000;
k1=accumarray(subs,1,[10, 10000]);
t1=toc % to compare with method two to check which one is faster
%method two: full(sparse(),1)
tic
k2=full(sparse(test_labels,1:10000,1));
t2=toc
I have a 5-by-200 matrix where the i:50:200, i=1:50 are related to each other, so for example the matrix columns 1,51,101,151 are related to each other, and columns 49,99,149,199 are also related to each other.
I want to use a for-loop to create another matrix that re-sorts the previous matrix based on this relationship.
My code is
values=zeros(5,200);
for j=1:50
for m=1:4:200
a=factor_mat(:,j:50:200)
values(:,m)=a
end
end
However, the code does not work.
Here's what's happening. Let's say we're on the first iteration of the outer loop, so j == 1. This effectively gives you:
j = 1;
for m=1:4:200
a=factor_mat(:,j:50:200)
values(:,m)=a;
end
So you're creating the same submatrix for a (j doesn't change) 50 times and storing it at different places in the values matrix. This isn't really what you want to do.
To create each 4-column submatrix once and store them in 50 different places, you need to use j to tell you which of the 50 you're currently processing:
for j=1:50
a=factor_mat(:,j:50:200);
m=j*4; %// This gives us the **end** of the current range
values(:,m-3:m)=a;
end
I've used a little trick here, because the indices of Matlab arrays start at 1 rather than 0. I've calculated the index of the last column we want to insert. For the first group, this is column 4. Since j == 1, j * 4 == 4. Then I subtract 3 to find the first column index.
That will fix the problem you have with your loops. But loops aren't very Matlab-ish. They used to be very slow; now they're adequate. But they're still not the cool way to do things.
To do this without loops, you can use reshape and permute:
a=reshape(factor_mat,[],50,4);
b=permute(a,[1,3,2]);
values=reshape(b,[],200);
I am working on 2D rectangular packing. In order to minimize the length of the infinite sheet (Width is constant) by changing the order in which parts are placed. For example, we could place 11 parts in 11! ways.
I could label those parts and save all possible permutations using perms function and run it one by one, but I need a large amount of memory even for 11 parts. I'd like to be able to do it for around 1000 parts.
Luckily, I don't need every possible sequence. I would like to index each permutation to a number. Test a random sequence and then use GA to converge the results to find the optimal sequence.
Therefore, I need a function which gives a specific permutation value when run for any number of times unlike randperm function.
For example, function(5,6) should always return say [1 4 3 2 5 6] for 6 parts. I don't need the sequences in a specific order, but the function should give the same sequence for same index. and also for some other index, the sequence should not be same as this one.
So far, I have used randperm function to generate random sequence for around 2000 iterations and finding a best sequence out of it by comparing length, but this works only for few number of parts. Also using randperm may result in repeated sequence instead of unique sequence.
Here's a picture of what I have done.
I can't save the outputs of randperm because I won't have a searchable function space. I don't want to find the length of the sheet for all sequences. I only need do it for certain sequence identified by certain index determined by genetic algorithm. If I use randperm, I won't have the sequence for all indexes (even though I only need some of them).
For example, take some function, 'y = f(x)', in the range [0,10] say. For each value of x, I get a y. Here y is my sheet length. x is the index of permutation. For any x, I find its sequence (the specific permutation) and then its corresponding sheet length. Based on the results of some random values of x, GA will generate me a new list of x to find a more optimal y.
I need a function that duplicates perms, (I guess perms are following the same order of permutations each time it is run because perms(1:4) will yield same results when run any number of times) without actually storing the values.
Is there a way to write the function? If not, then how do i solve my problem?
Edit (how i approached the problem):
In Genetic Algorithm, you need to crossover parents(permutations), But if you crossover permutations, you will get the numbers repeated. for eg:- crossing over 1 2 3 4 with 3 2 1 4 may result something like 3 2 3 4. Therefore, to avoid repetition, i thought of indexing each parent to a number and then convert the number to binary form and then crossover the binary indices to get a new binary number then convert it back to decimal and find its specific permutation. But then later on, i discovered i could just use ordered crossover of the permutations itself instead of crossing over their indices.
More details on Ordered Crossover could be found here
Below are two functions that together will generate permutations in lexographical order and return the nth permutation
For example, I can call
nth_permutation(5, [1 2 3 4])
And the output will be [1 4 2 3]
Intuitively, how long this method takes is linear in n. The size of the set doesn't matter. I benchmarked nth_permutations(n, 1:1000) averaged over 100 iterations and got the following graph
So timewise it seems okay.
function [permutation] = nth_permutation(n, set)
%%NTH_PERMUTATION Generates n permutations of set in lexographical order and
%%outputs the last one
%% set is a 1 by m matrix
set = sort(set);
permutation = set; %First permutation
for ii=2:n
permutation = next_permute(permutation);
end
end
function [p] = next_permute(p)
%Following algorithm from https://en.wikipedia.org/wiki/Permutation#Generation_in_lexicographic_order
%Find the largest index k such that p[k] < p[k+1]
larger = p(1:end-1) < p(2:end);
k = max(find(larger));
%If no such index exists, the permutation is the last permutation.
if isempty(k)
display('Last permutation reached');
return
end
%Find the largest index l greater than k such that p[k] < p[l].
larger = [false(1, k) p(k+1:end) > p(k)];
l = max(find(larger));
%Swap the value of p[k] with that of p[l].
p([k, l]) = p([l, k]);
%Reverse the sequence from p[k + 1] up to and including the final element p[n].
p(k+1:end) = p(end:-1:k+1);
end
I'm struggling with one of my matlab assignments. I want to create 10 different models. Each of them is based on the same original array of dimensions 1x100 m_est. Then with for loop I am choosing 5 random values from the original model and want to add the same random value to each of them. The cycle repeats 10 times chosing different values each time and adding different random number. Here is a part of my code:
steps=10;
for s=1:steps
for i=1:1:5
rl(s,i)=m_est(randi(numel(m_est)));
rl_nr(s,i)=find(rl(s,i)==m_est);
a=-1;
b=1;
r(s)=(b-a)*rand(1,1)+a;
end
pert_layers(s,:)=rl(s,:)+r(s);
M=repmat(m_est',s,1);
end
for k=steps
for m=1:1:5
M_pert=M;
M_pert(1:k,rl_nr(k,1:m))=pert_layers(1:k,1:m);
end
end
In matrix M I am storing 10 initial models and want to replace the random numbers with indices from rl_nr matrix into those stored in pert_layers matrix. However, the last loop responsible for assigning values from pert_layers to rl_nr indices does not work properly.
Does anyone know how to solve this?
Best regards
Your code uses a lot of loops and in this particular circumstance, it's quite inefficient. It's better if you actually vectorize your code. As such, let me go through your problem description one point at a time and let's code up each part (if applicable):
I want to create 10 different models. Each of them is based on the same original array of dimensions 1x100 m_est.
I'm interpreting this as you having an array m_est of 100 elements, and with this array, you wish to create 10 different "models", where each model is 5 elements sampled from m_est. rl will store these values from m_est while rl_nr will store the indices / locations of where these values originated from. Also, for each model, you wish to add a random value to every element that is part of this model.
Then with for loop I am choosing 5 random values from the original model and want to add the same random value to each of them.
Instead of doing this with a for loop, generate all of your random indices in one go. Since you have 10 steps, and we wish to sample 5 points per step, you have 10*5 = 50 points in total. As such, why don't you use randperm instead? randperm is exactly what you're looking for, and we can use this to generate unique random indices so that we can ultimately use this to sample from m_est. randperm generates a vector from 1 to N but returns a random permutation of these elements. This way, you only get numbers enumerated from 1 to N exactly once and we will ensure no repeats. As such, simply use randperm to generate 50 elements, then reshape this array into a matrix of size 10 x 5, where the number of rows tells you the number of steps you want, while the number of columns is the total number of points per model. Therefore, do something like this:
num_steps = 10;
num_points_model = 5;
ind = randperm(numel(m_est));
ind = ind(1:num_steps*num_points_model);
rl_nr = reshape(ind, num_steps, num_points_model);
rl = m_est(rl_nr);
The first two lines are pretty straight forward. We are just declaring the total number of steps you want to take, as well as the total number of points per model. Next, what we will do is generate a random permutation of length 100, where elements are enumerated from 1 to 100, but they are in random order. You'll notice that this random vector uses only a value within the range of 1 to 100 exactly once. Because you only want to get 50 points in total, simply subset this vector so that we only get the first 50 random indices generated from randperm. These random indices get stored in ind.
Next, we simply reshape ind into a 10 x 5 matrix to get rl_nr. rl_nr will contain those indices that will be used to select those entries from m_est which is of size 10 x 5. Finally, rl will be a matrix of the same size as rl_nr, but it will contain the actual random values sampled from m_est. These random values correspond to those indices generated from rl_nr.
Now, the final step would be to add the same random number to each model. You can certainly use repmat to replicate a random column vector of 10 elements long, and duplicate them 5 times so that we have 5 columns then add this matrix together with rl.... so something like:
a = -1;
b = 1;
r = (b-a)*rand(num_steps, 1) + a;
r = repmat(r, 1, num_points_model);
M_pert = rl + r;
Now M_pert is the final result you want, where we take each model that is stored in rl and add the same random value to each corresponding model in the matrix. However, if I can suggest something more efficient, I would suggest you use bsxfun instead, which does this replication under the hood. Essentially, the above code would be replaced with:
a = -1;
b = 1;
r = (b-a)*rand(num_steps, 1) + a;
M_pert = bsxfun(#plus, rl, r);
Much easier to read, and less code. M_pert will contain your models in each row, with the same random value added to each particular model.
The cycle repeats 10 times chosing different values each time and adding different random number.
Already done in the above steps.
I hope you didn't find it an imposition to completely rewrite your code so that it's more vectorized, but I think this was a great opportunity to show you some of the more advanced functions that MATLAB has to offer, as well as more efficient ways to generate your random values, rather than looping and generating the values one at a time.
Hopefully this will get you started. Good luck!