Related
Given three arrays of char, say size(a) = [N,80], size(b) = [N,100]; size(c) = [N,10];
When N=5
a, b and c look something like,
ans =
5×80 char array
‘efawefref’
‘Afreafraef’
‘afeafefaef’
‘afeafeaffa’
‘afeafaefae’
I want to find the unique entries (not combinations), this is, the unique entries of x = [a, b, c]
Of course I can do unique([a, b, c]) but this is terrible slow for this data. N~1e7
Example,
a = [ 'timon ';
'simba ';
'nala ';
'timon ';
'mufasa'];
b = [ 'boar ';
'lion ';
'lionese';
'boar ';
'lion '];
c = [ 'chubby';
'small ';
'fat ';
'chubby';
'fit '];
unique([a,b,c],'rows')
ans =
4×19 char array
'mufasalion fit '
'nala lionesefat '
'simba lion small '
'timon boar chubby'
size(unique([a,b,c],'rows'),1)
ans =
4
IS there a smarter way to do this?
EDIT: results from answers
For entries of these sizes,
>> size(a)
ans =
11724952 76
>> size(b)
ans =
11724952 64
>> size(c)
ans =
11724952 6
Results
#myradio
>> tic, size(unique(horzcat(a,b,c),'rows')), toc
ans =
1038303 146
Elapsed time is 74.402044 seconds.
#gnovice 1
>> tic, size(unique(cellstr([a b c]))), toc
ans =
1038303 1
Elapsed time is 77.044463 seconds.
#gnovice 2
>> tic, map = containers.Map(cellstr([a b c]), ones(length(a), 1)); size(map.keys.'), toc
ans =
1038303 1
Elapsed time is 58.732947 seconds.
#Wolfie
>> tic, size(unique( [categorical(cellstr(a)),categorical(cellstr(b)),categorical(cellstr(c))], 'rows' )), toc
ans =
1038303 3
Elapsed time is 189.517131 seconds.
#obchardon
>> tic, x = primes(2000); a1 = prod(x(a+0),2); b1 = prod(x(b+0),2); c1 = prod(x(c+0),2); size(unique([a1,b1,c1],'rows')), toc
ans =
1038258 3
Elapsed time is 46.889431 seconds.
I am puzzled about this last one, I tried with other examples and it always gives a slightly lower value.
To mimic the larger set of data in the question, I created the following randomized character arrays using randi:
a = char(randi([65 90], [100 76])); % Generate 100 76-character arrays
a = a(randi([1 100], [11724952 1]), :); % Replicate rows: 11724952-by-76 result
b = char(randi([65 90], [100 64])); % Generate 100 64-character arrays
b = b(randi([1 100], [11724952 1]), :); % Replicate rows: 11724952-by-64 result
c = char(randi([65 90], [100 6])); % Generate 100 6-character arrays
c = c(randi([1 100], [11724952 1]), :); % Replicate rows: 11724952-by-6 result
With up to 100 unique strings in each of a, b, and c, this will yield close to 1,000,000 unique combinations when concatenated.
I then tested 3 solutions: the original using unique, a variant that converts the character array to a cell array of strings using cellstr to avoid using the 'rows' argument, and one using a containers.Map object. The last one feeds the strings as keys to the containers.Map class (with dummy associated values) and lets it create a map that will have only the unique strings as its keys, which you can then extract.
Since these tests took a minimum of 1 minute to run, it wasn't feasible to use the more accurate timing routine timeit (which runs the function many times over to get an average measurement). I therefore used tic/toc. Here are some typical results using version R2018a:
>> clear d
>> tic; d = unique(horzcat(a, b, c), 'rows'); toc
Elapsed time is 726.324408 seconds.
>> clear d
>> tic; d = unique(cellstr([a b c])); toc
Elapsed time is 99.312927 seconds.
>> clear d
>> tic; map = containers.Map(cellstr([a b c]), ones(size(a, 1), 1)); d = map.keys.'; toc
Elapsed time is 89.853430 seconds.
The two faster solutions typically averaged around the same, with the containers.Map being slightly faster on average. They are both much faster than using unique with the 'rows' argument, although this is in disagreement with the results in the post using version R2018b. Maybe unique had significant updates in the newer version, or maybe the specific content of the character arrays matters greatly (e.g. whether all strings repeat with roughly equal frequency, if the arrays are sorted versus unsorted, etc.).
Categorical arrays are often quicker for this sort of thing, as they are roughly treated as ordinals internally.
% Set up your example
a = [ 'timon '; 'simba '; 'nala '; 'timon '; 'mufasa'];
b = [ 'boar '; 'lion '; 'lionese'; 'boar '; 'lion '];
c = [ 'chubby'; 'small '; 'fat '; 'chubby'; 'fit '];
% Make the arrays larger and join into one big categorical array
k = [categorical(cellstr(a)),categorical(cellstr(b)),categorical(cellstr(c))];
% Get unique rows
u = unique( k, 'rows' );
We can make the categorical(cellstr(...)) look a bit cleaner if operating on lots of variables by using an anonymous function:
cc = #(x) categorical(cellstr(x));
u = unique( [cc(a), cc(b), cc(c)], 'rows' );
Edit: Not sure this actually shows a speed-up, the categorical call is really slow for large char arrays,my test was rubbish.
I don't know if unique work faster with integer. If this is the case we could use this code to eventually speed up the operation:
%get at least ~200 primes numbers
x = primes(2000);
%prime multiplication will give an unique integer (prime factorization theorem)
a1 = prod(x(a+0),2);
b1 = prod(x(b+0),2);
c1 = prod(x(c+0),2);
%Now we apply unique on integer instead of char
[~,ind] = unique([a1,b1,c1],'rows')
%Get the unique sentence.
r = [a(ind,:),b(ind,:),c(ind,:)]
Of course if N is too big the prime multiplication will give Inf.
EDIT:
As pointed out by #gnovice my hashing function is highly surjective (which can lead to collision).
So we can use another "hashing" function:
% each sentence are converted to an unique number
x = [a,b,c]+0*[10.^(0:18)].'
%get index
[~,ind] = unique(x)
%unique sentence:
r = [a(ind,:),b(ind,:),c(ind,:)]
This time there will be no collision, but again the sentences should be shorter than ~110 characters
I have a lot of text files with two columns , for example :
1998-08 56
1998-09 78
1998-10 12
1998-11 76
and I have a MATLAB code , which I want to run it with octave. This code generates a random matrix (c) and then calculates Poisson inverse, now instead of a random matrix I want to tell it get the second column of the text file as values of c .
and then once I have calculated I want to plot the results in a graph with dates on the first columns as values of x axis.
my question is naive but I'm very very new to octave and I spent last day searching a documentation but I found nothing useful.
clc
clear all
#here I need to read in input the second columns of each file
C = [8,9,3,6,4,1,5,8,3,5,8,10,5,3,1,5,11,3,6,5,8,5,1,3,5,3,3,7,0,2,6,7,1,8,3,5,2,6,3,5,3,8,2,5,2,7,4,4,5,50,7,5,3,3,8,2,7,7,3,6,4,4,2,4,5,1,5,14,2,6,1,4,7,3,4,6,1,3,4,4,5,1,5,4,4,6,6,2,0,4,8,9,4,5,10,3,4,2,3,6,6,8,5,5,8,3,8,4,5,4,1,4,6,5,2,5,5,7,5,1,5,4,5,3,5,11,8,6,6,0,6,4,6,5,6,4,6,4,11,4,6,5,4,4,4,2,4,7,4,40,7,7,3,1,6,7,7,4,10,5,5,4,8,5,4,5,2,7,8,6,7,6,4,3,3,3,4,6,1,7,2,6,5,7,3,6,3,6,2,3,4,7,2,4,3,6,4,7,9,5,6,2,4,4,7,4,6,3,5,3,3,5,4,8,5,3,6,4,8,6,3,8,2,7,5,3,3,8,8,4,4,8,5,7,4,4,3,6,8,4,3,4,3,9,4,3,10,4,4,5];
C = C';
N = length(C);
H = 6;
w = [1:H H+1 H:-1:1]';
Lambda_Tilde = NaN * zeros(N, 1);
L_Tilde = NaN * zeros(N, 1);
U_Tilde = NaN * zeros(N, 1);
for t = (H+1):(N-H-1)
Lambda_Tilde(t) = sum(w .* C(t-H:t+H)) / sum(w);
L_Tilde(t) = poissinv(0.005, Lambda_Tilde(t));
U_Tilde(t) = poissinv(0.995, Lambda_Tilde(t));
end
clf
plot(1:N, C, 'bo-');
hold on
plot(1:N, L_Tilde, 'r-.');
plot(1:N, U_Tilde, 'r-.');
Import and Export of data is really well documented, so "found nothing useful" is hard to believe. Especially importdata could be interesting for you.
data = importdata('Data.txt');
C = data.data.';
returns:
C =
56 78 12 76
off-topic:
NaN * zeros(N, 1) is the same as NaN(N, 1)
I am trying to write some code which feeds a function with an unknown number of parameters. The idea is to feed the function the minimum, middle and maximum value within the possible range of values.
For example:
If the function takes 3 parameters
Parameter 1 can accept a range of 0 - 10
Parameter 2 can accept a range of 20 - 40
Parameter 3 can accept a range of 6 - 66
myfunction(para1, para2, para3)
myfunction(min,min,min)
myfunction(min,min,mid)
myfunction(min,min,max)
myfunction(min,mid,min)
myfunction(min,mid,mid)
myfunction(min,mid,max)
etc...
so using our example above:
The first time it loops I need to run
myfunction(0, 20, 0)
next time it loops it needs to run
myfunction(0, 20, 36)
next time it loops it needs to run
myfunction(0, 20, 66)
etc...
For all possible combinations (in this case all 27).
However, if the number of parameters changes to accept 4, it needs to be able to accommodate that and so on. I've looked into doing it as a loop or recursively, but I thought as a loop would be easier to understand, but either would be really helpful.
I don't want to have to do this manually so any help will be much appreciated.
For an arbitrary number of parameters (not fixed to 3), you can use the following code. It makes use of comma-separated lists, which is a powerful tool to handle a variable number of parameters.
Let n be the number of parameters. Then the number of combinations is N = 3^n.
params = {linspace(0,10,3), linspace(20,40,3), linspace(6,66,3)};
%// cell array with arbitrary number of elements. Each cell contains a 3-vector
%// which defines min, mid, max of one parameter.
n = numel(params); %// number of parameters
N = 3^n; %// number of combinations
paramCombs = cell(1,n); %// initialization for output of ndgrid
[paramCombs{end:-1:1}] = ndgrid(params{end:-1:1}); %// generate all combinations
%// in desired order. Gives n matrices, each containing values of one parameter
paramCombs = cellfun(#(c) c(:), paramCombs, 'uni', 0); %// linearize matrices
%// into n column vectors, each with N rows.
paramCombs = [paramCombs{:}]; %// concat column vectors into N x n matrix
paramCombs = mat2cell(paramCombs,ones(N,1),ones(n,1)); %// convert to
%// N x n cell array. Each row contains a combination of parameter values
result = arrayfun(#(n) myFun(paramCombs{n,:}), 1:N, 'uni', 0); %// call myFun
%// with each combination of parameter values
The result variable is a 1 x N cell array, where each cell contains the result of calling myFun with a combination of the n parameters.
Example 1: myFun's output simply replicates the input (as in #thewaywewalk's answer). params is defined as given above, so there are 3 parameters:
>> result{1}
ans =
0 20 6
>> result{2}
ans =
0 20 36
>> result{3}
ans =
0 20 66
>> result{4}
ans =
0 30 6
>> result{5}
ans =
0 30 36
etc.
Example 2: case with 2 parameters: params = {linspace(0,2,3), linspace(0,10,3)}. Again, myFun simply replicates the input:
>> result{1}
ans =
0 0
>> result{2}
ans =
0 5
>> result{3}
ans =
0 10
>> result{4}
ans =
1 0
>> result{5}
ans =
1 5
etc.
The method can be further generalized to an arbitrary (and possibly different) number of values for each parameter, just replacing the line N = 3^n; by
N = prod(cellfun(#numel, params)); %// number of combinations
Example 3: there are 2 parameters; the first with 3 values, and the second with 2: params = {[1 2 3], [10 20]};:
>> result{1}
ans =
1 10
>> result{2}
ans =
1 20
>> result{3}
ans =
2 10
>> result{4}
ans =
2 20
>> result{5}
ans =
3 10
>> result{6}
ans =
3 20
Think about something like this:
function result = permfunction()
% I assume you every parameter-range is defined by 3 values: min, mid, max
% you can define every triple as follows:
para1 = linspace(0,10,3);
para2 = linspace(20,40,3);
para3 = linspace(6,66,3);
% all possible parameters
parameters = [para1(:),para2(:),para3(:)];
% possible combinations of parameters-indices
a = perms(1:3);
% transformed into a cell array with linear indices, to achieve this +3 and +6
% are added to the 2nd and 3rd row.
idx = mat2cell( [a(:,1) , a(:,2)+3 , a(:,3)+6] , ones(length(a),1) );
% all parameter combinations
combinations = cellfun(#(x) parameters(x),idx,'uni',0');
% apply to your function myfunction (custom)
result = cellfun(#myfunction, combinations,'uni',0' );
end
function y = myfunction( parametertriple )
%just pass the input to the output
y = parametertriple;
end
you finally get a cell array with the results of myfunction for all your combinations of parameters. In this case I just passed the parameters to the output:
>> celldisp(ans)
ans{1} =
10 30 6
ans{2} =
10 20 36
ans{3} =
5 40 6
ans{4} =
5 20 66
ans{5} =
0 30 66
ans{6} =
0 40 36
This is how I would solve it:
%dummy function
testfun=#(a,b,c)fprintf('call function for %d,%d,%d\n',a,b,c);
%functon name
testfunname='testfun';
%intended inputs
input={[0,5,10],[20,30,40],[6,36,66]};
%generate all inputs
eval(['[' sprintf('a{%d} ',1:numel(input)) ']=meshgrid(input{:});']);
%call function
eval(['arrayfun(' testfunname sprintf(',a{%d} ',1:numel(input)) ');']);
Using eval is a dirty solution, but I don't find an alternative which allows variable input size.
I am trying to find the Extremum of a 3-dim matrix along the 2nd dimension.
I started with
[~,index] = max(abs(mat),[],2), but I don't know how to advance from here. How is the index vector to be used together with the original matrix. Or is there a completely different solution to this problem?
To illustrate the task assume the following matrix:
mat(:,:,1) =
23 8 -4
-1 -26 46
mat(:,:,2) =
5 -27 12
2 -1 18
mat(:,:,3) =
-10 49 39
-13 -46 41
mat(:,:,4) =
30 -24 18
-40 -16 -36
The expected result would then be
ext(:,:,1) =
23
-46
ext(:,:,2) =
-27
18
ext(:,:,3) =
49
-46
ext(:,:,4) =
30
-40
I don't know how to use the index vector with mat to get the desired result ext.
1) If you want to find a maximum just along, let's say, 2d dimension, your variable index will be a matrix having dimensions (N,1,M), where N and M are number of elements of your matrix in the first and third dimensions respectively. In order to remove dummy dimensions, there is function squeeze() exist: index=squeeze(index) After that size(index) gives N,M
2) Depending on your problem, you probably need matlab function ind2sub(). First, you take a slice of your matrix, than find its maximum with linear indexing, and than you can restore your indicies with int2sub(). Here is an example for a 2D matrix:
M = randn(5,5);
[C,I] = max(M(:));
[index1,index2] = ind2sub(size(M),I);
Same method allows to find the absolute maximal element in whole 3D matrix.
Use ndgrid to generate the values along dimensions 1 and 3, and then sub2ind to combine the three indices into a linear index:
[~, jj] = max(abs(mat),[],2); %// jj: returned by max
[ii, ~, kk] = ndgrid(1:size(mat,1),1,1:size(mat,3)); %// ii, kk: all combinations
result = mat(sub2ind(size(mat), ii, jj, kk));
A fancier, one-line alternative:
result = max(complex(mat),[],2);
This works because, acccording to max documentation,
For complex input A, max returns the complex number with the largest complex modulus (magnitude), computed with max(abs(A)).
i know one simple way to sum elements of array is following code
sum=0;
>> a=[12 31,45,10];
>> n=length(a);
>> for i=1:n
sum=sum+a(i);
end
>> sum
sum =
98
also there is another way to do it ,like this one
>> sum=0;
>> for a=[10 21 23 45]
sum=sum+a;
end
>> sum
sum =
99
i want to know which one faster and why,maybe second one is equivalent of pointer in c++?please help me
Why not use:
a = [10 21 23 45];
yourSum = sum(a);
To answer your question further: check your results using tic and toc