Recently I wrote this statement
v = arrayfun(#(x) sum(randn(1, 4).^2), zeros(1, 1000000));
to create a new vector and now I'm asking if there exists a function in Matlab to avoid the creation of the unnecessary second vector zeros(1, 1000000). I'm looking for something like
v = FUN(#someInitFunction, [rows, cols]);
without loops, recursion and unnecessary allocation where someInitFunction is given and can't be changed. Does Matlab provide such a function FUN? A simple "No, it doesn't exist" would be a valid answer for me.
To summarize the function FUN: I want to create a new array by calling a function someInitFunction for each element of this new array. The array should be equivalent to
[
someInitFunction() someInitFunction() ...;
someInitFunction() someInitFunction() ...;
.
.
.
someInitFunction() someInitFunction() ...
]
As far as I know there is no builtin function for that. It's relatively easy to create your own however.
You asked a solution without loop but the current solution you are using (arrayfun) uses loop under the hood, and generally coding the same in a properly organised loop is actually faster than arrayfun.
For your case, the function GenArrayFun.m :
function out = GenArrayFun(initFunction , arraySize)
out = zeros(arraySize) ;
for k=1:numel(out)
out(k) = initFunction() ;
end
It has a loop, but no more than arrayfun, and seem to perform twice as fast (at least on my installation, R2016a, win10):
initFunction = #() sum(randn(1, 4).^2) ;
tic
v = arrayfun(#(x) sum(randn(1, 4).^2), zeros(1, 1000000));
toc
tic
out = GenArrayFun( initFunction , [1,1000000] );
toc
Sorry I did not take the time to build a proper timeit benchmark for such a small example, I think the results are significant enough to notice a difference:
Elapsed time is 6.815043 seconds. % arrayfun
Elapsed time is 3.060161 seconds. % GenArrayFun
And just to make sure it evaluate the initFunction for every element:
>> out = GenArrayFun( initFunction , [2,3] )
out =
6.25676106665387 6.52758807745462 2.99236122767462
0.386750258201569 0.566092999842791 2.21158011908878
Related
I want to assign one value to a list of objects in Matlab without using a for-loop (In order to increase efficiency)
Basically this works:
for i=1:Nr_of_Objects
Objectlist(i,1).weight=0.2
end
But I would like something like this:
Objectlist(:,1).weight=0.2
Which is not working. I get this error:
Expected one output from a curly brace or dot indexing expression, but there were 5 results.
Writing an array to the right hand side is also not working.
I`m not very familiar with object oriented programming in Matlab, so I would be happy if someone could help me.
Your looking for the deal function:
S(1,1).a = 1
S(2,1).a = 2
S(1,2).a = 3
[S(:,1).a] = deal(4)
Now S(1,1).a and S(2,1).a equal to 4.
In matlab you can concatenate several output in one array using []. And deal(X) copies the single input to all the requested outputs.
So in your case:
[Objectlist(:,1).weight] = deal(0.2)
Should work.
Noticed that I'm not sure that it will be faster than the for loop since I don't know how the deal function is implemented.
EDIT: Benchmark
n = 1000000;
[S(1:n,1).a] = deal(1);
tic
for ii=1:n
S(ii,1).a = 2;
end
toc
% Elapsed time is 3.481088 seconds
tic
[S(1:n,1).a] = deal(2);
toc
% Elapsed time is 0.472028 seconds
Or with timeit
n = 1000000;
[S(1:n,1).a] = deal(1);
g = #() func1(S,n);
h = #() func2(S,n);
timeit(g)
% ans = 3.67
timeit(h)
% ans = 0.41
function func1(S,n)
for ii=1:n
S(ii,1).a = 2;
end
end
function func2(S,n)
[S(1:n,1).a] = deal(2);
end
So it seems that using the deal function reduce the computational time.
I have quite a heavy function in MATLAB:
function [out] = f ( in1, in2, in3)
Which is called quite often with the same parameters.
The function is deterministic so for given input parameters its output will always be the same.
What would be the simplest way of storing the results of computed inputs in the function such that if the function will be called again with the same output it would be able to answer quickly?
Is a persistent variable which maps (using containers.Map or some other class) input set <in1, in2, in3> to a result the way to go?
Note that any method which requires saving the data to disk is out of the question in my application.
Below is an idea for a CacheableFunction class
It seems all of the answers to your main question are pointing the same direction - a persistent Map is the consensus way to cache results, and I do this too.
If the inputs are arrays, they'll need to be hashed to a string or scalar to be used as a map key. There are a lot of ways to hash your 3 input arrays to a key, I used DataHash in my solution below.
I chose to make it a class rather than a function like memoize so that the input hashing function can be dynamically specified one time, rather than hardcoded.
Depending on the form of your output, it also uses dzip/dunzip to reduce the footprint of the saved outputs.
Potential improvement: a clever way of deciding which elements to remove from the persistent map when its memory footprint approaches some limit.
Class definition
classdef CacheableFunction < handle
properties
exeFun
hashFun
cacheMap
nOutputs
zipOutput
end
methods
function obj = CacheableFunction(exeFun, hashFun, nOutputs)
obj.exeFun = exeFun;
obj.hashFun = hashFun;
obj.cacheMap = containers.Map;
obj.nOutputs = nOutputs;
obj.zipOutput = [];
end
function [result] = evaluate(obj, varargin)
thisKey = obj.hashFun(varargin);
if isKey(obj.cacheMap, thisKey)
if obj.zipOutput
result = cellfun(#(x) dunzip(x), obj.cacheMap(thisKey), 'UniformOutput', false);
else
result = obj.cacheMap(thisKey);
end
else
[result{1:obj.nOutputs}] = obj.exeFun(varargin);
if isempty(obj.zipOutput)
obj.zipCheck(result);
end
if obj.zipOutput
obj.cacheMap(thisKey) = cellfun(#(x) dzip(x), result, 'UniformOutput', false);
else
obj.cacheMap(thisKey) = result;
end
end
end
function [] = zipCheck(obj,C)
obj.zipOutput = all(cellfun(#(x) isreal(x) & ~issparse(x) & any(strcmpi(class(x), ...
{'double','single','logical','char','int8','uint8',...
'int16','uint16','int32','uint32','int64','uint64'})), C));
end
end
end
Testing it out...
function [] = test_caching_perf()
A = CacheableFunction(#(x) long_annoying_function(x{:}), #(x) DataHash(x), 3);
B = rand(50, 50);
C = rand(50, 50);
D = rand(50, 50);
tic;
myOutput = A.evaluate(B, C, D);
toc
tic;
myOutput2 = A.evaluate(B, C, D);
toc
cellfun(#(x, y) all(x(:) == y(:)), myOutput, myOutput2)
end
function [A, B, C] = long_annoying_function(A, B, C)
for ii = 1:5000000
A = A+1;
B = B+2;
C = C+3;
end
end
And results
>> test_caching_perf
Elapsed time is 16.781889 seconds.
Elapsed time is 0.011116 seconds.
ans =
1 1 1
MATLAB now ships with a function just for this purpose.
The technique used is called "memoization" and the function's name is "memoize".
Check out : https://www.mathworks.com/help/matlab/ref/memoize.html
Persistent map is indeed a nice way to implement cached results. Advantages I can think of:
No need to implement hash function for every data type.
Matlab matrices are copy-on-write, which can offer certain memory efficiency.
If memory usage is an issue, one can control how many results to cache.
There is a file exchange submission, A multidimensional map class by David Young, comes with a function memoize() does exactly this. It's implementation uses a bit different mechanism (referenced local variable), but the idea is about the same. Compared with persistent map inside each function, this memoize() function allows existing function to be memoized without modification. And as pointed out by Oleg, using DataHash (or equivalent) can further reduce the memory usage.
PS: I have used the MapN class extensively and it is quite reliable. Actually I have submitted a bug report and the author fixed it promptly.
I have a code that actually works but the output is supposed to be multiplied together. Also, the time of execution was not in milliseconds. This is the Matlab code
function product = prod(A)
tic;
A=input('matrix A =');
[rows, columns] = size(A);
for i=1:rows
prod=A(i,i)*A(i,end)
end
seconds=toc
For instance if given A=[1,2,3;4,5,6;7,8,9] when i=1 we have 1*3=3 when i=2 we have 2*6=12 when i=3 we have 9*9=81.
The output I want should be 3*12*81=2916 (the product of the values above) and the time of execution should be milliseconds.
When I extend the solution you proffer for the code below, it does not multiply them together and the time elapsed seems to be big
When the above code is used for the matrix A = [2,3,6,4;2,1,7,-2;6,8,1,-3;5,3,4,1].
My output is
out = -22, -23, -7, -3
Elapsed time is 10.200446 seconds.
I want all the output to multiple each other so that it will be
out = -22*-23*-7*-3 = 10626
Well, it took me some time to decipher what you have been trying to do, but finally, I was able to reconstruct your output. So first, there is no need for the prod function, it only makes things messy.
Here is a simple function that does the trick:
function out = testValue
A=input('matrix A =');
out=1;
if mod(size(A),2)==0
for i = 1:length(A)
out = out*(A(i,i)*A(i,end)-A(i,end-1)*A(end,i));
end
else
for i = 1:(length(A)-1)/2
out = out*(A(i,i)*A(i,end)-A(i,end-1)*A(end,i));
end
end
end
Some remarks on the code above:
No need for the columns variable
I omitted the tic toc - you can add them back if needed
No use for the prod function, instead the input for prod is simply what you consider as its output.
You have to initialize out=1 for a cumulative product
I'm sure there are cleverer ways to do this, but as #beaker wrote it is not clear what exactly is the purpose of this code.
Hope this helps :)
/edit: The loop doesn't become slower. I didn't take the time correctly. See Rasman's answer.
I'm looping over 3 parameters for a somewhat long and complicated function and I noticed two things that I don't understand:
The execution gets slower with each successive iteration, although the function only returns one struct (of which I only need one field) that I overwrite with each iteration.
The profiler shows that the end statement for the innermost for takes a quite long time.
Consider the following example (I'm aware that this can easily be vectorized, but as far as I understand the function I call can't):
function stuff = doSomething( x, y, z )
stuff.one = x+y+z;
stuff.two = x-y-z;
end
and how I execute the function
n = 50;
i = 0;
currenttoc = 0;
output = zeros(n^3,4);
tic
for x = 1:n
for y = 1:n
for z = 1:n
i = i + 1;
output(i,1) = x;
output(i,2) = y;
output(i,3) = z;
stuff = doSomething(x,y,z);
output(i,4) = stuff.one;
if mod(i,1e4) == 0 % only for demonstration, not in final script
currenttoc = toc - currenttoc;
fprintf(1,'time for last 10000 iterations: %f \n',currenttoc)
end
end
end
end
How can I speed this up? Why does every iteration take longer than the one before? I'm pretty sure this is horrible programming, sorry for that.
When I replace the call to doSomething with output(i,4)=toc;, and I plot diff(output(:,4)), I see that it's the call to fprintf that takes longer and longer every time, apparently.
Removing the if-clause returns to every iteration taking about the same amount of time.
So, the problem gets largely eliminated when I replace the if statement with:
if mod(i,1e4) == 0 % only for demonstration, not in final script
fprintf(1,'time for last 10000 iterations: %f \n',toc); tic;
end
I think the operation on toc may be causing the problem
It's MUCH faster if doSomething returns multiple output variables rather than a struct
function [out1,out2] = doSomething( x, y, z )
out1 = x+y+z;
out2 = x-y-z;
end
The fact that it gets slower on each subsequent iteration is strange and i have no explanation for it but hopefully that gives you some speed up at least.
How to apply a function that returns non scalar output to arrays using arrayfun?
For example - How to vectorize the following code?
array = magic(5);
A = cell(size(array));
for i=1:5
for j=1:5
A{i,j} = 1:array(i,j);
end
end
This naive attempt to vectorize does not work, because the output is not a scalar
array = magic(5);
result = arrayfun(#(x)(1:x),array);
There are 2 methods to achieve it:
It is possible to set 'UniformOutput' to false. Then, the result is a cell array.
result = arrayfun(#(x)(1:x),array,'UniformOutput',false);
But there is a nice trick that I have found today, the function itself can return a cell. This removes the need of typing 'UniformOutput',false each and every time.
result = arrayfun(#(x){1:x},array)
What is really interesting here that I don't have to type #(X)({1:x}) but I can define it only by using curly bracers #(X){1:x}
Edit(1): As #Jonas correctly points out, there is no wonder that the regular bracers () are not needed, as they are optional. For example, #(x) x+1 is a valid syntax.
Edit(2): There is a small difference between using the curly bracers method or the UniformOutput,false. When the input array is empty, their behavior is different.
In addition to the answer of Andrey I want to note that it seems, that the first approach, using the option UniformOutput, is slightly faster:
>> tic; cellfun(#(x) {single(x)}, data); toc;
Elapsed time is 0.031817 seconds.
>> tic; cellfun(#(x) single(x), data,'UniformOutput',0); toc;
Elapsed time is 0.025526 seconds.