How MATLAB code generation infers output size with nested branches - matlab

When generating C code using MATLAB Coder, the behaviour is different when an if happens in body of another if or in its elsesection. The following case easily creates C code with output having size 5x5:
function y = foo1(u)
if u > 0
y = zeros(2,2);
else
y = zeros(5,5);
end
Now this one works as well
function y = foo2(u,v)
if u > 0
y = zeros(2,2);
else
y = zeros(5,5);
if v > 0
y = 2 * y;
end
end
But this one fails to generate code, complaining about size mismatch:
function y = foo3(u,v)
if u > 0
y = zeros(2,2);
if v > 0
y = 2 * y;
end
else
y = zeros(5,5);
end
Here is the output in command-line:
>> codegen foo1.m -args {0}
>> codegen foo2.m -args {0,0}
>> codegen foo3.m -args {0,0}
??? Size mismatch (size [2 x 2] ~= size [5 x 5]).
The size to the left is the size of the left-hand side of the assignment.
Error in ==> foo3 Line: 8 Column: 5
Code generation failed: Open error report.
Error using codegen (line 144)
I have seen this behaviour in MATLAB R2013b and R2015a.

From the docs, Matlab codegen must know the size of a matrix at compile time unless codegen is told or infers that the matrix is of variable size. There are several ways to let Matlab know a matrix will be of variable size:
Using coder.varsize function, a matrix can be explicitly declared to be of variable size.
MATLAB can infer a matrix is of variable size from the structure of the code.
As your code suggests, option (2) apparently isn't robust. Matlab tries to infer in some cases when there's a simple if else statement, but that inference appears to be quite fragile, as shown by your example.
Rather than rely on MATLAB to correctly infer whether a matrix is variable size, a solution is to make an explicit declaration:
function y = foo3(u,v)
coder.varsize('y', []); % Let codegen know y is variable sized
% and can be arbitrary dimensions
% an alternative is: coder.varsize('y',[5,5]);
if u > 0
y = zeros(2,2);
if v > 0
y = 2 * y;
end
else
y = zeros(5,5);
end
Why might Matlab want to know this information? If a matrix's size is known at compile time, all kinds of additional optimizations may be possible (loop unrolling etc...).

I agree with Matthew Gunn's answer, this is to add some explanation for the behavior. A good mental model to have about how Coder analyzes your MATLAB code is that it looks at it from the top to the bottom.
Applying that mental model, in your first two examples, the assignments which determine sizes of y: y = zeros(2,2) and y = zeros(5,5), occur before the value of y is ever used. So Coder can merge both sizes to automatically make y a variable-sized array. In the third example, the assignment y = zeros(2,2) occurs and then y is used: y = 2 * y. At this point, Coder needs to determine the size and type of the multiplication 2 * y. Only the 2-by-2 assignment is seen so it is inferred that 2 * y must also return a 2-by-2 matrix.
Performing this inference then embeds the assumption that y is 2-by-2 into the code and essentially locks the size of y so that the subsequent assignment to y with a 5-by-5 matrix must fail.

Related

"out of memory" error for mvregress in matlab

I am trying to use mvregress with the data I have with dimensionality of a couple of hundreds. (3~4). Using 32 gb of ram, I can not compute beta and I get "out of memory" message. I couldn't find any limitation of use for mvregress that prevents me to apply it on vectors with this degree of dimensionality, am I doing something wrong? is there any way to use multivar linear regression via my data?
here is an example of what goes wrong:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
%without residual term:
A_hat=mvregress(X',Y');
%wit residual term:
[B, y_hat]=mlrtrain(X,Y)
where
function [B, y_hat]=mlrtrain(X,Y)
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = [kron([Xmat(i,:)],eye(d))];
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
end
the error is:
Error using bsxfun
Out of memory. Type HELP MEMORY for your options.
Error in kron (line 36)
K = reshape(bsxfun(#times,A,B),[ma*mb na*nb]);
Error in mvregress (line 319)
c{j} = kron(eye(NumSeries),Design(j,:));
and this is result of whos command:
whos
Name Size Bytes Class Attributes
A 400x400 1280000 double
N 400x1000 3200000 double
X 400x1000 3200000 double
Y 400x1000 3200000 double
dataVariance 1x1 8 double
dim 1x1 8 double
mixtureCenters 400x1 3200 double
noiseVariance 1x1 8 double
nsamp 1x1 8 double
Okay, I think I have a solution for you, short version first:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = kron(Xmat(i,:),speye(d));
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
Strangely, I could not access the function's workspace, it did not appear in the call stack. This is why I put the function after the script here.
Here's the explanation that might also help you in the future:
Looking at the kron definition, the result when inserting an m by n and a p by q matrix has size mxp by nxq, in your case 400 by 1001 and 1000 by 1000, that makes a 400000 by 1001000 matrix, which has 4*10^11 elements. Now you have four hundred of them, and each element takes up 8 bytes for double precision, that is a total size of about 1.281 Petabytes of memory (or 1.138 Pebibytes, if you prefer), well out of reach even with your grand 32 Gibibyte.
Seeing that one of your matrices, the eye one, contains mostly zeros, and the resulting matrix contains all possible element product combinations, most of them will be zero, too. For such cases specifically, MATLAB offers the sparse matrix format, which saves a lot of memory depending on the number of zero elements in a matrix by only storing nonzero ones. You can convert a full matrix to a sparse representation with sparse(X), or you get an eye matrix directly by using speye(n), which is what I did above. The sparse property propagates to the result, which you should now have enough memory for (I have with 1/4 of your memory available, and it works).
However, what remains is the problem Matthew Gunn mentioned in a comment. I get an error saying:
Error using mvregress (line 260)
Insufficient data to estimate either full or least-squares models.
Preface
If your regressors are all the same across each regression equation and you're interested in the OLS estimate, you can replace a call to mvregress with a simple call to \.
It appears in the call to mlrtrain you had a matrix transposition error (since corrected). In the language of mvregress, n is the number of observations, d is the number of outcome variables. You generate a matrix Y that is d by n. But THEN when you should call mlrtrain(X', Y') not mlrtrain(X, Y).
If below isn't specifically, what you're looking for, I suggest you precisely define what you're trying to estimate.
What I would have written if I were you
So much that's been said here is completely off base that I'm posting code of what I would have written if I were you. I've reduced the dimensionality to show the equivalence in your special case to simply calling \. I've also written stuff in a more standard way (i.e. having observations run down the rows and not making matrix transposition errors).
dim=5; % These can go way higher but only if you use my code
nsamp=20; % rather than call mvregress
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X = randn(nsamp, dim)*sqrt(dataVariance ) + repmat(mixtureCenters', nsamp, 1); %'
E = randn(nsamp, dim)*sqrt(noiseVariance); %noise should be mean zero
B = 2*eye(dim);
Y = X*B+E;
% without constant:
B_hat = mvregress(X,Y); %<-------- slow, blows up with high dimension
B_hat2 = X \ Y; %<-------- fast, fine with higher dimensions
norm(B_hat - B_hat2) % show numerical equivalent if basically 0
% with constant:
B_constant_hat = mlrtrain(X,Y) %<-------- slow, blows up with high dimension
B_constant_hat2 = [ones(nsamp, 1), X] \ Y; % <-- fast, and fine with higher dimensions
norm(B_constant_hat - B_constant_hat2) % show numerical equivalent if basically 0
Explanation
I'll assume you have:
An nsamp by dim sized data matrix X.
An nsamp by ny sized matrix of outcome variables Y
You want the results from regressing each column of Y on data matrix X. That is, we're doing multivariate regression but there's a common data matrix X.
That is, we're estimating:
y_{ij} = \sum_k b_k * x_{ik} + e_{ijk} for i=1...nsamp, j = 1...ny, k=1...dim
If you're trying to do something different than this, you need to clearly state what you're trying to do!
To regress Y on X you could do:
[beta_mvr, sigma_mvr, resid_mvr] = mvregress(X, Y);
This appears to be horribly slow. The following should match mvregress for the case where you're using the same data matrix for each regression.
beta_hat = X \ Y; % estimate beta using least squares
resid = Y - X * beta_hat; % calculate residual
If you want to construct a new data matrix with a vector of ones, you would do:
X_withones = [ones(nsamp, 1), X];
Further clarification for some that are confused
Let's say we want to run the regression
y_i = \sum_j x_{ij} + e_i i=1...n, j=1...k
We can construct the data matrix n by k datamatrix X and an n by 1 outcome vector y. The OLS estimate is bhat = pinv(X' * X) * X' * y which can also be computed in MATLAB with bhat = X \ y.
If you want to do this multiple times (i.e. run multivariate regression on the same data matrix X), you can construct an outcome matrix Y where EACH column represents a separate outcome variable. Y = [ya, yb, yc, ...]. Trivially, the OLS solution is B = pinv(X'*X)*X'*Y which can be computed as B = X \ Y. The first column of B is the result of regressing Y(:,1) on X. The second column of B is the result of regressing Y(:,2) on X, etc... Under these conditions, this is equivalent to a call to B = mvregress(X, Y)
Even more test code
If regressors are the same and estimation is by simple OLS, there is an equivalence between multivariate regression and equation by equation ordinary least squares.
d = 10;
k = 15;
n = 100;
C = RandomCorr(d + k, 1); %Use any method you like to generate a random correlation matrix
s = randn(d+k , 1) * 10;
S = (s * s') .* C; % generate covariance matrix
mu = randn(d+k,1);
data = mvnrnd(ones(n, 1) * mu', S);
Y = data(:,1:d);
X = data(:,d+1:end);
[b1, sigma] = mvregress(X, Y);
b2 = X \ Y;
norm(b1 - b2)
You will notice b1 and b2 are numerically equivalent. They are equivalent even though sigma is EXTREMELY different from zero.

Use fminsearch to find local minimizer and the minima at that value

I am having trouble using fminsearch: getting the error that there were not enough input arguments for my function.
f = #(x1,x2,x3) x1.^2 + 3.*x2.^2 + 4.*x3.^2 - 2.*x1.*x2 + 5.*x1 + 3.*x2 + 2.*x3;
[x, val] = fminsearch(f,0)
Is there something wrong with my function? I keep getting errors anytime I want to use it as an input function with any other command.
I am having trouble using fminsearch [...]
Stop right there and think some more about the function you're trying to minimize.
Numerical optimization (which is what fminsearch does) is unnecessary, here. Your function is a quadratic function of vector x; in other words, its value at x can be expressed as
x^T A x + b^T x
where matrix A and vector b are defined as follows (using MATLAB notation):
A = [ 1 -1 0;
-1 3 0;
0 0 4]
and
b = [5 3 2].'
Because A is positive definite, your function has one and only one minimum, which can be computed in MATLAB with
x_sol = -0.5 * A \ b;
Now, if you're curious about the cause of the error you ran into, have a look at fuesika's answer; but do without fminsearch whenever you can.
It is exactly what Matlab is telling you: your function expects three arguments. You are passing only one.
Instead of
[x, val] = fminsearch(f,0)
you should call it like
[x, val] = fminsearch(f,[0,0,0])
since you define the function f to accept a three dimensional vector as input only.
You can read more about the specification of fminsearch in the online documentation at http://mathworks.com/help/matlab/ref/fminsearch.html:
x = fminsearch(fun,x0) starts at the point x0 and returns a value x
that is a local minimizer of the function described in fun. x0 can be
a scalar, vector, or matrix. fun is a function_handle.

Why does my function return two values when I only return one?

So I'm trying to implement the Simpson method in Matlab, this is my code:
function q = simpson(x,f)
n = size(x);
%subtracting the last value of the x vector with the first one
ba = x(n) - x(1);
%adding all the values of the f vector which are in even places starting from f(2)
a = 2*f(2:2:end-1);
%adding all the values of the f vector which are in odd places starting from 1
b = 4*f(1:2:end-1);
%the result is the Simpson approximation of the values given
q = ((ba)/3*n)*(f(1) + f(n) + a + b);
This is the error I'm getting:
Error using ==> mtimes
Inner matrix dimensions must agree.
For some reason even if I set q to be
q = f(n)
As a result I get:
q =
0 1
Instead of
q =
0
When I set q to be
q = f(1)
I get:
q =
0
q =
0
I can't explain this behavior, that's probably why I get the error mentioned above. So why does q have two values instead of one?
edit: x = linspace(0,pi/2,12);
f = sin(x);
size(x) returns the size of the array. This will be a vector with all the dimensions of the matrix. There must be at least two dimensions.
In your case n=size(x) will give n=[N, 1], not just the length of the array as you desire. This will mean than ba will have 2 elements.
You can fix this be using length(x) which returns the longest dimension rather than size (or numel(x) or size(x, 1) or 2 depending on how x is defined which returns only the numbered dimension).
Also you want to sum over in a and b whereas now you just create an vector with these elements in. try changing it to a=2*sum(f(...)) and similar for b.
The error occurs because you are doing matrix multiplication of two vectors with different dimensions which isn't allowed. If you change the code all the values should be scalars so it should work.
To get the correct answer (3*n) should also be in brackets as matlab doesn't prefer between / and * (http://uk.mathworks.com/help/matlab/matlab_prog/operator-precedence.html). Your version does (ba/3)*n which is wrong.

matlab constant anonymous function returns only one value instead of an array

I've been searching the net for a couple of mornings and found nothing, hope you can help.
I have an anonymous function like this
f = #(x,y) [sin(2*pi*x).*cos(2*pi*y), cos(2*pi*x).*sin(2*pi*y)];
that needs to be evaluated on an array of points, something like
x = 0:0.1:1;
y = 0:0.1:1;
w = f(x',y');
Now, in the above example everything works fine, the result w is a 11x2 matrix with in each row the correct value f(x(i), y(i)).
The problem comes when I change my function to have constant values:
f = #(x,y) [0, 1];
Now, even with array inputs like before, I only get out a 1x2 array like w = [0,1];
while of course I want to have the same structure as before, i.e. a 11x2 matrix.
I have no idea why Matlab is doing this...
EDIT 1
Sorry, I thought it was pretty clear from what I wrote in the original question, but I see some of you asking, so here is a clarification: what I want is to have again a 11x2 matrix, since I am feeding the function with arrays with 11 elements.
This means I expect to have an output exactly like in the first example, just with changed values in it: a matrix with 11 rows and 2 columns, with only values 0 in the first column and only values 1 in the second, since for all x(i) and y(i) the answer should be the vector [0,1].
It means I expect to have:
w = [0 1
0 1
0 1
...
0 1]
seems pretty natural to me...
You are defining a function f = #(x,y) [0, 1]; which has the input parameters x,y and the output [0,1]. What else do you expect to happen?
Update:
This should match your description:
g=#(x,y)[zeros(size(x)),ones(size(y))]
g(x',y')
Defining an anonymous function f as
f = #(x,y) [0,1];
naturally returns [0,1] for any inputs x and y regardless of the length of those vectors.
This behavior puzzled me also until I realized that I expected f(a,b) to loop over a and b as if I had written
for inc = 1:length(a)
f(a(inc), b(inc))
end
However, f(a,b) does not loop over the length of its inputs, so it merely returns [0,1] regardless of the length of a and b.
The desired behavior can be obtained by defining f as
g=#(x,y)[zeros(size(x)),ones(size(y))]
as Daniel stated in his answer.

MATLAB: Test if anonymous vector is a subset of R^n

I'm trying to use MatLab code as a way to learn math as a programmer.
So reading I'm this post about subspaces and trying to build some simple matlab functions that do it for me.
Here is how far I got:
function performSubspaceTest(subset, numArgs)
% Just a quick and dirty function to perform subspace test on a vector(subset)
%
% INPUT
% subset is the anonymous function that defines the vector
% numArgs is the the number of argument that subset takes
% Author: Lasse Nørfeldt (Norfeldt)
% Date: 2012-05-30
% License: http://creativecommons.org/licenses/by-sa/3.0/
if numArgs == 1
subspaceTest = #(subset) single(rref(subset(rand)+subset(rand))) ...
== single(rref(rand*subset(rand)));
elseif numArgs == 2
subspaceTest = #(subset) single(rref(subset(rand,rand)+subset(rand,rand))) ...
== single(rref(rand*subset(rand,rand)));
end
% rand just gives a random number. Converting to single avoids round off
% errors.
% Know that the code can crash if numArgs isn't given or bigger than 2.
outcome = subspaceTest(subset);
if outcome == true
display(['subset IS a subspace of R^' num2str(size(outcome,2))])
else
display(['subset is NOT a subspace of R^' num2str(size(outcome,2))])
end
And these are the subset that I'm testing
%% Checking for subspaces
V = #(x) [x, 3*x]
performSubspaceTest(V, 1)
A = #(x) [x, 3*x+1]
performSubspaceTest(A, 1)
B = #(x) [x, x^2, x^3]
performSubspaceTest(B, 1)
C = #(x1, x3) [x1, 0, x3, -5*x1]
performSubspaceTest(C, 2)
running the code gives me this
V =
#(x)[x,3*x]
subset IS a subspace of R^2
A =
#(x)[x,3*x+1]
subset is NOT a subspace of R^2
B =
#(x)[x,x^2,x^3]
subset is NOT a subspace of R^3
C =
#(x1,x3)[x1,0,x3,-5*x1]
subset is NOT a subspace of R^4
The C is not working (only works if it only accepts one arg).
I know that my solution for numArgs is not optimal - but it was what I could come up with at the current moment..
Are there any way to optimize this code so C will work properly and perhaps avoid the elseif statments for more than 2 args..?
PS: I couldn't seem to find a build-in matlab function that does the hole thing for me..
Here's one approach. It tests if a given function represents a linear subspace or not. Technically it is only a probabilistic test, but the chance of it failing is vanishingly small.
First, we define a nice abstraction. This higher order function takes a function as its first argument, and applies the function to every row of the matrix x. This allows us to test many arguments to func at the same time.
function y = apply(func,x)
for k = 1:size(x,1)
y(k,:) = func(x(k,:));
end
Now we write the core function. Here func is a function of one argument (presumed to be a vector in R^m) which returns a vector in R^n. We apply func to 100 randomly selected vectors in R^m to get an output matrix. If func represents a linear subspace, then the rank of the output will be less than or equal to m.
function result = isSubspace(func,m)
inputs = rand(100,m);
outputs = apply(func,inputs);
result = rank(outputs) <= m;
Here it is in action. Note that the functions take only a single argument - where you wrote c(x1,x2)=[x1,0,x2] I write c(x) = [x(1),0,x(2)], which is slightly more verbose, but has the advantage that we don't have to mess around with if statements to decide how many arguments our function has - and this works for functions that take input in R^m for any m, not just 1 or 2.
>> v = #(x) [x,3*x]
>> isSubspace(v,1)
ans =
1
>> a = #(x) [x(1),3*x(1)+1]
>> isSubspace(a,1)
ans =
0
>> c = #(x) [x(1),0,x(2),-5*x(1)]
>> isSubspace(c,2)
ans =
1
The solution of not being optimal barely scratches the surface of the problem.
I think you're doing too much at once: rref should not be used and is complicating everything. especially for numArgs greater then 1.
Think it through: [1 0 3 -5] and [3 0 3 -5] are both members of C, but their sum [4 0 6 -10] (which belongs in C) is not linear product of the multiplication of one of the previous vectors (e.g. [2 0 6 -10] ). So all the rref in the world can't fix your problem.
So what can you do instead?
you need to check if
(randn*subset(randn,randn)+randn*subset(randn,randn)))
is a member of C, which, unless I'm mistaken is a difficult problem: Conceptually you need to iterate through every element of the vector and make sure it matches the predetermined condition. Alternatively, you can try to find a set such that C(x1,x2) gives you the right answer. In this case, you can use fminsearch to solve this problem numerically and verify the returned value is within a defined tolerance:
[s,error] = fminsearch(#(x) norm(C(x(1),x(2)) - [2 0 6 -10]),[1 1])
s =
1.999996976386119 6.000035034493023
error =
3.827680714104862e-05
Edit: you need to make sure you can use negative numbers in your multiplication, so don't use rand, but use something else. I changed it to randn.