Related
I am trying to evaluate the matrices Y(p,k) and Z(p,k) using the following simplified Matlab code.
They depend on some matrices A(j,k), B(j,p) and C(j,k) which I am able to precalculate, so I have just initialised them as random arrays for this MWE. (Note that B is a different size to A and C).
Nj = 5000;
Nk = 1000;
Np = 500; % max loop iterations
A = rand(Nj,Nk); % dummy precalculated matrices
B = rand(Nj,Np);
C = rand(Nj,Nk);
Y = zeros(Np,Nk); % allocate storage
Z = zeros(Np,Nk);
tic
for p = 1:Np
X = A .* B(:,p);
Y(p,:) = sum( X , 1 );
Z(p,:) = sum( C .* X , 1 );
end
toc % Evaluates to 11 seconds on my system
As can be seen above, I am repeating my calculation by looping over index p (because the matrix B depends on p).
I have managed to get this far by moving everything which can be precalculated outside the loop (contained in A, B and C), but on my system this code still takes around 11 seconds to execute. Can anyone see a way in Matlab to speed this up, or perhaps even remove the loop and process all at once?
Thank you
I think the following should be equivalent and much faster:
Y = B' * A;
Z = B' * (A.*C);
Notes:
If B is complex-valued then you should use .' for transposition instead.
You may also want to pre-compute B directly in transposed form (i.e. as a Np by Nj matrix) to avoid the transposition altogether.
If C is not needed anywhere else, then pre-compute it as A.*C instead in order to avoid the extra element-wise multiplication.
I am trying to solve a linear system (A*x=B) where B is a 64-bit length or more. I have used linsolve function in Matlab to solve the system of equations. I have used (inv(A)*B), A\B, and ttimes(A,B) as well, and they suffer from the same issues.
I am facing two problems:
linslove function cannot find the exact solution if ( A and B) are not symbolic.
If A and B are symbolic, linsolve manage to find the exact solution but it takes too much time.
Is there any way to find the exact solution but fast.
time=[]
i=50
a=magic(i);
% B is a rendom numbers where each number is 64 bit length
B=double(bi2de(randi([0 1],i,64)));
%%****************************************
% to make sure th matrix is not **ill-conditioned***
C = 1; % desired condition number
[u s v] = svd(a);
s = diag(s); % s is vector
% ===== linear stretch of existing s
s = s(1)*( 1-((C-1)/C)*(s(1)-s)/(s(1)-s(end)));
% =====
s = diag(s); % back to matrix
A = u*s*v';
%%****************************************
tic
x1=linsolve(A,B);
time(1,1)=toc;
%-------------------------------------
% convert A and B into symbolic
Aa=sym(A); Bb=sym(B);
tic
x2=linsolve(Aa,Bb);
time(1,2)=toc;
%-------------------------------------
% Show the accuracy of the first B(1), exact vs computed
Exact=sym(double(B(1)))
Computed=[ A(1,:)*x1 Aa(1,:)*x2]
time
x1 and x2 are the two solution. x2 is the solution of the sumbolic A and B.
Only X2 gives us the exact solution
Exact solution = 2350911785583947776
Computed using x1= 2350911785583965184
Computed using x2= 2350911785583947776
Time required in seconds:
x1 time = 0.0007
x2 time = 6.7242
This is not an answer to your question, it is a demonstration of why your "exact" solution is not exact: the intput B is approximated. Try this in MATLAB:
a = randi([0 1],1,64);
a(1) = 0;
a1 = bi2de(a);
a(1) = 1;
a2 = bi2de(a);
a1-a2
You'll notice that a1 and a2 are identical, even though I flipped the least significant bit in both numbers. This is because a double-precision float cannot hold 64 bits of precision. It holds only 52. The other 12 bits in the 64-bit representation are to store the sign bit and the exponent (which scales the number).
I have built my own bi2de that converts the binary vector to an actual number.
I have replaced sym by vpa, and the performance enhanced.
sym(A) ----> vpa(A)
I am trying to use mvregress with the data I have with dimensionality of a couple of hundreds. (3~4). Using 32 gb of ram, I can not compute beta and I get "out of memory" message. I couldn't find any limitation of use for mvregress that prevents me to apply it on vectors with this degree of dimensionality, am I doing something wrong? is there any way to use multivar linear regression via my data?
here is an example of what goes wrong:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
%without residual term:
A_hat=mvregress(X',Y');
%wit residual term:
[B, y_hat]=mlrtrain(X,Y)
where
function [B, y_hat]=mlrtrain(X,Y)
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = [kron([Xmat(i,:)],eye(d))];
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
end
the error is:
Error using bsxfun
Out of memory. Type HELP MEMORY for your options.
Error in kron (line 36)
K = reshape(bsxfun(#times,A,B),[ma*mb na*nb]);
Error in mvregress (line 319)
c{j} = kron(eye(NumSeries),Design(j,:));
and this is result of whos command:
whos
Name Size Bytes Class Attributes
A 400x400 1280000 double
N 400x1000 3200000 double
X 400x1000 3200000 double
Y 400x1000 3200000 double
dataVariance 1x1 8 double
dim 1x1 8 double
mixtureCenters 400x1 3200 double
noiseVariance 1x1 8 double
nsamp 1x1 8 double
Okay, I think I have a solution for you, short version first:
dim=400;
nsamp=1000;
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X=randn(dim, nsamp)*sqrt(dataVariance ) + repmat(mixtureCenters,1,nsamp);
N=randn(dim, nsamp)*sqrt(noiseVariance ) + repmat(mixtureCenters,1,nsamp);
A=2*eye(dim);
Y=A*X+N;
[n,d] = size(Y);
Xmat = [ones(n,1) X];
Xmat_sz=size(Xmat);
Xcell = cell(1,n);
for i = 1:n
Xcell{i} = kron(Xmat(i,:),speye(d));
end
[beta,sigma,E,V] = mvregress(Xcell,Y);
B = reshape(beta,d,Xmat_sz(2))';
y_hat=Xmat * B ;
Strangely, I could not access the function's workspace, it did not appear in the call stack. This is why I put the function after the script here.
Here's the explanation that might also help you in the future:
Looking at the kron definition, the result when inserting an m by n and a p by q matrix has size mxp by nxq, in your case 400 by 1001 and 1000 by 1000, that makes a 400000 by 1001000 matrix, which has 4*10^11 elements. Now you have four hundred of them, and each element takes up 8 bytes for double precision, that is a total size of about 1.281 Petabytes of memory (or 1.138 Pebibytes, if you prefer), well out of reach even with your grand 32 Gibibyte.
Seeing that one of your matrices, the eye one, contains mostly zeros, and the resulting matrix contains all possible element product combinations, most of them will be zero, too. For such cases specifically, MATLAB offers the sparse matrix format, which saves a lot of memory depending on the number of zero elements in a matrix by only storing nonzero ones. You can convert a full matrix to a sparse representation with sparse(X), or you get an eye matrix directly by using speye(n), which is what I did above. The sparse property propagates to the result, which you should now have enough memory for (I have with 1/4 of your memory available, and it works).
However, what remains is the problem Matthew Gunn mentioned in a comment. I get an error saying:
Error using mvregress (line 260)
Insufficient data to estimate either full or least-squares models.
Preface
If your regressors are all the same across each regression equation and you're interested in the OLS estimate, you can replace a call to mvregress with a simple call to \.
It appears in the call to mlrtrain you had a matrix transposition error (since corrected). In the language of mvregress, n is the number of observations, d is the number of outcome variables. You generate a matrix Y that is d by n. But THEN when you should call mlrtrain(X', Y') not mlrtrain(X, Y).
If below isn't specifically, what you're looking for, I suggest you precisely define what you're trying to estimate.
What I would have written if I were you
So much that's been said here is completely off base that I'm posting code of what I would have written if I were you. I've reduced the dimensionality to show the equivalence in your special case to simply calling \. I've also written stuff in a more standard way (i.e. having observations run down the rows and not making matrix transposition errors).
dim=5; % These can go way higher but only if you use my code
nsamp=20; % rather than call mvregress
dataVariance = .10;
noiseVariance = .05;
mixtureCenters=randn(dim,1);
X = randn(nsamp, dim)*sqrt(dataVariance ) + repmat(mixtureCenters', nsamp, 1); %'
E = randn(nsamp, dim)*sqrt(noiseVariance); %noise should be mean zero
B = 2*eye(dim);
Y = X*B+E;
% without constant:
B_hat = mvregress(X,Y); %<-------- slow, blows up with high dimension
B_hat2 = X \ Y; %<-------- fast, fine with higher dimensions
norm(B_hat - B_hat2) % show numerical equivalent if basically 0
% with constant:
B_constant_hat = mlrtrain(X,Y) %<-------- slow, blows up with high dimension
B_constant_hat2 = [ones(nsamp, 1), X] \ Y; % <-- fast, and fine with higher dimensions
norm(B_constant_hat - B_constant_hat2) % show numerical equivalent if basically 0
Explanation
I'll assume you have:
An nsamp by dim sized data matrix X.
An nsamp by ny sized matrix of outcome variables Y
You want the results from regressing each column of Y on data matrix X. That is, we're doing multivariate regression but there's a common data matrix X.
That is, we're estimating:
y_{ij} = \sum_k b_k * x_{ik} + e_{ijk} for i=1...nsamp, j = 1...ny, k=1...dim
If you're trying to do something different than this, you need to clearly state what you're trying to do!
To regress Y on X you could do:
[beta_mvr, sigma_mvr, resid_mvr] = mvregress(X, Y);
This appears to be horribly slow. The following should match mvregress for the case where you're using the same data matrix for each regression.
beta_hat = X \ Y; % estimate beta using least squares
resid = Y - X * beta_hat; % calculate residual
If you want to construct a new data matrix with a vector of ones, you would do:
X_withones = [ones(nsamp, 1), X];
Further clarification for some that are confused
Let's say we want to run the regression
y_i = \sum_j x_{ij} + e_i i=1...n, j=1...k
We can construct the data matrix n by k datamatrix X and an n by 1 outcome vector y. The OLS estimate is bhat = pinv(X' * X) * X' * y which can also be computed in MATLAB with bhat = X \ y.
If you want to do this multiple times (i.e. run multivariate regression on the same data matrix X), you can construct an outcome matrix Y where EACH column represents a separate outcome variable. Y = [ya, yb, yc, ...]. Trivially, the OLS solution is B = pinv(X'*X)*X'*Y which can be computed as B = X \ Y. The first column of B is the result of regressing Y(:,1) on X. The second column of B is the result of regressing Y(:,2) on X, etc... Under these conditions, this is equivalent to a call to B = mvregress(X, Y)
Even more test code
If regressors are the same and estimation is by simple OLS, there is an equivalence between multivariate regression and equation by equation ordinary least squares.
d = 10;
k = 15;
n = 100;
C = RandomCorr(d + k, 1); %Use any method you like to generate a random correlation matrix
s = randn(d+k , 1) * 10;
S = (s * s') .* C; % generate covariance matrix
mu = randn(d+k,1);
data = mvnrnd(ones(n, 1) * mu', S);
Y = data(:,1:d);
X = data(:,d+1:end);
[b1, sigma] = mvregress(X, Y);
b2 = X \ Y;
norm(b1 - b2)
You will notice b1 and b2 are numerically equivalent. They are equivalent even though sigma is EXTREMELY different from zero.
Problem : How do I use a continuous map - The Link1: Bernoulli Shift Map to model binary sequence?
Concept :
The Dyadic map also called as the Bernoulli Shift map is expressed as x(k+1) = 2x(k) mod 1. In Link2: Symbolic Dynamics, explains that the Bernoulli Map is a continuous map and is used as the Shift Map. This is explained further below.
A numeric trajectory can be symbolized by partitioning into appropriate regions and assigning it with a symbol. A symbolic orbit is obtained by writing down the sequence of symbols corresponding to the successive partition elements visited by the point in its orbit. One can learn much about the dynamics of the system by studying its symbolic orbits. This link also says that the Bernoulli Shift Map is used to represent symbolic dynamics.
Question :
How is the Bernoulli Shift Map used to generate the binary sequence? I tried like this, but this is not what the document in Link2 explains. So, I took the numeric output of the Map and converted to symbols by thresholding in the following way:
x = rand();
y = mod(2* x,1) % generate the next value after one iteration
y =
0.3295
if y >= 0.5 then s = 1
else s = 0
where 0.5 is the threshold value, called the critical value of the Bernoulli Map.
I need to represent the real number as fractions as explained here on Page 2 of Link2.
Can somebody please show how I can apply the Bernoulli Shift Map to generate symbolized trajectory (also called time series) ?
Please correct me if my understanding is wrong.
How do I convert a real valued numeric time series into symbolized i.e., how do I use the Bernoulli Map to model binary orbit /time series?
You can certainly compute this in real number space, but you risk hitting precision problems (depending on starting point). If you're interested in studying orbits, you may prefer to work in a rational fraction representation. There are more efficient ways to do this, but the following code illustrates one way to compute a series derived from that map. You'll see the period-n definition on page 2 of your Link 2. You should be able to see from this code how you could easily work in real number space as an alternative (in that case, the matlab function rat will recover a rational approximation from your real number).
[EDIT] Now with binary sequence made explicit!
% start at some point on period-n orbit
period = 6;
num = 3;
den = 2^period-1;
% compute for this many steps of the sequence
num_steps = 20;
% for each step
for n = 1:num_steps
% * 2
num = num * 2;
% mod 1
if num >= den
num = num - den;
end
% simplify rational fraction
g = gcd(num, den);
if g > 1
num = num / g;
den = den / g;
end
% recover 8-bit binary representation
bits = 8;
q = 2^bits;
x = num / den * q;
b = dec2bin(x, bits);
% display
fprintf('%4i / %4i == 0.%s\n', num, den, b);
end
Ach... for completeness, here's the real-valued version. Pure mathematicians should look away now.
% start at some point on period-n orbit
period = 6;
num = 3;
den = 2^period-1;
% use floating point approximation
x = num / den;
% compute for this many steps of the sequence
num_steps = 20;
% for each step
for n = 1:num_steps
% apply map
x = mod(x*2, 1);
% display
[num, den] = rat(x);
fprintf('%i / %i\n', num, den);
end
And, for extra credit, why is this implementation fast but daft? (HINT: try setting num_steps to 50)...
% matlab vectorised version
period = 6;
num = 3;
den = 2^period-1;
x = zeros(1, num_steps);
x(1) = num / den;
y = filter(1, [1 -2], x);
[a, b] = rat(mod(y, 1));
disp([a' b']);
OK, this is supposed to be an answer, not a question, so let's answer my own questions...
It's fast because it uses Matlab's built-in (and highly optimised) filter function to handle the iteration (that is, in practice, the iteration is done in C rather than in M-script). It's always worth remembering filter in Matlab, I'm constantly surprised by how it can be turned to good use for applications that don't look like filtering problems. filter cannot do conditional processing, however, and does not support modulo arithmetic, so how do we get away with it? Simply because this map has the property that whole periods at the input map to whole periods at the output (because the map operation is multiply by an integer).
It's daft because it very quickly hits the aforementioned precision problems. Set num_steps to 50 and watch it start to get wrong answers. What's happening is the number inside the filter operation is getting to be so large (order 10^14) that the bit we actually care about (the fractional part) is no longer representable in the same double-precision variable.
This last bit is something of a diversion, which has more to do with computation than maths - stick to the first implementation if your interest lies in symbol sequences.
If you only want to deal with rational type of output, you'll first have to convert the starting term of your series into a rational number if it is not. You can do that with:
[N,D] = rat(x0) ;
Once you have a numerator N and a denominator D, it is very easy to calculate the series x(k+1)=mod(2*x(k), 1) , and you don't even need a loop.
for the part 2*x(k), it means all the Numerator(k) will be multiplied by successive power of 2, which can be done by matrix multiplication (or bsxfun for the lover of the function):
so 2*x(k) => in Matlab N.*(2.^(0:n-1)) (N is a scalar, the numerator of x0, n is the number of terms you want to calculate).
The Mod1 operation is also easy to translate to rational number: mod(x,1)=mod(Nx,Dx)/Dx (Nx and Dx being the numerator and denominator of x.
If you do not need to simplify the denominator, you could get all the numerators of the series in one single line:
xn = mod( N.*(2.^(0:n-1).'),D) ;
but for visual comfort, it is sometimes better to simplify, so consider the following function:
function y = dyadic_rat(x0,n)
[N,D] = rat(x0) ; %// get Numerator and Denominator of first element
xn = mod( N.*(2.^(0:n-1).'),D) ; %'// calculate all Numerators
G = gcd( xn , D ) ; %// list all "Greatest common divisor"
y = [xn./G D./G].' ; %'// output simplified Numerators and Denominators
If I start with the example given in your wiki link (x0=11/24), I get:
>> y = dyadic_rat(11/24,8)
y =
11 11 5 2 1 2 1 2
24 12 6 3 3 3 3 3
If I start with the example given by Rattus Ex Machina (x0=3/(2^6-1)), I also get the same result:
>> y = dyadic_rat(3/63,8)
y =
1 2 4 8 16 11 1 2
21 21 21 21 21 21 21 21
Just wondering... I tried doing by hand (with the multiply and square method) the operation (111^11)mod143 and I got the result 67. I also checked that this is correct, in many online tools. Yet, in matlab plugging:
mod(111^11,143)
gives 127! Is there any particular reason for this? I didn't find anything in the documentation...
The value of 111^11 (about 3.1518e+022) exceeds the maximum integer that is guaranteed to be represented exactly as a double, which is 2^53 (about 9.0072e+015). So the result is spoilt by insufficient numerical precision.
To achieve the correct result, use symbolic computation:
>> syms x y z
>> r = mod(x^y, z);
>> subs(r, [x y z], [111 11 143])
ans =
67
Alternatively, for this specific operation (modulo of a large number that is expressed as a product of small numbers), you can do the computation very easily using the following fact (where ∗ denotes product):
mod(a∗b, z) = mod(mod(a,z)∗mod(b,z), z)
That is, you can apply the modulo operation to factors of your large number and the final result is unchanged. If you choose factors sufficiently small so that they can be represented exactly as double, you can do the computation numerically without any loss of precision.
For example: using the decomposition 111^11 = 111^4*111^4*111^3, since all factors are small enough, gives the correct result:
>> mod((mod(111^4, 143))^2 * mod(111^3, 143), 143)
ans =
67
Similarly, using 111^2 and 111 as factors,
>> mod((mod(111^2, 143))^5 * mod(111, 143), 143)
ans =
67
from the matlab website they recommend using powermod(b, e, m) (b^e mod m)
"If b and m are numbers, the modular power b^e mod m can also be computed by the direct call b^e mod m. However, powermod(b, e, m) avoids the overhead of computing the intermediate result be and computes the modular power much more efficiently." ...
Another way is to use symfun
syms x y z
f = symfun(mod(x^y,z), [x y z])
f(111,11,143)