I have two matrices in the form of the variables A and b that are inputs
to my matlab function (posted below). I would like to calculate the
significant figures used for the matrix inverse operation (matrix division)
from the result A, b matrices. However, I have no idea where to start
(matlab or mathematically) to go about this approach. Help?
More context, using a square linear system (Ax=b) and I am seeing if its
singular or nonsingular and trying to find a solution(s).
% x = answer
% y = 0 if no solution, 1 if nonsingular, 2 if many solutions
% z = p is number of sig figs
%
function [ x, y, z ] = squareLinSysSolv(A, b)
if det(A) == 0
% Matrix is singular and therefor many solutions
x = A\b;
y = 0; % Used as place holder to compile
z = 5; % Used as place holder to compile
elseif det(A) ~= 0
% Matrix does not equal to zero (perhaps a number very close to it or
% far from it) and therefor has a unique solution.
x = A\b;
y = 1; % Used as place holder to compile
z = 5; % Used as place holder to compile
end
end
Edit:
To make it a bit clear, z should be some integer which approximates (ceiling or floor value) a decimal number of significant figures at which A\b were calculated at.
Test Cases:
Test/Spec sheet of what is expected. Both A and b are matrices and the result should be something like so.
A =
1.5000 2.3000 7.9000
6.1000 3.2000 13.0000
13.0000 21.0000 76.0000
b =
1
3
5
>> [x,y,z] = squareLinSysSolv(A,b)
% the result of x = A\b
x =
0.8580
3.0118
-0.9132
% determinant is not equal to zero
y =
1
% Amount of sig figs/precision in calculation
z =
15
I'm with Dan here. I don't understand the question, nor do I see how/where you're calculating z. First of all, the number digits in the display of your answer is independent of the number of significant digits in the calculation of the answer. Second, you have two cases: det(A)==0, and det(A)~=0. In both cases, it appears that you are setting z = 5. (You must be doing something elsewhere that you're not showing to calculate z = 15? How did you calculate your z?)
Also, recognize that the number of significant digits will be a function of the class of your data. Double>single>integer...
And...just for efficiency, I don't know the size of A (nor how much overhead is entailed in computing its determinant), but there's no reason to compute it twice:
if det(A)==0
% case 1
else % NOT: elseif det(A)~=0
% case 2
end
I guess I'm being dense, too. :)
Brett
How close a matrix is to being singular is typically quantified by a "condition number". There are several ways to estimate the condition number of a matrix. One common technique is to calculate the ratio of the largest and smallest magnitude eigenvalues.
Matlab has a built-in function called cond that gives the condition number for a matrix (with respect to inversion). Values near 1 are "good"--I'm not sure how to relate it to anything concrete like "significant figures".
If the question that you are asking is "Given matlab's decimal representation of a floating point number, what is the smallest number z such that after the first z digits all digits are zero", then I think the following function will more-or-less work.
I will add the caveat that although I've tested this on a bunch of numbers, it's quite possible that I've missed a case, so you will want to test it carefully. This could be done more efficiently in a single pass through the string.
function n = sig_decimal_digits(f)
% get string representation, any decimal point, and any leading zeros
str = num2str(f,20);
% strip any exponent part
loc = regexp(str, 'e');
if ~isempty(loc)
str = str(1:(loc-1));
end
% strip any decimal point
str = strrep(str, '.', '');
% strip any leading zeros (and minus sign!)
loc = regexp(str, '[1-9]');
if isempty(loc)
%if *only* leading zeros, f is 0, so n = 1
n = 1;
return;
else
str = str(loc:end);
end
% count length of string, excluding trailing zeros
loc = regexp(str, '0+$');
if isempty(loc)
n = length(str);
else
n = loc-1;
end
end
However, I will add two comments:
This clearly has nothing to do with matrix multiplication, so I'm not sure why you brought that into it.
This is a strange quantity to want to calculate. Unless your matrices have very special numbers in them, the answer is nearly always going to be 17, since most floating point numbers don't have short decimal expansions. For example, for the A and b that you give in your question, all three numbers in x have 17 significant digits according to my function.
Related
I would like to generate a vector of 20 random numbers knowing the standard deviation, the mean and contaning two 0 values. Is it possible?
These are the desired values of mean and standard deviation:
Mean: 3.3
Standard deviation: 25%
This seems to work fine: https://fr.mathworks.com/help/matlab/math/random-numbers-with-specific-mean-and-variance.html however, I need to have in my vector two 0 values.
You can do this in a few steps, as detailed in the comments. The "at least" two zeros condition is easier than "exactly" two zeros would be - in the latter case you would have to write a quick function to return randn values except zero and then make sure the distribution shifting doesn't create new zeros.
% Create an array of length N+z (=20 here), which is N=18 randn values and z zeros
% Could make this more complicated to limit/expand on specific values
N = 18;
z = 2;
v = [randn(N,1); zeros(z,1)];
% Define a target mean (mu) and standard dev (sd)
mu = 3.3;
sd = 25;
% We have to iterate several times (100 is arbitrary) to get closer to both
% the s.d. and mean criteria. More passes for arbitrary accuracy
for ii = 1:100
% Scale the vector to satisfy the standard dev
v = sd*v/std(v);
% Shift the vector to satisfy the mean, but only the non-zero randn values
v(1:N) = v(1:N) - mean(v(1:N)) + ((N+z)/N)*mu;
end
% Output check
fprintf( 'Standard dev: %.4f\nMean: %.4f\n', std(v), mean(v) );
% >> Standard dev: 25.0000
% >> Mean: 3.3000
Note that by forcing specific values this isn't necessarily a perfectly normally distributed set of numbers any more, but it does satisfy your mean and standard deviation conditions. This is an issue with your problem statement - any solution will have this footnote.
I'm kind've new to Matlab and stack overflow to begin with, so if I do something wrong outside of the guidelines, please don't hesitate to point it out. Thanks!
I have been trying to do convolution between two functions and I have been having a hard time trying to get it to work.
t=0:.01:10;
h=exp(-t);
x=zeros(size(t)); % When I used length(t), I would get an error that says in conv(), A and B must be vectors.
x(1)=2;
x(4)=5;
y=conv(h,x);
figure; subplot(3,1,1);plot(t,x); % The discrete function would not show (at x=1 and x=4)
subplot(3,1,2);plot(t,h);
subplot(3,1,3);plot(t,y(1:length(t))); %Nothing is plotted here when ran
I commented my issues with the code. I don't understand the difference of length and size in this case and how it would make a difference.
For the second comment, x=1 should have an amplitude of 2. While x=4 should have an amplitude of 5. When plotted, it only shows nothing in the locations specified but looks jumbled up at x=0. I'm assuming that's the reason why the convoluted plot won't be displayed.
The original problem statement is given if it helps to understand what I was thinking throughout.
Consider an input signal x(t) that consists of two delta functions at t = 1 and t = 4 with amplitudes A1 = 5 and A2 = 2, respectively, to a linear system with impulse response h that is an exponential pulse (h(t) = e ^−t ). Plot x(t), h(t) and the output of the linear system y(t) for t in the range of 0 to 10 using increments of 0.01. Use the MATLAB built-in function conv.
The initial question regarding size vs length
length yields a scalar that is equal to the largest dimension of the input. In the case of your array, the size is 1 x N, so length yields N.
size(t)
% 1 1001
length(t)
% 1001
If you pass a scalar (N) to ones, zeros, or a similar function, it will create a square matrix that is N x N. This results in the error that you see when using conv since conv does not accept matrix inputs.
size(ones(length(t)))
% 1001 1001
When you pass a vector to ones or zeros, the output will be that size so since size returns a vector (as shown above), the output is the same size (and a vector) so conv does not have any issues
size(ones(size(t)))
% 1 1001
If you want a vector, you need to explicitly specify the number of rows and columns. Also, in my opinion, it's better to use numel to the number of elements in a vector as it's less ambiguous than length
z = zeros(1, numel(t));
The second question regarding the convolution output:
First of all, the impulses that you create are at the first and fourth index of x and not at the locations where t = 1 and t = 4. Since you create t using a spacing of 0.01, t(1) actually corresponds to t = 0 and t(4) corresponds to t = 0.03
You instead want to use the value of t to specify where to put your impulses
x(t == 1) = 2;
x(t == 4) = 5;
Note that due to floating point errors, you may not have exactly t == 1 and t == 4 so you can use a small epsilon instead
x(abs(t - 1) < eps) = 2;
x(abs(t - 4) < eps) = 5;
Once we make this change, we get the expected scaled and shifted versions of the input function.
I have found several questions/answers for vectorizing and speeding up routines for multiplying a matrix and a vector in a single loop, but I am trying to do something a little more general, namely multiplying an arbitrary number of matrices together, and then performing that operation an arbitrary number of times.
I am writing a general routine for calculating thin-film reflection from an arbitrary number of layers vs optical frequency. For each optical frequency W each layer has an index of refraction N and an associated 2x2 transfer matrix L and 2x2 interface matrix I which depends on the index of refraction and the thickness of the layer. If n is the number of layers, and m is the number of frequencies, then I can vectorize the index into an n x m matrix, but then in order to calculate the reflection at each frequency, I have to do nested loops. Since I am ultimately using this as part of a fitting routine, anything I can do to speed it up would be greatly appreciated.
This should provide a minimum working example:
W = 1260:0.1:1400; %frequency in cm^-1
N = rand(4,numel(W))+1i*rand(4,numel(W)); %dummy complex index of refraction
D = [0 0.1 0.2 0]/1e4; %thicknesses in cm
[n,m] = size(N);
r = zeros(size(W));
for x = 1:m %loop over frequencies
C = eye(2); % first medium is air
for y = 2:n %loop over layers
na = N(y-1,x);
nb = N(y,x);
%I = InterfaceMatrix(na,nb); % calculate the 2x2 interface matrix
I = [1 na*nb;na*nb 1]; % dummy matrix
%L = TransferMatrix(nb) % calculate the 2x2 transfer matrix
L = [exp(-1i*nb*W(x)*D(y)) 0; 0 exp(+1i*nb*W(x)*D(y))]; % dummy matrix
C = C*I*L;
end
a = C(1,1);
c = C(2,1);
r(x) = c/a; % reflectivity, the answer I want.
end
Running this twice for two different polarizations for a three layer (air/stuff/substrate) problem with 2562 frequencies takes 0.952 seconds while solving the exact same problem with the explicit formula (vectorized) for a three layer system takes 0.0265 seconds. The problem is that beyond 3 layers, the explicit formula rapidly becomes intractable and I would have to have a different subroutine for each number of layers while the above is completely general.
Is there hope for vectorizing this code or otherwise speeding it up?
(edited to add that I've left several things out of the code to shorten it, so please don't try to use this to actually calculate reflectivity)
Edit: In order to clarify, I and L are different for each layer and for each frequency, so they change in each loop. Simply taking the exponent will not work. For a real world example, take the simplest case of a soap bubble in air. There are three layers (air/soap/air) and two interfaces. For a given frequency, the full transfer matrix C is:
C = L_air * I_air2soap * L_soap * I_soap2air * L_air;
and I_air2soap ~= I_soap2air. Thus, I start with L_air = eye(2) and then go down successive layers, computing I_(y-1,y) and L_y, multiplying them with the result from the previous loop, and going on until I get to the bottom of the stack. Then I grab the first and third values, take the ratio, and that is the reflectivity at that frequency. Then I move on to the next frequency and do it all again.
I suspect that the answer is going to somehow involve a block-diagonal matrix for each layer as mentioned below.
Not next to a matlab, so that's only a starter,
Instead of the double loop you can write na*nb as Nab=N(1:end-1,:).*N(2:end,:);
The term in the exponent nb*W(x)*D(y) can be written as e=N(2:end,:)*W'*D;
The result of I*L is a 2x2 block matrix that has this form:
M = [1, Nab; Nab, 1]*[e-, 0;0, e+] = [e- , Nab*e+ ; Nab*e- , e+]
with e- as exp(-1i*e), and e+ as exp(1i*e)'
see kron on how to get the block matrix form, to vectorize the propagation C=C*I*L just take M^n
#Lama put me on the right path by suggesting block matrices, but the ultimate answer ended up being more complicated, and so I put it here for posterity. Since the transfer and interface matrix is different for each layer, I leave in the loop over the layers, but construct a large sparse block matrix where each block represents a frequency.
W = 1260:0.1:1400; %frequency in cm^-1
N = rand(4,numel(W))+1i*rand(4,numel(W)); %dummy complex index of refraction
D = [0 0.1 0.2 0]/1e4; %thicknesses in cm
[n,m] = size(N);
r = zeros(size(W));
C = speye(2*m); % first medium is air
even = 2:2:2*m;
odd = 1:2:2*m-1;
for y = 2:n %loop over layers
na = N(y-1,:);
nb = N(y,:);
% get the reflection and transmission coefficients from subroutines as a vector
% of length m, one value for each frequency
%t = Tab(na, nb);
%r = Rab(na, nb);
t = rand(size(W)); % dummy vector for MWE
r = rand(size(W)); % dummy vector for MWE
% create diagonal and off-diagonal elements. each block is [1 r;r 1]/t
Id(even) = 1./t;
Id(odd) = Id(even);
Io(even) = 0;
Io(odd) = r./t;
It = [Io;Id/2].';
I = spdiags(It,[-1 0],2*m,2*m);
I = I + I.';
b = 1i.*(2*pi*D(n).*nb).*W;
B(even) = -b;
B(odd) = b;
L = spdiags(exp(B).',0,2*m,2*m);
C = C*I*L;
end
a = spdiags(C,0);
a = a(odd).';
c = spdiags(C,-1);
c = c(odd).';
r = c./a; % reflectivity, the answer I want.
With the 3 layer system mentioned above, it isn't quite as fast as the explicit formula, but it's close and probably can get a little faster after some profiling. The full version of the original code clocks at 0.97 seconds, the formula at 0.012 seconds and the sparse diagonal version here at 0.065 seconds.
Background:
Basically I'm using a dynamic time warping algorithm like used in speech recognition to try to warp geological data (filter out noise from environmental conditions) The main difference between these two problems is that dtw prints a warping function that allows both vectors that are input to be warped, whereas for the problem I'm trying to solve I need to keep one reference vector constant while stretching and shrinking the test variable vector to fit.
here is dtw in matlab:
function [Dist,D,k,w]=dtw()
%Dynamic Time Warping Algorithm
%Dist is unnormalized distance between t and r
%D is the accumulated distance matrix
%k is the normalizing factor
%w is the optimal path
%t is the vector you are testing against
%r is the vector you are testing
[t,r,x1,x2]=randomtestdata();
[rows,N]=size(t);
[rows,M]=size(r);
%for n=1:N
% for m=1:M
% d(n,m)=(t(n)-r(m))^2;
% end
%end
d=(repmat(t(:),1,M)-repmat(r(:)',N,1)).^2; %this replaces the nested for loops from above Thanks Georg Schmitz
D=zeros(size(d));
D(1,1)=d(1,1);
for n=2:N
D(n,1)=d(n,1)+D(n-1,1);
end
for m=2:M
D(1,m)=d(1,m)+D(1,m-1);
end
for n=2:N
for m=2:M
D(n,m)=d(n,m)+min([D(n-1,m),D(n-1,m-1),D(n,m-1)]);
end
end
Dist=D(N,M);
n=N;
m=M;
k=1;
w=[];
w(1,:)=[N,M];
while ((n+m)~=2)
if (n-1)==0
m=m-1;
elseif (m-1)==0
n=n-1;
else
[values,number]=min([D(n-1,m),D(n,m-1),D(n-1,m-1)]);
switch number
case 1
n=n-1;
case 2
m=m-1;
case 3
n=n-1;
m=m-1;
end
end
k=k+1;
w=cat(1,w,[n,m]);
end
w=flipud(w)
%w is a matrix that looks like this:
% 1 1
% 1 2
% 2 2
% 3 3
% 3 4
% 3 5
% 4 5
% 5 6
% 6 6
so what this is saying is that the both the first and second points of the second vector should be mapped to the first point of the first vector. i.e. 1 1
1 2
and that the fifth and sixth points on the first vector should be mapped to the second vector at point six. etc. so w contains the x coordinates of the warped data.
Normally I would be able to say
X1=w(:,1);
X2=w(:,2);
for i=1:numel(reference vector)
Y1(i)=reference vector(X1(i));
Y2(i)=test vector(X2(i));
end
but I need not to stretch the reference vector so I need to use the repeats in X1 to know how to shrink Y2 and the repeats in X2 to know how to stretch Y2 rather than using repeats in X1 to stretch Y1 and repeats in X2 to stretch Y2.
I tried using a find method to find the repeats in both X1 and X2 and then average(shrink) or interpolate linearly(stretch) as needed but the code became very complicated and difficult to debug.
Was this really unclear? I had a hard time explaining this problem, but I just need to know how to take w and create a Y2 that is stretched and shrunk accordingly.
First, here's DTW in Matlab translated from the pseudocode on wikipedia:
t = 0:.1:2*pi;
x0 = sin(t) + rand(size(t)) * .1;
x1 = sin(.9*t) + rand(size(t)) * .1;
figure
plot(t, x0, t, x1);
hold on
DTW = zeros(length(x0), length(x1));
DTW(1,:) = inf;
DTW(:,1) = inf;
DTW(1,1) = 0;
for i0 = 2:length(x0)
for i1 = 2:length(x1)
cost = abs(x0(i0) - x1(i1));
DTW(i0, i1) = cost + min( [DTW(i0-1, i1) DTW(i0, i1-1) DTW(i0-1, i1-1)] );
end
end
Whether you are warping x_0 onto x_1, x_1 onto x_0, or warping them onto each other, you can get your answer out of the matrix DTW. In your case:
[cost, path] = min(DTW, [], 2);
plot(t, x1(path));
legend({'x_0', 'x_1', 'x_1 warped to x_0'});
I don't have an answer but I have been playing with the code of #tokkot implemented from the pseudocode in the Wikipedia article. It works, but I think it lacks three requeriments of DTW:
The first and last points of both sequences must be a match, with the use of min(), some (or many) of the first and ending points of one of the sequences are lost.
The output sequence is not monotonically increasing. I have used x1(sort(path)) instead, but I don't believe it is the real minimum distance.
Additionally, for a reason I haven't found yet, some intermediate points of the warped sequences are lost, which I believe is not compatible with DTW.
I'm still searching for an algorithm like DTW in which one of the sequences is fixed (not warped). I need to compare a time series of equally spaced temperature measurements with another sequence. The first one cannot be time shifted, it does not make sense.
I have a matrix (X) of doubles containing time series. Some of the observations are set to NaN when there is a missing value. I want to calculate the standard deviation per column to get a std dev value for each column. Since I have NaNs mixed in, a simple std(X) will not work and if I try std(X(~isnan(X)) I end up getting the std dev for the entire matrix, instead of one per column.
Is there a way to simply omit the NaNs from std dev calculations along the 1st dim without resorting to looping?
Please note that I only want to ignore individual values as opposed to entire rows or cols in case of NaNs. Obviously I cannot set NaNs to zero or any other value as that would impact calculations.
Have a look at nanstd (stat toolbox).
The idea is to center the data using nanmean, then to replace NaN with zero, and finally to compute the standard deviation.
See nanmean below.
% maximum admissible fraction of missing values
max_miss = 0.6;
[m,n] = size(x);
% replace NaNs with zeros.
inan = find(isnan(x));
x(inan) = zeros(size(inan));
% determine number of available observations on each variable
[i,j] = ind2sub([m,n], inan); % subscripts of missing entries
nans = sparse(i,j,1,m,n); % indicator matrix for missing values
nobs = m - sum(nans);
% set nobs to NaN when there are too few entries to form robust average
minobs = m * (1 - max_miss);
k = find(nobs < minobs);
nobs(k) = NaN;
mx = sum(x) ./ nobs;
See nanstd below.
flag = 1; % default: normalize by nobs-1
% center data
xc = x - repmat(mx, m, 1);
% replace NaNs with zeros in centered data matrix
xc(inan) = zeros(size(inan));
% standard deviation
sx = sqrt(sum(conj(xc).*xc) ./ (nobs-flag));