Well, I am not sure if some of you has encountered the same issue.
I need to convert a matlab program into julia where 'cell2mat' was extensively used.
For example a big matrix A is composed of 3*2 small matrix, see a11, a12, a21, a22, a31, a32, whose dimensions are all 4*2.
Or A = [a11 a12; a21 a22; a31 a32] with a11 = rand(4,2) for example.
I first used an Array{Array{Float64,2},1} type to create the whole matrix A.
Then, I need to convert this A into a usual matrix, say, Array{Float64,2}.
I did try to do like hvcat((NUM),A...), but the order of the converted matrix doesn't correspond to the original Array{Array{Float64,2},1} type.
Thanks for anyone who could provide some pistes.
Wish you all a good day!
Let us start with an array:
A = [reshape((1:8) .+ (10i+100j), 4, 2) for i in 1:3, j in 1:2]
so that we can visually verify that the result is correct.
Now the approach could be:
hcat([vcat(A[:,i]...) for i in 1:size(A,2)]...)
In Julia 0.7 you can use the fact that permutedims is not recursive to get the same in a simpler way:
hvcat(size(A,2), permutedims(A)...)
This will also work under Julia 0.6 but you have to write permutedims(A, (2,1)).
As a side note it is interesting (and was problematic here) that hvcat traverses arguments in rows, but matrices are stored in columns.
EDIT: actually transpose is recursive in 0.7, changed to permutedims which also works under 0.6.
By transposing your original element of the array of array---which is matrix of course----and then transposing the final array of array in the command, such as hvcat(NUM,A...'), the correct answer can be found.
Related
I'm filternig an audio file with matlab inside a function script which has a function named "filtro_pf" created by me which is a band-pass filter and its outputs are the IIR corresponding coefficients:
[b,a] = filtro_pf(Ap,As,fp1,fs1,fp2,fs2,1); //BAND PASS FILTER
b2 = [0.0039,0,-0.0156,0,0.0234,0,-0.0156,0,0.0039];
a2 =[1.0000,-6.2682,17.2660,-27.3421,27.2582,-17.5284,7.0998,-1.6555,0.1701];
wfpf = filter(b,a,audio_stream);
wavplay(wfpf,fs);
Note that the second and third lines (b2 and a2) are the values which the function 'filtro_pb()' gives me for those inputs. I've run once and then copied them to these variables. Now, after I run this script, if I ask for 'a' and 'a2' in the console I will have:
a =
1.0000 -6.2682 17.2660 -27.3421 27.2582 -17.5284 7.0998 -1.6555 0.1701
a2 =
1.0000 -6.2682 17.2660 -27.3421 27.2582 -17.5284 7.0998 -1.6555 0.1701
They are pretty much the same. But if I use 'a2' instead of 'a' in the filter() function, it does not work. I hear a kind of tick sound and that's it. With 'a' I can hear the sound correctly filtered. This same code is used before and it does work:
%[b,a] = filtro_pa(Ap,As,fp,fs,1); //HIGH PASS FILTER
b = [0.5411 -1.6232 1.6232 -0.5411];
a = [1.0000 -1.8062 1.2298 -0.2925];
wfpa = filter(b2, a2, audio_stream);
wavplay(wfpa,fs);
Again, I used this script previously and saw that these values (from a2 and b2) were the output to these inputs. Now instead of calling the function again (which is commented by the way) I use the 'a' and 'b'vectors directly. It does work for the LowPass and for the HighPass filter.
All of this are for test purposes so I do not expect suggestions like "why use the vector instead of calling the function then?".
I just want to know, how can the function not work with the second variable if they are pretty much the same?
There is more precision present in the variables than is displayed, which means that your a2 and b2 vectors are not the same as a and b. It might appear surprising that errors on that order would make the filter unstable, but it appears that is what is happening. You should be able to explore this by looking at the filter response with freqz, and by plotting up the resulting audio vector rather than just listening to it.
You can use format long to print more precision, but these will still have some rounding error. To avoid, save the vectors to a .mat file and reload that. The .mat file will use binary format and store the full precision of your vectors.
The reason it works for the other filters is probably because those filters are less sensitive to rounding errors in their coefficients: they have fewer coefficients, and those coefficients are less extreme in value.
Here's a sample comparison of frequency response:
[H W] = freqz(b2, a2); % your filter (with error)
a_error = zeros(size(a2));
a_error(9) = a_error(9)+.001; % a little bit of error in a single coefficient
[HE WE] = freqz(b2, a2 + a_error); % frequency response of THAT filter
plot(log10(abs([H HE])))
As you can see, a small change makes a large difference.
An actual analysis of the instability comes from looking at the poles and zeros of the filter:
[z p k] = tf2zp(b2, a2);
abs(p)
If any poles have magnitude greater than 1 (this one does), the filter will be unstable. Try the real values, then your "approximate" values, and see what happens.
I have a vector, A.
A=[3,4,5,2,2,4;2,3,4,5,3,4;2,4,3,2,3,1;1,2,3,2,3,4]
Some of the records in A must be replaced by NaN values, as they are inaccurate.
I have created vector rowid, which records the last value that must be kept after which the existing values must be swapped to NaN.
rowid=[4,5,4,3]
So the vector I wish to create, B, would look as follows:
B=[3,4,5,2,NaN,NaN;2,3,4,5,3,NaN;2,4,3,2,NaN,NaN;1,2,3,NaN,NaN,NaN]
I am at a loss as to how to do this. I have tried to use
A(:,rowid:end)
to start selecting out the data from vector A. I am expecting to be able to use sub2ind or some sort of idx to do this, possibly an if loop, but I don't know where to start and cannot find an appropriate similar question to base my thoughts on!
I would very much appreciate any tips/pointers, many thanks
If you are not yet an expert of matlab, I would stick to simple for-loops for now:
B = A;
for i=1:length(rowid)
B(i, rowid(i)+1:end) = NaN;
end
It is always a sport to write this as a one-liner (see Mohsen's answer), but in many cases an explicit for-loop is much clearer.
A compact one is:
B = A;
B(bsxfun(#lt, rowid.', 1:size(A,2)))=NaN;
my script creates a matrix and 2 vectors using several 'for' loops and as an example they are returned as follows:
K =
1.0e+006 *
1.2409 0.6250 0.8153 0.1250
0.6250 3.6591 -0.1250 3.5375
0.8153 -0.1250 1.2409 -0.6250
0.1250 3.5375 -0.6250 3.6591
F =
1.0e+006 *
0.1733
1.3533
-0.1066
1.3371
U =
u3
v3
u4
v4
As can be seen, the 'U' vector is a set of variables and I need to solve 'K*U=F' for variables contained in 'U'.
When I try to do that using linsolve or solve I get unexpected results and a message that the inverse of my matrix is close to singular.
HOWEVER, when I make another script and put in the SAME matrix and vectors BY HANDS it all works fine and I can't figure out what's wrong.
Is that somehow related to the way MATLAB stores matrices created by loop functions and I need to change the state of the matrix to something after the loop?
Also, when I put the matrix by hands it displays it without the 1.0e+006 multiplier in front of it:
K11 =
1240900 625000 815300 125000
625000 3659100 -125000 3537500
815300 -125000 1240900 -625000
125000 3537500 -625000 3659100
can that be related??
Thanks in advance.
Try the backslash operator:
U = K\F
See this reference.
From the previous discussion it's clear that your matrix is singular. This means that your equations are not linearly independent. When this happens there are two possibilities. Your system may be inconsistent (over-constrained), in which case no solutions exist. Or alternatively, it can also mean that your equations are under-constrained, in which case there is an infinite set of solutions.
To determine which case it is you can use rref to get the "row reduce echelon form" of the matrix. Do this as follows:
KF = [K,F]
rref(KF)
If the last row goes entirely to zeros then you're under-constrained and can extract a solution set (but not a unique solution) from your reduced matrix.
In this case however I get a row of [0 0 0 0 1], which makes the system over-constrained and hence without any solution.
Let me ask whether using Matlab for my particular problem is nonsense or some people do the similar.
I have an initial sequence S(1), where each term is a 2D point.
I create a new sequence S(2) by inserting a new term point p
between each consecutive 2 term points p(i) and p(i+1).
Where p is a function f of 4 term points of nearest indices on S(2).
Namely,
p= f( p(i-1),p(i),p(i+1),p(i+2) )
And the function f is written in a C like style
but not in the pure style of matrix language.
In the same way , I repeat generating the new longer sequence S(i+1) up to S(m).
The above may be vague for you, but please give some advice.
I do not ask whether Matlab is the best choice for the problem , but whether no expert will use Matlab for such a problem or some will.
Thank you in advance.
It heavily depends on f. If f could be coded efficiently in Matlab or you are willing to spend the time to MEX it (Matlab C extension), then Matlab will perform efficiently.
The code could be vectorized like this:
f = #(x) mean(x,3);
m=3;
S{1}=[1,2,3;4,5,6];
for i=2:m
S{i} = cat(3,...
[[0;0] S{i-1}(:,1:end-2)],...
S{i-1}(:,1:end-1),...
S{i-1}(:,2:end),...
[S{i-1}(:,3:end) [0;0]]);
S{i} = [f(S{i}) [0;0]];
S{i} = cat(3,S{i-1},S{i});
S{i} = permute(S{i},[1 3 2]);
S{i} = S{i}(:,:);
S{i}(:,end)=[];
end
Yes, Matlab seems to be suitable for such a task. For the data structure of your list of sequences, consider using cell arrays. You could have S as a cell array, and S{1} would correspond to your S(1), and could again be a cell array of points, or a usual matrix if points are just pairs or triples of numbers.
As an alternative, Python in my opinion is particulary strong when it comes to all kind of sequences.
I have cell array each containing a sequence of values as a row vector. The sequences contain some missing values represented by NaN.
I would like to replace all NaNs using some sort of interpolation method, how can I can do this in MATLAB? I am also open to other suggestions on how to deal with these missing values.
Consider this sample data to illustrate the problem:
seq = {randn(1,10); randn(1,7); randn(1,8)};
for i=1:numel(seq)
%# simulate some missing values
ind = rand( size(seq{i}) ) < 0.2;
seq{i}(ind) = nan;
end
The resulting sequences:
seq{1}
ans =
-0.50782 -0.32058 NaN -3.0292 -0.45701 1.2424 NaN 0.93373 NaN -0.029006
seq{2}
ans =
0.18245 -1.5651 -0.084539 1.6039 0.098348 0.041374 -0.73417
seq{3}
ans =
NaN NaN 0.42639 -0.37281 -0.23645 2.0237 -2.2584 2.2294
Edit:
Based on the responses, I think there's been a confusion: obviously I'm not working with random data, the code shown above is simply an example of how the data is structured.
The actual data is some form of processed signals. The problem is that during the analysis, my solution would fail if the sequences contain missing values, hence the need for filtering/interpolation (I already considered using the mean of each sequence to fill the blanks, but I am hoping for something more powerful)
Well, if you're working with time-series data then you can use Matlab's built in interpolation function.
Something like this should work for your situation, but you'll need to tailor it a little ... ie. if you don't have equal spaced sampling you'll need to modify the times line.
nseq = cell(size(seq))
for i = 1:numel(seq)
times = 1:length(seq{i});
mask = ~isnan(seq{i});
nseq{i} = seq{i};
nseq{i}(~mask) = interp1(times(mask), seq{i}(mask), times(~mask));
end
You'll need to play around with the options of interp1 to figure out which ones work best for your situation.
I would use inpaint_nans, a tool designed to replace nan elements in 1-d or 2-d matrices by interpolation.
seq{1} = [-0.50782 -0.32058 NaN -3.0292 -0.45701 1.2424 NaN 0.93373 NaN -0.029006];
seq{2} = [0.18245 -1.5651 -0.084539 1.6039 0.098348 0.041374 -0.73417];
seq{3} = [NaN NaN 0.42639 -0.37281 -0.23645 2.0237];
for i = 1:3
seq{i} = inpaint_nans(seq{i});
end
seq{:}
ans =
-0.50782 -0.32058 -2.0724 -3.0292 -0.45701 1.2424 1.4528 0.93373 0.44482 -0.029006
ans =
0.18245 -1.5651 -0.084539 1.6039 0.098348 0.041374 -0.73417
ans =
2.0248 1.2256 0.42639 -0.37281 -0.23645 2.0237
If you have access to the System Identification Toolbox, you can use the MISDATA function to estimate missing values. According to the documentation:
This command linearly interpolates
missing values to estimate the first
model. Then, it uses this model to
estimate the missing data as
parameters by minimizing the output
prediction errors obtained from the
reconstructed data.
Basically the algorithm alternates between estimating missing data and estimating models, in a way similar to the Expectation Maximization (EM) algorithm.
The model estimated can be any of the linear models idmodel (AR/ARX/..), or if non given, uses a default-order state-space model.
Here's how to apply it to your data:
for i=1:numel(seq)
dat = misdata( iddata(seq{i}(:)) );
seq{i} = dat.OutputData;
end
Use griddedInterpolant
There also some other functions like interp1. For curved plots spline is the the best method to find missing data.
As JudoWill says, you need to assume some sort of relationship between your data.
One trivial option would be to compute the mean of your total series, and use those for missing data. Another trivial option would be to take the mean of the n previous and n next values.
But be very careful with this: if you're missing data, you're generally better to deal with those missing data, than to make up some fake data that could screw up your analysis.
Consider the following example
X=some Nx1 array
Y=F(X) with some NaNs in it
then use
X1=X(find(~isnan(Y)));
Y1=Y(find(~isnan(Y)));
Now interpolate over X1 and Y1 to compute all values at all X.