This might be simple and I apologize if it is so.
In matlab I have a double precision matrix which can theoretically have the range of +/- infinity.
I would to use the histogram function in matlab to change the values of the matrix.
For instance, if data elements fall within histogram bin 1 then I would like to assign the value of 1 to this and all of its instances.
Is there a quick and cheap way of doing this?
I have tried lookuptables etc but matlabs LUT is a pain.
Thank you for looking at my question
I think I just cracked it ...
Make a new function out of hist and after edges in the m file add this line:
[~,my_labels] = histc(y,edges,1);
and my_labels will contain your matrix with the histogram values instead of the actual values.
Related
I have got a matrix of AirFuelRatio values at certain engine speeds and throttlepositions. (eg. the AFR is 14 at 2500rpm and 60% throttle)
The matrix is now 25x10, and the engine speed ranges from 1200-6000rpm with interval 200rpm, the throttle range from 0.1-1 with interval 0.1.
Say i have measured new values, eg. an AFR of 13.5 at 2138rpm and 74,3% throttle, how do i merge that in the matrix? The matrix closest values are 2000 or 2200rpm and 70 or 80% throttle. Also i don't want new data to replace the older data. How can i make the matrix take this value in and adjust its values to take the new value in account?
Simplified i have the following x-axis values(top row) and 1x4 matrix(below):
2 4 6 8
14 16 18 20
I just measured an AFR value of 15.5 at 3 rpm. If you interpolate the AFR matrix you would've gotten a 15, so this value is out of the ordinary.
I want the matrix to take this data and adjust the other variables to it, ie. average everything so that the more data i put in the more reliable and accurate the matrix becomes. So in the simplified case the matrix would become something like:
2 4 6 8
14.3 16.3 18.2 20.1
So it averages between old and new data. I've read the documentation about concatenation but i believe my problem can't be solved with that function.
EDIT: To clarify my question, the following visual clarification.
The 'matrix' keeps the same size of 5 points whil a new data point is added. It takes the new data in account and adjusts the matrix accordingly. This is what i'm trying to achieve. The more scatterd data i get, the more accurate the matrix becomes. (and yes the green dot in this case would be an outlier, but it explains my case)
Cheers
This is not a matter of simple merge/average. I don't think there's a quick method to do this unless you have simplifying assumptions. What you want is a statistical inference of the underlying trend. I suggest using Gaussian process regression to solve this problem. There's a great MATLAB toolbox by Rasmussen and Williams called GPML. http://www.gaussianprocess.org/gpml/
This sounds more like a data fitting task to me. What you are suggesting is that you have a set of measurements for which you wish to get the best linear fit. Instead of producing a table of data, what you need is a table of values, and then find the best fit to those values. So, for example, I could create a matrix, A, which has all of the recorded values. Let's start with:
A=[2,14;3,15.5;4,16;6,18;8,20];
I now need a matrix of points for the inputs to my fitting curve (which, in this instance, lets assume it is linear, so is the set of values 1 and x)
B=[ones(size(A,1),1), A(:,1)];
We can find the linear fit parameters (where it cuts the y-axis and the gradient) using:
B\A(:,2)
Or, if you want the points that the line goes through for the values of x:
B*(B\A(:,2))
This results in the points:
2,14.1897 3,15.1552 4,16.1207 6,18.0517 8,19.9828
which represents the best fit line through these points.
You can manually extend this to polynomial fitting if you want, or you can use the Matlab function polyfit. To manually extend the process you should use a revised B matrix. You can also produce only a specified set of points in the last line. The complete code would then be:
% Original measurements - could be read in from a file,
% but for this example we will set it to a matrix
% Note that not all tabulated values need to be present
A=[2,14; 3,15.5; 4,16; 5,17; 8,20];
% Now create the polynomial values of x corresponding to
% the data points. Choosing a second order polynomial...
B=[ones(size(A,1),1), A(:,1), A(:,1).^2];
% Find the polynomial coefficients for the best fit curve
coeffs=B\A(:,2);
% Now generate a table of values at specific points
% First define the x-values
tabinds = 2:2:8;
% Then generate the polynomial values of x
tabpolys=[ones(length(tabinds),1), tabinds', (tabinds').^2];
% Finally, multiply by the coefficients found
curve_table = [tabinds', tabpolys*coeffs];
% and display the results
disp(curve_table);
I have a matrix which is 1*1*10000, the slightly odd dimensions are the result of the matrix algebra used to calculate it.
I simply want to be able to plot the 10000 data points contained in it, but matlab seems unable to do it?
Can someone please tell me how I can plot the data?
Seems simple but I really can't figure out how to do it!
Baz
yes you need to reduce the dimensions to a vector:
A = zeros(1,1,100)
vector = squeeze(A(1,1,:))
as when you'd access the third dimension this would only return a 3D-Matrix again:
z = A(1,1,:)
would NOT work. So use squeeze() ;-) Then plot as usual.
Doc-Link: http://www.mathworks.de/de/help/matlab/ref/squeeze.html
And as Ander pointed out in comments, no need to give any dimensions, as it removes singleton-dimensions by itself. So just use vector = squeeze(A). MATLAB recognizes the way to go itself.
I have a huge sparse matrix (1,000 x 1,000,000) that I cannot load on matlab (not enough RAM).
I want to visualize this matrix to have an idea of its sparsity and of the differences of the values.
Because of the memory constraints, I want to proceed as follows:
1- Divide the matrix into 4 matrices
2- Load each matrix on matlab and visualize it so that the colors give an idea of the values (and of the zeros particularly)
3- "Stick" the 4 images I will get in order to have a global idea for the original matrix
(i) Is it possible to load "part of a matrix" in matlab?
(ii) For the visualization tool, I read about spy (and daspect). However, this function only enables to visualize the non-zero values indifferently of their scales. Is there a way to add a color code?
(iii) How can I "stick" plots in order to make one?
If your matrix is sparse, then it seems that the currently method of storing it (as a full matrix in a text file) is very inefficient, and certainly makes loading it into MATLAB very hard. However, I suspect that as long as it is sparse enough, it can still be leaded into MATLAB as a sparse matrix.
The traditional way of doing this would be to load it all in at once, then convert to sparse representation. In your case, however, it would make sense to read in the text file, one line at a time, and convert to a MATLAB sparse matrix on-the-fly.
You can find out if this is possible by estimating the sparsity of your matrix, and using this to see if the whole thing could be loaded into MATLAB's memory as a sparse matrix.
Try something like: (untested code!)
% initialise sparse matrix
sparse_matrix = sparse(num_rows, num_cols);
row_num = 1;
fid = fopen(filename);
% read each line of text file in turn
while ~feof(fid)
this_line = fscanf(fid, '%f');
% add row to sparse matrix (note transpose, which I think is required)
sparse_matrix(row_num, :) = this_line';
row_num = row_num + 1;
end
fclose(fid)
% visualise using spy
spy(sparse_matrix)
Visualisation
With regards to visualisation: visualising a sparse matrix like this via a tool like imagesc is possible, but I believe it may internally create the full matrix – maybe someone can confirm if this is true or not. If it does, then it's going to cause you memory problems.
All spy is really doing is plotting in 2D the locations of the non-zero elements. You can fairly easily write your own spy function, which can have different coloured or sized points depending on the values at each location. See this answer for some examples.
Saving sparse matrices
As I say above, the method your matrix is saved as is pretty inefficient – for a matrix with 10% sparsity, around 95% of your text file will be a zero or a space. I don't know where this data has come from, but if you have any control over its creation (e.g. it comes from another program you have written) it would make much more sense to save only the non-zero elements in the format row_idx, col_idx, value.
You can then use spconvert to import the sparse matrix directly.
One of the simplest methods (if you can actually store the full sparse matrix in RAM) is to use gnuplot to visualize the sparisty pattern.
I was able to spy matrices of size 10-20GB using gnuplot without problems. But make sure you use png or jpeg formats to output the image. Note that you don't need the value of the non-zero entry only the integers (row, col). And plot them "plot "row_col.dat" using 1:2 with points".
This chooses your row as x axis and cols as your y axis and start plotting the non-zero entries. It is very easy to do this. This is the most scalable solution I know. Gnuplot works at decent speed even for very large datasets (>10GB of [row, cols]), but Matlab just hangs (with due respect)
I use imagesc() to visualise arrays. It scales the values in array to values between 0 and 1, then plots the array like a greyscale bitmap image (of course you can change the colormap to make it easier to see detail).
I have simulation data in a vector of size 50,000 x 1, which has NaNs and non-NaNs. I would like to average the non-NaNs, but the function nanmean returns NAN. I have tried removing the NANs, but I only get a vector of zeros. Visual inspection of the vector leads me to doubt that the true mean of this vector is really NaN.
Also, I would like to use this vector to compute covariance with several other vectors (at some point). My alternative is doing this in Excel, which would be painful.
Any thoughts?
Thank you
Let's say your data in stored in a vector A, you can take the mean of the vector excluding the NaNs as well as any Inf and -Inf values via:
meanA = mean( A(isfinite(A)) );
Assuming you have a vector that only contains finite numeric values, and a NaN here and there, the solution is very simple
nanmean(A)
This should only bring trouble if there are non finite values in your vector.
In this case you could filter them out as suggested by #Ryan, but then you need to realize that you are not actually calculating the mean of the vector.
Ask yourself whether you may instead be interested in something like
nanmedian(A)
About the calculation of covariances and the likes, assuming you have vectors v and w, then I would recommend you to do something like this:
idx = isfinite(v) & isfinite(w);
cov(v(idx),w(idx))
Inside a MATLAB function I have built a matrix A, whose dimensions M and N are set as parameters of the function. I would like to plot all the columns of this matrix, given a vector of indices B with length M. Hence, I use these lines:
figure
plot(B,A)
I specified figure as the MATLAB function returns more different plots.
My problem is that the program plots just two columns of the matrix with different colours (blue and violet). Where is my mistake?
Thank you for your attention.
go for
plot(repmat(B,1,N),A);
or
plot(repmat(B,N,1),A);
(depending on your rows/columns). You need to have same size matrices in plot.
Moreover, if B are just consecutive indexes, you may want to consider Plot(A) (or Plot(A')).
I noticed that there was an error which caused the overlap of the different curves, so the way which I used to plot the colums of a matrix is valid. However, the method proposed by Acorbe is a possibility, too.