I have a 34200 x 4 table. This table shows the 30 years monthly amount of pr (precipitation) in some latitudes and longitudes. So the columns are lat, lon, date, and pr. I want to convert it to a 3D matrix that longitude x latitude x month.
I attach my table.
Please tell me how to do it I'm a beginner. If I don't want to convert it based on the month I could but this issue is so complicated for me.
Please look at my table it's only 235 KB I upload it to my DropBox so please click on Open in the top right side and click download.
Here is my image
Inspecting your data, your latitude and logitude values actually represent 95 unique locations which are scattered seemingly randomly. You can see that in the figure below.
length(unique(C.lat)) % 95
length(unique(C.lon)) % 95
scatter(C.lat, C.lon)
If the locations were spaced in a grid, it would make sense to use lat and lon as the axes of a data matrix. But instead, it is better to use only one axis representing the unique locations. This then leaves you with a second axis representing the date.
length(unique(C.date)) % 360
360 * 95 % 34200 - the number of values we have
Reformatting the data
Therefore, I would store the data in a 2D matrix as follows.
locations_lat = unique(C.lat, 'stable');
locations_lon = unique(C.lon, 'stable');
dates = unique(C.date, 'stable');
data = reshape(C.pr, length(dates), length(locations_lat)); % size 360 x 95
Then, to check that this has worked, choose a random example.
location_num = 27;
date_num = 242;
lat = locations_lat(location_num) % 14.68055556
lon = locations_lon(location_num) % 65.23111111
date = dates(date_num) % 2/1/2009
precipitation = data(date_num, location_num) % 16.7179
Searching for that position and date in the original tale, we have:
9602| 14.6805555600000 65.2311111100000 '2/1/2009' 16.7179000000000
If A is your data with the 4 columns [34200x4] then you can create a 3-dimensional matrix like this:
B = zeros(len(A),3)
B(:, 1) = A(:, 1)
B(:, 2) = A(:, 2)
B(:, 3) = A(:, 3)
Possibly even:
B(:,1:3) = A(:, 1:3)
Depending how your data is set, you may need to transpose which is:
B = 'B
You can map data over as long as the dimensions match.
You can implement a for loop if you have even more columns or for dynamic data entries.
Related
I need to make a 3D surface where colour will represent the fourth variable. I know "surf" is SIMILAR to what I need, but that did not work fine. Basically my data obtained from photododie sensor scanning along x,y,z and the forth variable is the intensity( data). The data are stored in text files in the following order.
Total files=2380
in each file there are 4 columns and 480 rows. the columns represent x,y,z,data
I was trying to plot the 3D surface but got this error " Error using
matlab.graphics.chart.primitive.Surface/set
Value must be a vector or 2D array of numeric
type". Please help
Here is the code
close all;
close all;
Axstep=2.5; % scanning with 2.5 micron step along x and y directions
Axlines=2380; %number of text files( in each file there are 4 column of equal
sized data (x,y,z,data) and 480 rows)
Path = 'C:\Users\asay0001\Google Drive\matlab\exp2_27-11-17_CONF09\all data - Copy';
Data = zeros(480,Axlines); % 480 rows in one file out of 2380
for i = 1:Axlines
DataTemp = importdata([Path '\DATA (' num2str(i) ').txt']);
Data(:,i) = DataTemp(:,4); % read column 4 which represents data to be ploted
end
[x y,z] = meshgrid([-3200:2.5:-2602.5],[325:2.5:672.5],[-800:100:800]);
surf(x,y,z,Data')
I'm loading a .csv file to do a few calculations in matlab. The file itself has ~1600 lines, but I'm interested in only a subset.
load file.csv; %load file
for i = 400:1200 %rows I am interested in
rh_x= file(i,60); % columns interested, in column 60 for the x, 61 for y
rh_y= file(i,61);
rh_x2 = file(i+1, 60); % next point (x,y)
rh_y2 = file(i+1, 61);
p1 = [rh_x, rh_y];
p2 = [rh_x2, rh_y2];
coord = [p1, p2];
Distan = pdist(coord, 'euclidean'); ****
disp(Distan);
end
Nothing is being stored in my Distan variable (distance formula), where I tried to input two points. Why is that the case? I'm just wanting to calculate the distance formula for all the pairs of points in rows 60 and 61 for frames 400-1200.
Thank you.
Change your coord assignment to the following:
coord = [p1; p2];
The way you have it, it is storing all of the x, y pairs on the same row, as a 1x4 matrix. The above method stores it as a 2x2 matrix and pdist gives an answer.
Consider the 37x101 matrix below:
Each black cell has value 1, the rest of the cells value 0. I would like to fill in the "gaps" by means of cubic spline interpolation, as well as scaling the y axis from 37 to 181. The latter can be done by using the interp1 function, as in:
interp1(37,matrix,181,'pchip')
However, the outcome is now interpolated along the y-axis, but the gaps remain:
I don't want to interpolate along the x-axis, because I want the final matrix to have dimension 181 x 101. I just want to fill in the gaps using the existing cells (of the 181 x 101 matrix).
How can the original matrix (top), be scaled from 37 x 101 to 181 x 101 (without the "smoothing" in the second image), as well as filling in the gaps using some kind of spline interpolation as if this was a proper function?
It appears that your truth value grid has a single one where the true value is in each row. If the true/1 values do in fact create a line through the image, I would recommend parametrize the line with respect to t so that y = fy(t) and x = fx(t). If you're not familiar with this you can find some parametrization info on youtube tutorials or google. The main idea is that if you have , say a truth table that looks like this:
Then you could plot the the location of each pixel with respect to another variable, t and then use interp1(...) on each of these individually. In my case I defined the x and y values as follows:
n = 32;
rand('seed', 1982);
y_orig = 1:n;
x_orig = ceil(n*sin(y_orig/n*pi));
So I can plot as:
t1 = linspace(0,1, n);
plot(t1,x_orig, 'r', 'linewidth', 3);
hold all
plot(t1,y_orig, 'b', 'linewidth', 3);
legend('X', 'Y')
Note that I can get any truth value I want just by using interp1 like this (if you wanted to find the value half way between the 5th and 6th row):
desiredY = 5.5;
t= 1:n;
truthValue= interp1(t, x_orig, desiredY, 'cubic')
But we are looking to make a new image so I chose a more convenient parametrization of t between zero and one. Unfortunately, you may not have x and y off hand, so we need to pull them out of the image. Assuming you have a single true/1 value in each row we can yank out the values with max(...):
[maxVals, x1] = max(data,[],2);
x1(maxVals == 0) = [];
y1 = find(maxVals ~= 0);
Some form of find on each row would also work. If you have a truth value in each row then y1 should equal 1:n. The max function returns the index of the max in dimension 2 in the second return value. I use the next two lines to remove any entries where there truth table was empty (max is zero) and then y1 = 1:n minus those entries that were empty.
A quick and dirty way to get lots of points along this line is:
t2 = linspace(0,1,1024);
x2 = interp1(t1, x1, t2, 'cubic');
y2 = interp1(t1, y1, t2, 'cubic');
I can then plot the original points/image and this newly discovered finer line together like this:
imagesc(data);
hold all;
plot(x2,y2, 'linewidth', 2);
axis image
colormap(flipud(colormap(gray)));
To get this:
Finally, you can quickly turn this into a new image by scaling the parametrization up. My method is not particularly efficient for clarity:
y2_scaled = floor((y2(:)-1)*scaleValue + 1);
x2_scaled = floor((x2(:)-1)*scaleValue + 1);
scaleValue = 2;
data2 = zeros(n*scaleValue);
for ind = 1:length(x2_scaled)
data2(y2_scaled(ind),x2_scaled(ind)) = 1;
end
Which results in:
Note that this table has connected all the points and you now have multiple true/1's in each row. This is because I chose a very small step size for t2. You could fix this by either choosing t2 smarter, skipping multiple values in each row, or average the location of each indices in each row. Or ignoring this issue.
To fix t2 with the scaling value, you could use
t2 = linspace(0,1,n*scaleValue);
to get only one true/1 per row in the above code.
Also, if you want to only scale one dimension, you could do it like this:
y2_scaled = floor((y2(:)-1)*scaleValue + 1);
x2_scaled = floor((x2(:)-1) + 1);
scaleValue = 2;
data2 = zeros(n*scaleValue,n);
for ind = 1:length(x2_scaled)
data2(y2_scaled(ind),x2_scaled(ind)) = 1;
end
I see this as a bitmap, so why not a clamped blur?
I=yourmatrixhere
%gaussian blur
% it looks like bump-to-bump distance is 3 empties
% therefore hsize should be about 7
% it looks like bump vertical size is about 4
% therefore simga should be about 10
hsize=[3 3];
sigma = 10;
h=fspecial('gaussian',hsize,sigma)
I2=imfilter(I,h,'replicate');
At this point you have spread information to adjacent columns, but you need to "tidy up" from continuous to binary.
%threshold
th = 0.25;
I3=zeros(size(I));
ind=find(I>=th);
I3(ind)=1;
At this point, I3 is your matrix of interest to do the "erode" or interpolation.
I have a time series of prices, 2000 entries.
I have created 12 vectors where each one contains only the data for one month. They don't have the same length, vary about 20 values between 160 and 180 values.
So now I need to plot all these vectors in the same plot, in sequence of course, starting with January data, and a little space in between, and on the x-axis put the month names (which I have in an array ['jan' 'feb' etc]
For an example click on the link and scroll down to seasonal subseries plot
http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc443.htm
To obtain a similar graph to the above, you can insert a row of NaNs after each month. Since, each month has a different number of rows, you cannot simply reshape, concatenate NaNs, and reshape back.
Suppose you have a timestamps in the first column and some data in the second column:
data = [(now-11:now+13)' rand(25,1)];
% Count in 'idx' when each year-month pair ends
[y,m] = datevec(data(:,1));
[~, idx] = unique([y,m],'rows','last');
% Preallocate expanded Out with NaN separations between each month
szData = size(data);
Out = NaN(szData(1) + numel(idx)-1,2);
% Reposition 'data' within 'Out'
pos = ones(szData(1),1);
pos(idx(1:end-1)+1) = 2;
Out(cumsum(pos),:) = data;
% Example plot
plot(Out(:,1),Out(:,2))
set(gca,'Xtick',data([1 11 12 25],1))
datetick('x','dd-mmm','keepticks')
My question is related to this link stackoverflow ques
In essence repeating the figure drawn there .. I have a central point ( x , y ) in an image around which I have to draw 4 circles of 1-4 unit radius with 8 angles between them.
In this diagram there are 12 angle bins but I have 8. There is a code solution there but it is for plotting the above figure.
I want to calculate the maximum intensity point in each of the 4 regions of each wedge. Is there any inbuilt function in matlab ? I looked at rose but could'nt understand if it would help me....
I would greatly appreciate if someone could help me how to calculate it in matlab....
Thanks
I put some code below that should be the basic skeleton of what you want to do. But I left an important function unimplemented because I think you will be able to do it and it will help you understand this process better.
% I assume that data_points is an M-by-2 array, where each row corresponds
% to an (x,y) coordinate pair, and M is the number of data points.
data_points = ... ;
% I assume this array stores the intensities at each data point.
intensities = ... ;
% I assume that this stores the total number of gridded polar regions you want
% to find the max intensity in (i.e. 4*(number of cells) in your picture above).
total_num_bins = ... ;
% This will store the max intensities. For places that have no nearby
% data points, the max intensity will remain zero.
max_intensities = zeros(total_num_bins);
% I assume these store the values of the center point.
x = ... ; y = ... ;
% The number of different data points.
num_data_points = length(intensities); % also equals size(data_points,1)
% Now, loop through the data points, decide which polar bin they fall in, and
% update the max intensity of that area if needed.
for ii = 1:num_data_points
% Grab the current point coordinates.
cur_x = data_points[ii,1];
cur_y = data_points[ii,2];
% Convert the current data point to polar coordinates,
% keeping in mind that we are treating (x,y) like the center.
cur_radius = sqrt( (cur_x - x)^2 + (cur_y - y)^2 );
cur_angle = atan2(cur_y - y, cur_x - x)
% You have to write this yourself, but it
% will return an index for the bin that this
% data point falls into, i.e. which of the 4 segments
% of one of the radial cells it falls into.
cur_bin = get_bin_number(cur_radius, cur_angle);
% Check if this data point intensity is larger than
% the current max value for its bin.
if ( intensities(ii) >= max_intensities(cur_bin))
max_intensities(cur_bin) = intensities(ii);
end
end
You will now have to make the function get_bin_number() which takes as its input the angle and radius of the data point away from the center point. It should return just an index between 1 and total_num_bins, because you will be keeping the max intensities in a linear array. So, for example, index number 1 might correspond to the first 1/4 piece of the closest radial cell in the upper right quadrant, index 2 might correspond to the next 1/4 of that same cell, moving counter-clockwise, or something like this. You have to devise your own convention for keeping track of the bins.
A late answer, but I believe an even easier solution would just be to convert your data from (x,y) coordinates to (r,theta) by using (r = sqrt(x.^2 + y.^2), theta = atan(y,x)) then use the hist3 function on the (r,theta) data set to get a radial histogram.
Therefore solution is as follows:
% I assume you have some M-by-2 matrix X that's in the form (x,y)
% Convert (x,y) to (r,theta)
xVect = X(:,1);
yVect = X(:,2);
X = [sqrt(xVect.^2 + yVect.^2), ...%formula for r
atan(yVect,xVect)]; %formula for theta
% 5 is the number of wedges along 'r', your radial axis
% 12 is the number of wedges along 'theta', your theta 'axis'
dist = hist3(X,5,12);
Even if you have solved this, I hope this helps anybody else who wants to create a radial/angular histogram!