I have a large XYZ file (300276x3, this file includes x and y coordinates (not lat/lon, but polar stereographic) and elevation z) and I'm wondering if it would be possible to convert this into a gridded dataset (n x m matrix). The xyz file can be downloaded from:
https://wetransfer.com/downloads/4ae4ce51072dceef93486314d161509920191021213532/48e4ee68c17269bd6f7a72c1384b3c9a20191021213532/60b04d
and imported in matlab by:
AIS_SEC = importdata('AIS_SEC.xyz');
I tried:
X= XYZ(:,1);
Y= XYZ(:,2);
Z= XYZ(:,3);
xr = sort(unique(X));
yr = sort(unique(Y));
gRho = zeros(length(yr),length(xr));
gRho = griddata(X,Y,Z,xr,yr')
imagesc(gRho)
Requested 300276x300276 (671.8GB) array exceeds maximum array size preference. Creation of arrays
greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size
limit or preference panel for more information.
I tried:
% Get coordinate vectors
x = unique(XYZ(:,1)) ;
y = unique(XYZ(:,2)) ;
% dimensions of the data
nx = length(x) ;
ny = length(y) ;
% Frame matrix of grid
D = reshape(XYZ(:,3),[ny,nx]) ;
% flip matrix to adjust for plot
H = flipud(H) ;
% Transpose the matrix
H = H' ; % Check if is required
surf(x,y,H) ;
Error using reshape
To RESHAPE the number of elements must not change.
I can now plot the nx3 file with scatter3 (see image)
scatter3(XYZ(:,1),XYZ(:,2),XYZ(:,3),2,XYZ(:,3)) ;
colorbar
But I'd like to do it with imagesc. Hence, I would like to convert the nx3 file into a nxm matrix (in raster/gridded format) and as en extra I would like it as a geotiff file for use in QGIS.
Thanks!
You were almost there... Looking at the message about array size you got, it seems likely that the result of unique(X) results in 300276 unique values, probably due to some noisy data.
So instead of using griddata with these large X and Y vectors, you can define some new ones on the domain you need:
% make some sample data
N = 1000;
xv = linspace(-10,10,N);
yv = linspace(-10,10,N);
[XV,YV] = meshgrid(xv,yv);
ZV = XV.^2 + YV.^2;
% make into long vectors:
X = XV(:);
Y = YV(:);
Z = ZV(:);
% make x and y vector to interpolate z
N = 50; % size of new grid
xv = linspace(min(X), max(X), N);
yv = linspace(min(Y), max(Y), N);
[XV,YV] = meshgrid(xv,yv);
% use griddata to find right Z for each x,y pair
ZV_grid = griddata(X,Y,Z,XV,YV);
% look at result
figure();
subplot(211)
imagesc(ZV);
subplot(212);
imagesc(ZV_grid)
Related
I have a cubic box with the size of, lets say 300 in each direction. I have some data (contains X,Y,Z coordinates) which represent the distribution of nearly 1 million data-points within this box. I want to specify a color to their Number density (its an intensive quantity used to describe the degree of concentration of countable objects in this case data-points). In another word, Using color to illustrate which part is more condensed in terms of data-points rather than the other parts. The index for the color-bar in the final image should represent the percentage of data-points specified with that color.
I have tried to do it by dividing the whole space in cubic box to 1 million smaller cube (each cube has a length of 3 in all direction). By counting the number of particles within those cube, I will know how they distributed in the box and the number of existed data-points within. Then I can specify a color to them which I wasn’t successful in counting and specifying. Any suggestion is appreciated.
%reading the files
[FileName,PathName,FilterIndex] = uigetfile('H:\*.txt','MultiSelect','on');
numfiles = size(FileName,2);%('C:\final1.txt');
j=1;
X=linspace(0,300,100);
for ii = 1:numfiles
FileName{ii}
entirefile = fullfile(PathName,FileName{ii});
a = importdata(entirefile);
x = a(:,2);
y = a(:,3);
z = a(:,4);
%% I DON'T KNOW HOW TO CREAT THIS LOOP TO COUNT FOR THE NUMBER OF PARTICLES WITHIN EACH DEFINED CUBE %%
for jj = 2:size(X,2)
%for kk=1:m
if x(:)<X(jj) & y(:)<X(jj) & z(:)<X(jj)
x;
end
%end
end
h=figure(j);
scatter3(x, y, z, 'filled', 'MarkerSize', 20);
cb = colorbar();
cb.Label.String = 'Probability density estimate';
end
I need to get a similar result like the following image. but I need the percentage of data-point specified by each color. Thanks in advance.
Here is a link to a sampled data.
Here is a way to count the 3D density of a point cloud. Although I am affraid the sample data you provided do not yield the same 3D distribution than on your example image.
To count the density, the approach is broken down in several steps:
Calculate a 2D density in the [X,Y] plane: This counts the number of points laying in each (x,y) bin. However, at this stage this number of point incorporates all the Z column for a given bin.
For each non-empty (x,y) bin, calculate the distribution along the Z column. We now have the number of point falling in each (x,y,z) bin. Counting the density/percentage is simply done by dividing each count by the total number of points.
Now for each non-empty (x,y,z) bin, we identify the linear indices of the points belonging to this bin. We then assign the bin value (color, percentage, or any value associated to this bin) to all the identified points.
display the results.
In code, it goes like this:
%% Import sample data
entirefile = '1565015520323.txt' ;
a = importdata(entirefile);
x = a(:,1);
y = a(:,2);
z = a(:,3);
npt = numel(x) ; % Total Number of Points
%% Define domain and grid parameters
nbins = 100 ;
maxDim = 300 ;
binEdges = linspace(0,maxDim,nbins+1) ;
%% Count density
% we start counting density along in the [X,Y] plane (Z axis aglomerated)
[Nz,binEdges,~,binX,binY] = histcounts2(y,x,binEdges,binEdges) ;
% preallocate 3D containers
N3d = zeros(nbins,nbins,nbins) ; % 3D matrix containing the counts
Npc = zeros(nbins,nbins,nbins) ; % 3D matrix containing the percentages
colorpc = zeros(npt,1) ; % 1D vector containing the percentages
% we do not want to loop on every block of the domain because:
% - depending on the grid size there can be many
% - a large number of them can be empty
% So we first find the [X,Y] blocks which are not empty, we'll only loop on
% these blocks.
validbins = find(Nz) ; % find the indices of non-empty blocks
[xbins,ybins] = ind2sub([nbins,nbins],validbins) ; % convert linear indices to 2d indices
nv = numel(xbins) ; % number of block to process
% Now for each [X,Y] block, we get the distribution over a [Z] column and
% assign the results to the full 3D matrices
for k=1:nv
% this block coordinates
xbin = xbins(k) ;
ybin = ybins(k) ;
% find linear indices of the `x` and `y` values which are located into this block
idx = find( binX==xbin & binY==ybin ) ;
% make a subset with the corresponding 'z' value
subZ = z(idx) ;
% find the distribution and assign to 3D matrices
[Nz,~,zbins] = histcounts( subZ , binEdges ) ;
N3d(xbin,ybin,:) = Nz ; % total counts for this block
Npc(xbin,ybin,:) = Nz ./ npt ; % density % for this block
% Now we have to assign this value (color or percentage) to all the points
% which were found in the blocks
vzbins = find(Nz) ;
for kz=1:numel(vzbins)
thisColorpc = Nz(vzbins(kz)) ./ npt * 100 ;
idz = find( zbins==vzbins(kz) ) ;
idx3d = idx(idz) ;
colorpc(idx3d) = thisColorpc ;
end
end
assert( sum(sum(sum(N3d))) == npt ) % double check we counted everything
%% Display final result
h=figure;
hs=scatter3(x, y, z, 3 , colorpc ,'filled' );
xlabel('X'),ylabel('Y'),zlabel('Z')
cb = colorbar ;
cb.Label.String = 'Probability density estimate';
As I said at the beginning, the result is slightly different than your example image. This sample set yields the following distribution:
If you want a way to "double check" that the results are not garbage, you can look at the 2D density results on each axis, and check that it matches the apparent distribution of your points:
%% Verify on 3 axis:
Nz = histcounts2(y,x,binEdges,binEdges) ./ npt *100 ;
Nx = histcounts2(z,y,binEdges,binEdges) ./ npt *100 ;
Ny = histcounts2(x,z,binEdges,binEdges) ./ npt *100 ;
figure
ax1=subplot(1,3,1) ; bz = plotDensity(Nz,ax1) ; xlabel('X'),ylabel('Y') ;
ax2=subplot(1,3,2) ; bx = plotDensity(Nx,ax2) ; xlabel('Y'),ylabel('Z') ;
ax3=subplot(1,3,3) ; by = plotDensity(Ny,ax3) ; xlabel('Z'),ylabel('X') ;
Click on the image to see it larger:
The code for plotDensity.m:
function hp = plotDensity(Ndist,hax)
if nargin<2 ; hax = axes ; end
hp = bar3(Ndist,'Parent',hax) ;
for k = 1:length(hp)
zdata = hp(k).ZData;
hp(k).CData = zdata;
hp(k).FaceColor = 'interp';
end
shading interp
I want to calculate the mean and Gaussian curvatures of some points in a point cloud.
I have x,y,z, that are coordinates and are 1d arrays. I want to use the below code src but in the input parameters X, Y and Z are 2d arrays, I don't know what means that, and how I can calculate 2d arrays corresponding to them.
function [K,H,Pmax,Pmin] = surfature(X,Y,Z),
% SURFATURE - COMPUTE GAUSSIAN AND MEAN CURVATURES OF A SURFACE
% [K,H] = SURFATURE(X,Y,Z), WHERE X,Y,Z ARE 2D ARRAYS OF POINTS ON THE
% SURFACE. K AND H ARE THE GAUSSIAN AND MEAN CURVATURES, RESPECTIVELY.
% SURFATURE RETURNS 2 ADDITIONAL ARGUMENTS,
% [K,H,Pmax,Pmin] = SURFATURE(...), WHERE Pmax AND Pmin ARE THE MINIMUM
% AND MAXIMUM CURVATURES AT EACH POINT, RESPECTIVELY.
% First Derivatives
[Xu,Xv] = gradient(X);
[Yu,Yv] = gradient(Y);
[Zu,Zv] = gradient(Z);
% Second Derivatives
[Xuu,Xuv] = gradient(Xu);
[Yuu,Yuv] = gradient(Yu);
[Zuu,Zuv] = gradient(Zu);
[Xuv,Xvv] = gradient(Xv);
[Yuv,Yvv] = gradient(Yv);
[Zuv,Zvv] = gradient(Zv);
% Reshape 2D Arrays into Vectors
Xu = Xu(:); Yu = Yu(:); Zu = Zu(:);
Xv = Xv(:); Yv = Yv(:); Zv = Zv(:);
Xuu = Xuu(:); Yuu = Yuu(:); Zuu = Zuu(:);
Xuv = Xuv(:); Yuv = Yuv(:); Zuv = Zuv(:);
Xvv = Xvv(:); Yvv = Yvv(:); Zvv = Zvv(:);
Xu = [Xu Yu Zu];
Xv = [Xv Yv Zv];
Xuu = [Xuu Yuu Zuu];
Xuv = [Xuv Yuv Zuv];
Xvv = [Xvv Yvv Zvv];
% First fundamental Coeffecients of the surface (E,F,G)
E = dot(Xu,Xu,2);
F = dot(Xu,Xv,2);
G = dot(Xv,Xv,2);
m = cross(Xu,Xv,2);
p = sqrt(dot(m,m,2));
n = m./[p p p];
% Second fundamental Coeffecients of the surface (L,M,N)
L = dot(Xuu,n,2);
M = dot(Xuv,n,2);
N = dot(Xvv,n,2);
[s,t] = size(Z);
% Gaussian Curvature
K = (L.*N - M.^2)./(E.*G - F.^2);
K = reshape(K,s,t);
% Mean Curvature
H = (E.*N + G.*L - 2.*F.*M)./(2*(E.*G - F.^2));
H = reshape(H,s,t);
% Principal Curvatures
Pmax = H + sqrt(H.^2 - K);
Pmin = H - sqrt(H.^2 - K);
You have ways to convert your x,y,z data to surface matrices/ 2D arrays. Way depends on, how and what your data is.
Structured grid data:
(i). If your x,y,z corresponds to a structured grid, then you can straight a way get unique values of x,y which gives number of points along (nx,ny) along x and y axes respectively. With this (nx,ny), you need to reshape x,y,z data into matrices X,Y,Z respectively and use your function.
(ii). If you are not okay with reshaping, you can get min and max values of x,y make your own grid using meshgrid and do interpolation using griddata.
Unstructured grid data: If your data is unstructured/ scattered, get min and max, make your grid using meshgrid and do interpolation using griddata , scatteredInterpolant.
Also have a look in the following links:
https://in.mathworks.com/matlabcentral/fileexchange/56533-xyz2grd
https://in.mathworks.com/matlabcentral/fileexchange/56414-xyz-file-functions
Let A be an n by 3 matrix, such that the first two columns are all ordered pairs of the form (5*i,5*i) for i from 1 to 200. The third column contains values from 0 to 1, which I will call intensities. I want to make a 1000 by 1000 plot so that the rectangle at (5*i,5*i) is shaded with intensity described by the third column entry.
I'm familiar with the heatmap function and imshow, but I don't see a way to include this "scaling by 5" to make a nice plot. And of course in general the x and y coordinates may not be scaled by the same amount.
Is there a nice way to do this in Matlab?
With imagesc it's actually pretty simple:
First some example data:
%// generate example data
ii = 1:200;
[xx,yy] = meshgrid(ii);
A(:,1) = 5*xx(:);
A(:,2) = 5*yy(:);
A(:,3) = randi([0,1],1,40000);
Actual answer
n = 200;
%// reshape data
D = reshape( A(:,3),n,n );
%// heatmap
imagesc(A(:,1),A(:,2),D)
colormap(gray)
caxis([0,1])
gives:
Important notice
If your coordinates are not sorted as required for imagesc you can sort them with:
A = sortrows(A,[2,1]);
Clown Example
%// original image
load clown
I = reshape(1:numel(X),size(X));
[R,C] = ind2sub(size(X),I);
A(:,1) = R(:);
A(:,2) = C(:);
A(:,3) = X(:);
D = reshape( A(:,3),200,320 );
figure(1)
subplot(1,3,1)
imagesc(A(:,1),A(:,2),D)
%// shuffled image -> shuffled data
shuffle = randperm(320*200);
A = A(shuffle,:);
D = reshape( A(:,3),200,320 );
subplot(1,3,2)
imagesc(A(:,1),A(:,2),D)
%// sorted image
A = sortrows(A,[2,1]);
D = reshape( A(:,3),200,320 );
subplot(1,3,3)
imagesc(A(:,1),A(:,2),D)
You see, even if your coordinates are sorted like a mess, you can rebuild the image with sortrows.
See this
function DrawHeatmap(X,Y,Z)
%DRAWHEATMAP Draw a 2D heatmap for (X,Y) coordinates whose values are in Z
% X, Y , Z must be columns
% By: Eng. Osama Talaat Abdel-Hafiz - PhD Student
% Egypt - Sept 2017
if size(X,2)==1 && size(Y,2)==1 && size(Z,2)==1
F = scatteredInterpolant(X,Y,Z); % create a function from interpolation
[X,Y] = meshgrid(min(X):0.1:max(X),min(Y):0.1:max(Y));
Z = F(X,Y);
contourf(X, Y, Z, linspace(floor(min(min(Z))),ceil(max(max(Z))),400), 'LineColor','none')
colorbar;
else
error('X, Y , Z must be columns')
end
end
I have a set of 2D points (not ordered) forming a closed contour, and I would like to resample them to 14 equally spaced points. It is a contour of a kidney on an image. Any ideas?
One intuitive approach (IMO) is to create an independent variable for both x and y. Base it on arc length, and interpolate on it.
% close the contour, temporarily
xc = [x(:); x(1)];
yc = [y(:); y(1)];
% current spacing may not be equally spaced
dx = diff(xc);
dy = diff(yc);
% distances between consecutive coordiates
dS = sqrt(dx.^2+dy.^2);
dS = [0; dS]; % including start point
% arc length, going along (around) snake
d = cumsum(dS); % here is your independent variable
perim = d(end);
Now you have an independent variable and you can interpolate to create N segments:
N = 14;
ds = perim / N;
dSi = ds*(0:N).'; %' your NEW independent variable, equally spaced
dSi(end) = dSi(end)-.005; % appease interp1
xi = interp1(d,xc,dSi);
yi = interp1(d,yc,dSi);
xi(end)=[]; yi(end)=[];
Try it using imfreehand:
figure, imshow('cameraman.tif');
h = imfreehand(gca);
xy = h.getPosition; x = xy(:,1); y = xy(:,2);
% run the above solution ...
Say your contour is defined by independent vector x and dependent vector y.
You can get your resampled x vector using linspace:
new_x = linspace(min(x),max(x),14); %14 to get 14 equally spaced points
Then use interp1 to get new_y values at each new_x point:
new_y = interp1(x,y,new_x);
There are a few interpolation methods to choose from - default is linear. See interp1 help for more info.
Question: is it possible to illustrate an image on non-uniform axis?
Details:
I need to illustrate a multidimensional timeseries as an image. But the time grid of this timeseries is very non-uniform. Here is an example:
m = 10;
n = 3;
t = sort(rand(m, 1)); % non-uniform time
values = randn(m, n); % some random values
The figure, plot(t, values); handles it well.
But imagesc() converts t into uniform time between t(1) and t(end) according to documentation:
imagesc(x,y,C) displays C as an image and specifies the bounds of the
x- and y-axis with vectors x and y.
Therefore, the command:
figure, imagesc(t, 1 : n, values'); colorbar;
illustrates the image on uniform time grid.
Edit: It's possible to re-sample the timeseries with higher uniform resolution. But my timeseries is already very large.
There is pcolor function in MATLAB. This function does exactly what you're asking.
m = 10;
n = 3;
t = sort(rand(m, 1)); % non-uniform time
values = randn(m, n); % some random values
figure
plot(t, values);
figure
pcolor(t, 1 : n, values');
colorbar;
try uimagesc from the file exchange.
Solution
Try using surface for non-uniform spacing.
First, create a 3D xyz surface of the same size as your input data:
m = 10;
n = 3;
t = sort(rand(m, 1)); % non-uniform time
values = randn(m, n); % some random values
x = repmat(t,1,n);
y = repmat(1:n,m,1);
z = zeros(size(y));
Then, colormap your values. There is a nice tool posted to the mathworks file exchange, real2rgb, that can do this for you:
cdata = real2rgb(values); % Where size(cdata) = [m n 3]
Lastly, plot the surface. You can even get fancy and set the transparency.
surface(x,y,z,cdata,'EdgeColor','none','FaceColor','texturemap',...
'CDataMapping','direct');
alpha(0.3)