Interact with very large Tiff (or rset) images in MATLAB - matlab

I have some very large Tiff images that I am trying to use in a MATLAB GUI application. If I try to load the images using imshow, I get an out-of-memory error. (Yes, I know MATLAB is not the best choice for GUIs or loading large images, but there is good reason for using MATLAB in this case).
I can obviously create a reduced resolution data set (rset file) and use imtool to view the image, but this is not helpful as I want a user to be able to interact with the image by clicking on it to extract (x,y) coords into the application. Imshow does not seem to be directly compatible with rset files. Is there a way for me to load an rset'd image in a panable/zoomable figure, or any other way I can achieve the goal?
I looked at the code for imtool but it seems to be using undocumented classes to read rset files and I can't replicate its behaviour.

You can use low-level file I/O functions of MATLAB to read the entirety or parts of the TIFF image in order to avoid the OOM problem.
fileName = 'LargeTiff.tif';
info = imfinfo(fileName)
% Determine number of frames
nFramesStr= regexp(info.ImageDescription, 'images=(\d*)', 'tokens');
nFrames = str2double(nFramesStr{1}{1});
% Use low-level File I/O functions to read the file
fp = fopen(fileName , 'rb');
% The "StripOffsets" field provides the offset to the first strip.
fseek(fp, info.StripOffsets, 'bof');
% Assume that the image format is 16-bit per pixel and is big-endian
% Also assume that the images are stored one after the other
% For example, read the first 100 frames
frameNum = 100;
imData = cell(1, frameNum);
for cnt = 1 : frameNum
imData{cnt} = fread(fp, [info.Width info.Height], 'uint16', 0, 'ieee-be');
end
fclose(fp);

It looks like my problem is that I simply don't have enough memory to load the whole tiff, and that there is no public specification for the rset files format. So I am going to instead solve the problem by creating my own version of a reduced resolution data set. I should be able to load block sections of the image, resave them and then dynamically load and unload only the needed high-res blocks at zoom, and load a reduced resolution overview when zoomed out.

You can write a callback function to get the pixel co-ordinates (X,Y) from imtool then convert to a tile number and tile index using the code below. You can then utilise the readencodedtile function in matlab
function [tileidx,Tile_num] = getTileInfo(tiffile,X,Y)
A = Tiff(tiffile);
tile_width = A.getTag('TileWidth');
tile_length = A.getTag('TileLength');
SizeA = size(A);
tt = sub2ind(SizeA,X,Y);
% Example only
% X = repmat((1:10)',1,10);
% Y = repmat((1:10),10,1);
% A = reshape(1:100,10,10);
% SizeA = size(A);
% tile_width = 3;
% tile_length = 2;
tileidx = rem(tt-(Y-1)*SizeA(1)-1,tile_length)+1 ...
+ tile_length*rem(Y+tile_width-1,tile_width);
Tile_num = ceil(Y/tile_width)+ ...
(ceil(X/tile_length)-1)*ceil(SizeA(2)/tile_width);

Related

How can I read a bunch of TIFF files faster in Matlab?

I have to read in hundreds of TIFF files, perform some mathematical operation, and output a few things. This is being done for thousands of instances. And the biggest bottleneck is imread. Using PixelRegion, I read in only parts of the file, but it is still very slow.
Currently, the reading part is here.
Can you suggest how I can speed it up?
for m = 1:length(pfile)
if ~exist(pfile{m}, 'file')
continue;
end
pConus = imread(pfile{m}, 'PixelRegion',{[min(r1),max(r1)],[min(c1),max(c1)]});
pEvent(:,m) = pConus(tselect);
end
General Speedup
The pixel region does not appear to change at each iteration. I'm not entirely sure if Matlab will optimize the min and max calls (though I'm pretty sure it won't). If you don't change them at each iteration, move them outside the for loop and calculate them once.
Parfor
The following solution assumes you have access to the parallel computing toolbox. I tested it with 10,840 tiffs, each image was 1000x1000 originally, but I only read in a 300x300 section of them. I am not sure how many big pConus(tselect) is, so I just stored the whole 300x300 image.
P.S. Sorry about the formatting. It refuses to format it as a block of code.
Results based on my 2.3 GHz i7 w/ 16GB of ram
for: 130s
parfor: 26s + time to start pool
% Setup
clear;clc;
n = 12000;
% Would be faster to preallocate this, but negligeble compared to the
% time it takes imread to complete.
fileNames = {};
for i = 1:n
name = sprintf('in_%i.tiff', i);
% I do the exist check here, assuming that the file won't be touched in
% until the program advances a files lines.
if exist(name, 'file')
fileNames{end+1} = name;
end
end
rows = [200, 499];
cols = [200, 499];
pics = cell(1, length(fileNames));
tic;
parfor i = 1:length(fileNames)
% I don't know why using the temp variable is faster, but it is
temp = imread(fileNames{i}, 'PixelRegion', {rows, cols});
pics{i} = temp;
end
toc;

How to generate valid HyperStack ImageJ data files from Matlab?

ImageJ HyperStacks have a data type (8-32 bit), a width, a height, a number of channels, a number of slices and a number of frames. They represent 5D (XYCZT) data sets. ImageJ stores them as multipage tiff files where number of channels times number of slices times number of frames 2D (XY) images are stored. The first image seems to have two custom tags with IDs 50838 and 50839.
I want to create tif files containing 5D data from Matlab that can be read by ImageJ as a valid 5D HyperStack.
I can store a number of 2D images in a multipage tiff file in Matlab using imwrite(matrix, file, 'WriteMode','append') but ImageJ will read that as 3D (XYZ) image stack only. The information about the channels, slices and frames is not contained.
I guess I could look at the ImageJ sources to find out where they store this missing information, then use Matlab's wrapper of LibTIFF to recreate the meta information of ImageJ. If you, however, already know what to do or if there is an alternative, I would like to hear it.
I also thought that Bio-Fomats was the only way. But then I realized that Fiji's ImageJ-MATLAB allows you to pass MATLAB data to ImageJ. Then, you can save the image by ImageJ, so ImageJ can open it for sure.
The problem was that people (or just one guy?) created ImageJ-MATLAB did a great job in creating a route from MATLAB to ImageJ, but they didn't seem to bother optmizing the behaviour. ImageJ-MATLAB's IJM.show('name') had a lot of hidden or undocumented limitations (I massively changed the documentation, so it should be clearer now). Eventually, I enede up wriring a wrapper function
ijmshow
https://github.com/kouichi-c-nakamura/ijmshow
Example MATLAB code
You can do it with 4 lines.
addpath '/Applications/Fiji.app/scripts' % depends your Fiji installation
ImageJ
imp = ijmshow(I,'YXCZT') % I is 5D array of uint8 or uint16
% the second argument determines which dimension represents what
% imp is a 5D hyperstack
ij.IJ.saveAsTiff(imp, 'image1.tif');
You may want to set Display Range of each channel before saving.
Added on 5/7/2018
copytoImagePlus provides better solution than ijmshow above (you can use the same syntax), because copytoImagePlus does not rely on the IJM variable in the base workspace anymore.
https://github.com/kouichi-c-nakamura/copytoImagePlus
addpath '/Applications/Fiji.app/scripts' % depends your Fiji installation
ImageJ
imp = copytoImagePlus(I,'YXCZT') % I is 5D array of uint8 or uint16
% the second argument determines which dimension represents what
% imp is a 5D hyperstack
ij.IJ.saveAsTiff(imp, 'image1.tif');
Also see copytoImgPlus which create ImageJ2 ImgPlus object instead.
https://github.com/fiji/fiji/blob/master/scripts/copytoImgPlus.m
ImageJ uses the ImageDescription tag of the TIFF's first IFD to store its meta-information. Here is an example from the Mitosis sample dataset (File > Open Samples > Mitosis):
ImageJ=1.51d
images=510
channels=2
slices=5
frames=51
hyperstack=true
mode=composite
unit=\u00B5m
finterval=0.14285714285714285
loop=false
min=1582.0
max=6440.0
MATLAB's Tiff class might give you fine-grained enough control to write a custom ImageDescription to the first IFD; I am not sure.
Probably easier would be to use the Bio-Formats library's bfsave.m function to write OME-TIFF, which can be 5-dimensional; ImageJ can read this format via the Bio-Formats plugin. I recommend using the Fiji distribution of ImageJ, which comes with Bio-Formats pre-installed.
There was also this same question posted on MATLAB Central.
Using Matlab's Tiff class.
I can assemble an ImageDescription tag that I think looks fine and ImageJ reads it (File>Open) as a 5D Hyperstack but the data is partly corrupted. There seem to be offset problems (I can read the data just fine in again with Matlab).
However, if reading the tiff file with File>Import>Bio-Formats it becomes 5D (XYCZT) stack in ImageJ if view stack with Hyperstack is marked in the Bio-Formats import dialog. Thanks to ctrueden for his helpful comment.
d = ones(100, 200, 10, 2, 3, 'single') * 3.765;
hyperstack_write('test.tif', d);
function hyperstack_write(file, hyperstack)
% Writes an (up to) 5D stack into a (XYCZT) HyperStack Tiff file
% readable by ImageJ
%
% hyperstack must have class single
% simple checks
assert(nargin == 2, 'Not enough arguments');
assert(isa(hyperstack, 'single'), 'hyperstack must be single');
% get all five dimensions
d = zeros(5, 1);
for i = 1 : 5
d(i) = size(hyperstack, i);
end
% assemble image description
s = sprintf('ImageJ=1.51\nnimages=%d\nchannels=%d\nslices=%d\nframes=%d\nhyperstack=true\nmode=color\nloop=false\nmin=%.1f\nmax=%.1f\n', prod(d(3:5)), d(3), d(4), d(5), floor(min(hyperstack(:))*10)/10, ceil(max(hyperstack(:))*10)/10);
% open tif file for writing and set file tags
t = Tiff(file, 'w');
ts.ImageLength = d(1);
ts.ImageWidth = d(2);
ts.Photometric = Tiff.Photometric.MinIsBlack;
ts.Compression = Tiff.Compression.None;
ts.BitsPerSample = 32;
ts.SamplesPerPixel = 1;
ts.SampleFormat = Tiff.SampleFormat.IEEEFP;
ts.RowsPerStrip = 5;
ts.PlanarConfiguration = Tiff.PlanarConfiguration.Chunky;
ts.Software = 'MATLAB';
ts.ImageDescription = s;
% loop over dimensions 3, 4, and 5
for k = 1 : d(5)
for j = 1 : d(4)
for i = 1 : d(3)
frame = hyperstack(:, :, i, j, k);
t.setTag(ts)
t.write(frame);
t.writeDirectory();
end
end
end
% close tif file
t.close();
end
It seems Bio-Formats is the only fully paved and available way to export >2D data from Matlab to ImageJ, although this solution here could be a lightweight alternative in cases where not much metadata is available and needs to be transferred.

MATLAB loading and saving a single image from a 32 bit tiff stack

I'm using MATLAB_R2011a_student. I have some image stacks saved as 32 bit tiffs, some over 1000 frames. I would like to be able to pull out a specific frame from the stack and save it as a 32 bit tiff or some readable format where there would be no data loss from the original. Currently my code looks like this:
clear, clc;
k=163;
image=('/Users/me/Filename.tiff');
A = uint8(imread(image, k));
B=A(:,:,1);
J=imadjust(B,stretchlim(B),[]);
imwrite(J,'/Users/me/163.tif','tif');
(I'm assuming reading it as 8 bit, and the way I'm saving are not the best way to do this)
Either way this code works for a seemingly random number of frames (for example in one file.tiff the above code works for frames 1-165 but none of the frames after 165, for a different file.tiff the code works for frames 1-8 but none of the frames after 8) I'm also getting a strange horizontal line in the output image when this does work:
??? Error using ==> rtifc
Invalid TIFF image index specified.
Error in ==> readtif at 52
[X, map, details] = rtifc(args);
Error in ==> imread at 443
[X, map] = feval(fmt_s.read, filename, extraArgs{:});
Thanks!
The best way (in my opinion) to handle tiff stacks is to use the Tiff library available since a few years. I must admit that I don't know much about OOP but I managed to understand enough to load a tiff stack and manipulate it.That's the kind of simple demo I wish I had seen a year ago haha.
I the following example I load a single stack and store it all into a 3D array. I use imfinfo to fetch infos about the images, notably the number of images/stack and the actual image dimensions. If you want you can choose to load only one image using appropriate indices. Please try the code below and play around with it; you'll understand what I mean.
clear
clc
%// Get tiff files you wish to open
hFiles = dir('*.tif');
%// Here I only have 1 multi-tiff file containing 30 images. Hence hInfo is a 30x1 structure.
hInfo = imfinfo(hFiles(1).name);
%// Set parameters.
ImageHeight = hInfo(1).Height;
ImageWidth = hInfo(1).Width;
SliceNumber = numel(hInfo);
%// Open Tiff object
Stack_TiffObject = Tiff(hFiles.name,'r');
%// Initialize array containing your images.
ImageMatrix = zeros(ImageHeight,ImageWidth,SliceNumber,'uint32');
for k = 1:SliceNumber
%// Loop through each image
Stack_TiffObject.setDirectory(k)
%// Put it in the array
ImageMatrix(:,:,k) = Stack_TiffObject.read();
end
%// Close the Tiff object
Stack_TiffObject.close
Hope that helps.

Matlab figures - remove filling from gaussian distributions

I am emulating a NAND Flash mlc and I am using hist() to display the results which are 4 gaussian distributions.
The problem is that I am trying to display 8 distributions at once one the same figure and they overlap. So, I want to find a way to remove the filling inside the gaussian distributions and only keep the outline. Is there a way to do this? I have found that maybe "basic fitting" option in Matlab can do this but it is "grey" and I cannot select it on this figure.
--
Well I tried adding a simple a picture which can explain everything but I could not due to my reputation.
I am working in FPGA(ZedBoard) in order to create the emulation but that has nothing to do with the question I am asking.
So, I get an input file with single-precision values out of ZedBoard, which then I insert in Matlab and get the figure I am talking about.
I would not think that my code in Matlab could help in any way that is why I did not put it.
Although, I should have mentioned that the eight distributions have almost the exact same characteristics so they appear almost on the same place. This is why I am looking a way to display them properly.
These "hists" below have to appear on the same figure without overlapping on each other. Normally, when Matlab is displaying gaussian distributions with "hist" uses a blue filling which I want to remove and only leave the outlining of the distribution if possible. I hope I have given a better view of my problem with this added.
fid1 = fopen('soft_without.txt');
A = textscan(fid1,'%s');
B = char(A{1});
fclose (fid1);
output = typecast(uint32(hex2dec(B)),'single');
hist(output,1000);
fid2 = fopen('soft_with.txt');
A = textscan(fid2,'%s');
B = char(A{1});
fclose (fid2);
output1 = typecast(uint32(hex2dec(B)),'single');
hist(output1,1000);
Let's first put all the data into one dataset, rather than having a bunch of different variables:
filenames = ls('*.txt'); % or whatever you do to make up your list of files
data = zeros(1000, 8); %preallocate
% going to use first file to set location of bins so they're all the same
fid = fopen(filenames(1,:))
A = textscan(fid,'%s');
B = char(A{1});
fclose(fid);
output = typecast(uint32(hex2dec(B)),'single');
[x, bins] = hist(output,1000);
data(:,1) = x;
% set rest to same bins
for n = 2:8
fid = fopen(filenames(n,:))
A = textscan(fid,'%s');
B = char(A{1});
fclose(fid);
output = typecast(uint32(hex2dec(B)),'single');
x = hist(output,bins);
data(:,n) = x;
end
bar(bins,data);
This will plot your eight sets as eight different (side by side) bars. Another route would be to plot the over the top of each other but set different edge colors and no face color (possibly also experiment with changing width).
colors = jet(8); % eight colors from the jet colormap
figure; hold on;
for n = 1:8
bar(bins,data(:,n),'facecolor','none','edgecolor', colors(n,:));
end

For Loop not advancing

I'm trying to read in a large number of jpg files using a for loop. But for some reason, the k index is not advancing. I just get A as a 460x520x3 uint8. Am I missing something?
My goal with this code is to convert all the jpg images to the same size. Since I haven't been able to advance through the images, I can't quite tell if I'm doing it right.
nFrames = length(date); % Number of frames.
for k = 1:nFrames-1 % Number of days
% Set file under consideration
A = imread(['map_EUS_' datestr(cell2mat(date_O3(k)),'yyyy_mm_dd') '_O3_MDA8.jpg']);
% Size of existing image A.
[rowsA, colsA, numberOfColorChannelsA] = size(A);
% Read in and get size of existing image B (the next image).
B = imread(['map_EUS_' datestr(cell2mat(date_O3(k+1)),'yyyy_mm_dd') '_O3_MDA8.jpg']);
[rowsB, colsB, numberOfColorChannelsB] = size(B);
% Size of B does not match A, so resize B to match A's size.
B = imresize(B, [rowsA colsA]);
eval(['print -djpeg map_EUS_' datestr(cell2mat(date_O3(k)),'yyyy_mm_dd') '_O3_MDA8_test.jpg']);
end
end
As you use imread to read the image in, it makes sense to use imwrite to write it out, rather than print/eval (also you should always think twice about using eval in MATLAB, and then think again).
You could also speed up this code somewhat - you just want to resize all the images to the size of whichever the first one is, so you don't need to keep reading in images to measure the size of them. Use imfinfo to get the size, then read in, resize, and write out B only.