I have been struggling to write the code to read in .txt data. I am in a directory where all of the file names are ‘img0001.txt’ through ‘img4200.txt’. Each file is a 2-D array of the same size (480x640), eventually I want to fill a 3-D data cube, but first I need to be able to read in all the data.
for i = 1:4200
i
if i<10
A = csvread(['img000',num2str(i),'.txt']);
elseif 10<=i<100
A = csvread(['img00',num2str(i),'.txt']); ***
elseif 100<=i<1000
A = csvread(['img0',num2str(i),'.txt']);
else i>=1000
A = csvread(['img',num2str(i),'.txt']);
end
end
The code prints i=100, and then gives me an error message for a file not found in the line where I added ***. The code is looking for file img00100.txt which does not exist, but I’m not sure why it is doing this.
I have been playing with different versions of writing the if, elseif, statements and the greater than and less than operators. I have also tried using eval and load commands.
Thank you.
m7913d's answer explains your if statement's logic, but a nice way to avoid the confusion would be to remove the if statements entirely using sprintf.
for i = 1:4200
filenum = sprintf('%04d', i); % Zero pads the number e.g. 59 => 0059
disp(['i =', filenum]) % Display current i
A = csvread(['img', filenum, '.txt']); % Load CSV
end
Your condition to check the range is wrong. You should write it as follows:
elseif 10 <= i && i < 100
What you calculated is the following (explained for i == 100):
10 <= i < 100 <=> (10 <= i) < 100 <=> (1) < 100 <=> 1
Note that this is the case for a lot of programming languages (C++, java, js, ...).
Related
I have an application, A, that writes to a File.
I want to use MATLAB to read N lines in realtime from this file.
My question is related to this stack post: How to plot real time data from text file in MATLAB
The author of one of the answers, mentions the following approach:
You can't plot using hard real time conditions, thus it can always happen that matlab misses a 10ms timeslot. You have to use option 2 to get all data.
To get started: Write a function which only reads the new data which was written since last call. To achieve this, do not close the file handle. It stores the position.
As such, here is my code:
myfile_fid=fopen(filePath, 'rt')
waitForFileToHaveData(filePath, 10);
for readingIdx = 1:10
fgetl(myfile_fid)
end
My waitForFileToHaveData function, is defined as follows:
function waitForFileToHaveData(filePath, desired_length)
if (getNumLinesOfFile(filePath) < desired_length)
disp('###### not enough data in the file');
pause(0.02);
waitForFileToHaveData(filePath, desired_length);
end
end
function num = getNumLinesOfFile(file_to_read)
[status, result] = system( ['wc -l ', file_to_read] );
if(status~=1)
scanCell = textscan(result,'%u %s');
num = scanCell{1} - 2;
else
num = 0;
end
end
Result:
When I get into the for loop, myfile_fid evaluates to 3, while fgetl(myfile_fid) evaluates to -1. If I print out the results of getNumLinesOfFile(filePath), I see 20. The odd part is that if I wait, say for the file to have 40 lines, and execute the code above, I do not get the error. I tried to look at the documentation to see why fgetl returns back -1, but I cannot seem to find it in 2018b MATLAB documentation. There is mention that the myfile_fid can return a -1, but that is only if the file cannot be opened. However, at runtime, myfile_id evaluates to 3.
Using MATLAB, is it possible to read N number of lines since last read in a file that is being written to by another application?
fgetl returns -1 when fileID reaches the end-of-file marker, See Matlab fgetl documentation. This means that if the first result from fgetl is -1 then the file is empty.
I'm not sure why you are getting -1 if getNumLinesOfFile returns 20, check the code carefully if you are reading the same file. Maybe the file has changed?
I wrote here MATLAB code that checks if 10 new lines were added and and then gets them with fgetl:
myfile_fid = fopen(filePath, 'rt');
newLines = 10;
linesRead = 0;
while(waitForFileToHaveData(filePath, linesRead + newLines))
linesRead = linesRead + newLines;
for readingIdx = 1:newLines
line = fgetl(myfile_fid)
end
end
fclose(myfile_fid);
I updated the waitForFileToHaveData function to return 1:
function ready = waitForFileToHaveData(filePath, desired_length)
while (getNumLinesOfFile(filePath) < desired_length)
disp('###### not enough data in the file');
pause(0.02);
end
ready = 1;
end
Note:
If the file had exactly 10 lines with no end-line marker at line 10, and you read them, then another 10 lines were added, fileID now points to the end-line of line 10 and the first line fgetl will return is the end-line at line 10, but since fgetl removes the end-line, it returns an empty array.
Side note:
the function waitForFileToHaveData uses recursion which is inefficient. You can easily use a while loop.
L=1; Nx=51; PeriodicFlag=1; T=15; Nt=51;
spacedef='Pade6FirstDeriv'; casedef='LinearAdvection';
if (spacedef == 'Pade6FirstDeriv')
D1 = sparse(Pade6(Nx,dx,PeriodicFlag));
elseif (spacedef == 'Upwind3FirstDeriv')
D1 = sparse(Upwind3(Nx,dx,PeriodicFlag));
elseif (spacedef == 'Central4FirstDeriv')
D1 = sparse(Central4(Nx,dx,PeriodicFlag));
elseif (spacedef == 'Central2FirstDeriv')
D1 = sparse(Central2(Nx,dx,PeriodicFlag));
else
error(sprintf('Unknown spacedef = %s',spacedef));
end
In the above code, the if section is a small segment from a function I've constructed. I'm trying to get the function to know which methods to use based on my input (spacedef). Central2, Central4, Upwind3, and Pade6 are other functions I've written. The weird thing is that when spacedef =/= to 'Pade6FirstDeriv', I would get an error stating Error using ==, Matrix dimensions must agree. I've tried swapping the order of the if loop (by placing Central4, Central2, Pade6, and Upwind3 in the first line of the loop), and it seems like only the top line of the loop will work (the elseifs are not working). I'd greatly appreciate it if anybody can help me out. Thanks!
As has been noted in the comments, this is a common error when people first start comparing strings in MATLAB, and strcmp or strcmpi is generally the solution.
However, one solution the questions links in the comments don't present, and a solution I think looks much nicer, is the switch statement:
switch (spacedef)
case('Pade6FirstDeriv')
D1 = sparse(Pade6(Nx,dx,PeriodicFlag));
case('Upwind3FirstDeriv')
D1 = sparse(Upwind3(Nx,dx,PeriodicFlag));
case('Central4FirstDeriv')
D1 = sparse(Central4(Nx,dx,PeriodicFlag));
case('Central2FirstDeriv')
D1 = sparse(Central2(Nx,dx,PeriodicFlag));
otherwise
error(sprintf('Unknown spacedef = %s',spacedef));
end
Note: if I expect others to use my code with string comparisons, I usually lower the input such that the comparison is case-insensitive, although I have not done that here.
the function dir returns an array like
.
..
Folder1
Folder2
and every time I have to get rid of the first 2 items, with methods like :
for i=1:numel(folders)
foldername = folders(i).name;
if foldername(1) == '.' % do nothing
continue;
end
do_something(foldername)
end
and with nested loops it can result in a lot of repeated code.
So can I avoid these "folders" by an easier way?
Thanks for any help!
Edit:
Lately I have been dealing with this issue more simply, like this :
for i=3:numel(folders)
do_something(folders(i).name)
end
simply disregarding the first two items.
BUT, pay attention to #Jubobs' answer. Be careful for folder names that start with a nasty character that have a smaller ASCII value than .. Then the second method will fail. Also, if it starts with a ., then the first method will fail :)
So either make sure you have nice folder names and use one of my simple solutions, or use #Jubobs' solution to make sure.
A loop-less solution:
d=dir;
d=d(~ismember({d.name},{'.','..'}));
TL; DR
Scroll to the bottom of my answer for a function that lists directory contents except . and ...
Detailed answer
The . and .. entries correspond to the current folder and the parent folder, respectively. In *nix shells, you can use commands like ls -lA to list everything but . and ... Sadly, MATLAB's dir doesn't offer this functionality.
However, all is not lost. The elements of the output struct array returned by the dir function are actually ordered in lexicographical order based on the name field. This means that, if your current MATLAB folder contains files/folders that start by any character of ASCII code point smaller than that of the full stop (46, in decimal), then . and .. willl not correspond to the first two elements of that struct array.
Here is an illustrative example: if your current MATLAB folder has the following structure (!hello and 'world being either files or folders),
.
├── !hello
└── 'world
then you get this
>> f = dir;
>> for k = 1 : length(f), disp(f(k).name), end
!hello
'world
.
..
Why are . and .. not the first two entries, here? Because both the exclamation point and the single quote have smaller code points (33 and 39, in decimal, resp.) than that of the full stop (46, in decimal).
I refer you to this ASCII table for an exhaustive list of the visible characters that have an ASCII code point smaller than that of the full stop; note that not all of them are necessarily legal filename characters, though.
A custom dir function that does not list . and ..
Right after invoking dir, you can always get rid of the two offending entries from the struct array before manipulating it. Moreover, for convenience, if you want to save yourself some mental overhead, you can always write a custom dir function that does what you want:
function listing = dir2(varargin)
if nargin == 0
name = '.';
elseif nargin == 1
name = varargin{1};
else
error('Too many input arguments.')
end
listing = dir(name);
inds = [];
n = 0;
k = 1;
while n < 2 && k <= length(listing)
if any(strcmp(listing(k).name, {'.', '..'}))
inds(end + 1) = k;
n = n + 1;
end
k = k + 1;
end
listing(inds) = [];
Test
Assuming the same directory structure as before, you get the following:
>> f = dir2;
>> for k = 1 : length(f), disp(f(k).name), end
!hello
'world
a similar solution from the one suggested by Tal is:
listing = dir(directoryname);
listing(1:2)=[]; % here you erase these . and .. items from listing
It has the advantage to use a very common trick in Matlab, but assumes that you know that the first two items of listing are . and .. (which you do in this case). Whereas the solution provided by Tal (which I did not try though) seems to find the . and .. items even if they are not placed at the first two positions within listing.
Hope that helps ;)
If you're just using dir to get a list of files and and directories, you can use Matlab's ls function instead. On UNIX systems, this just returns the output of the shell's ls command, which may be faster than calling dir. The . and .. directories won't be displayed (unless your shell is set up to do so). Also, note that the behavior of this function is different between UNIX and Windows systems.
If you still want to use dir, and you test each file name explicitly, as in your example, it's a good idea to use strcmp (or one of its relations) instead of == to compare strings. The following would skip all hidden files and folder on UNIX systems:
listing = dir;
for i = 1:length(listing)
if ~strcmp(listing(i).name(1),'.')
% Do something
...
end
end
You may also wanna exclude any other files besides removing dots
d = dir('/path/to/parent/folder')
d(1:2)=[]; % removing dots
d = d([d.isdir]) % [d.isdir] returns a logical array of 1s representing folders and 0s for other entries
I used: a = dir(folderPath);
Then used two short code that return struct:
my_isdir = a([a.isdir]) Get a struct which only has folder info
my_notdir = a(~[a.isdir]) Get a struct which only has non-folder info
Combining #jubobs and #Tal solutions:
function d = dir2(folderPath)
% DIR2 lists the files in folderPath ignoring the '.' and '..' paths.
if nargin<1; folderPath = '.'; elseif nargin == 1
d = dir(folderPath);
d = d(~ismember({d.name},{'.','..'}));
end
None of the above puts together the elements as I see the question having being asked - obtain a list only of directories, while excluding the parents.
Just combining the elements, I would go with:
function d = dirsonly(folderPath)
% dirsonly lists the unhidden directories in folderPath ignoring '.' and '..'
% creating a simple cell array without the rest of the dir struct information
if nargin<1; folderPath = '.'; elseif nargin == 1
d = dir(folderPath);
d = {d([d.isdir] & [~ismember({d.name},{'.','..'})]).name}.';
end
if hidden folders in general aren't wanted the ismember line could be replaced with:
d = {d([d.isdir] & [~strncmp({d.name},'.',1)]).name}.';
if there were very very large numbers of interfering non-directory files it might be more efficient to separate the steps:
d = d([d.isdir]);
d = {d([~strncmp({d.name},'.',1)]).name}.';
We can use the function startsWith
folders = dir("folderPath");
folders = string({folders.name});
folders = folders(~startsWith(folders,"."))
Potential Solution - just remove the fields
Files = dir;
FilesNew = Files(3:end);
You can just remove them as they are the first two "files" in the structure
Or if you are actually looking for specific file types:
Files = dir('*.mat');
I have a .dat file that I want to read with a program in Fortran 90. The data file contains only one very long column of complex numbers. My problem is that I want to read only one part of the column, e.g., only from the nth up to the mth element including, and create an one-dimensional array from it. However, if I write something like
open(unit = 100, file = 'datafile.dat', status = 'old', action = 'read')
do j = n,m
read(100,*) vec(j-n+1)
end do
close(100)
the program reads the first m-n+1 elements from my file and not the ones from n to m including. Is there any way to do what I want?
Well, there is an obvious solution - you can simply execute n-1 dummy reads before reading actual data:
open(unit = 100, file = 'datafile.dat', status = 'old', action = 'read')
do j = 1,n-1
read(100,*)
end do
do j = n,m
read(100,*) vec(j-n+1)
end do
close(100)
You can also use the / edit descriptor and condense the first loop into a single read statement:
nm2 = n-2
read(100,'(<nm2>/)')
do j = n,m
read(100,*) vec(j-n+1)
end do
(this is n-2 records are being skipped because of the <nm2>/ edit descriptor and one record is skipped by the dummy read)
With compilers that do not support the <x> format extension, you can construct the format descriptor in an internal file:
character(len=20) :: fmt
write(fmt, "('(',I0,'/)')") n-2
read(100,fmt)
do j = n,m
read(100,*) vec(j-n+1)
end do
how to write a program in matlab that reads a certain number of images let's say 20 for example which are saved in a given directory (C:) such that later i can use them. suppose that the images are saved by numbers. later, i am gonna use them.
I'd have the code look something like this. Assuming cell array im holds your images.
Write out:
IMG_DIR = 'C:\';
filename_root = 'image';
IMG_EXT = '.jpg';
NUM_IMAGES = 20;
for i = 1:NUM_IMAGES
imwrite(im{i}, [IMG_DIR filename_root num2str(i) IMG_EXT]);
end
Read in:
for i = 1:NUM_IMAGES
im{i} = imread([IMG_DIR filename_root num2str(i) IMG_EXT]);
end
If you don't know how many there are, you can also use ls command (works differently in Windows vs. Linux).
If you don't know, in advance, which files will be in there, but you know that they have the string in them, 'rawImage' (like 'rawImage001.jpg' etc.) you can do something like
a = dir('c:\temp');
requiredBaseFileName = 'rawImage'; % you want them to contain the substring 'rawImage'
for i = 1:length(a),
fileName = a(i).name;
if(isempty(strfind(fileName,'.jpg')) & isempty(strfind(fileName,'.png')))
continue;
end
if(isempty(strfind(fileName,requiredBaseFileName)))
continue;
end
% do your processing here
end