How do i open a number of xlsx files when I only know part of the filename in Matlab? - matlab

I am using Matlab 2013b. I have a list of excel files in a directory and I want to open chosen ones within a loop to read out the data.
I only know the start of each of the file names however, not the ending. I have a vector of numbers that provide the identifying portion of the file's name (filenumber), and I want to loop through the excel files one by one, opening them and extracting the data, then closing them.
There are 500 files, each of the format: img_****ff*******.xlsx, where the first set of asterisks is my filenumber, while the second set of asterisks is unknown.
So far, I have tried listing what is in the directory using:
list=dir('E:\processed\Img*');
filenames={list.name}
This provides with with the full file names.
I tried then within a loop to create the part of the filename I know exists:
x = sprintf('Img_%d_FF_',img(1,1));
I then thought I could use 'Find' to look for my my partial filename/string in the 'filenames' structure above. I don't think I have the code correct for this datatype though:
index = find(strcmp({list.name}, x)==1)

You're pretty close, but the issue is that strcmp compares the whole string and since you only have the beginning part, it is not going to match. I would use strncmp to compare only the first n characters of the string. We can determine what n is based upon the length of your string x.
matches = strncmp({list.name}, x, numel(x));
thisfile = list(matches).name;

Related

How can I prevent MATLAB from automatically modifying .dat file variable names upon import using the dataset function?

So, I currently have a MATLAB script that does stuff with data and then, using a template .dat file, creates about 20 more .dat files with only a single column being changed (I've been using the dataset and export functions to read and write the files, respectively). The program that will use the .dat files, ExperimentBuilder, requires that the headers have names that start with dollar signs (for example: $image). However, when I use the dataset function in MATLAB to import the template file, I get this warning:
Warning: Variable names were modified to make them valid MATLAB identifiers.
It then replaces all the dollar signs in the variables to x_ (for example, x_image), which would be fine if it would let me change it back to the $ format. But whenever I try to using set , it just gives me this warning again and reverts it back to x_, which is unreadable by ExperimentBuilder.
I know I could just do a quick copy and paste on each file with the original headings, but I would like to know if there's a way to fix this problem in the actual code.
Thanks!
Thing is the MATLAB database uses the header names to provide access to the columns by name, this is why the header names must be valid identifiers (isvarname() states that it must starts with a letter, and contains only valid alphanumeric characters [a-zA-Z0-9_]).
The easiest solution would be manually write the header line yourself (including names starting with $), while separately exporting the data without the headers:
export(ds, ..., 'WriteObsNames',false)
(Note that dataset.export overwrites files by default, so you'll have to export first, then prepend the header line at the beginning of the file. Or if you're comfortable modifying MATLAB own functions, then go edit dataset.export and change the fopen mode from overwrite 'wt' to append 'at' mode).

Count number of files in a FTP folder

I use this code to see existed files in a specific folder from FTP :
ftp_client = ftp('IP','Username','Password');
aa = dir(ftp_client,'First_folder/Second_folder');
I can see file names with these codes :
aa(1,1).name
aa(2,1).name
aa(3,1).name
How can I see all files names in a cell aray in this specific folder? Is there any command for it?
How can I count number of existed files in this folder?
How can I count number of existed files in this folder with a specific format?
Thanks.
A simple way is to collect the values into the cell array by using curly brackets: filenames = {aa.name};
The simplest method would be length(aa); or length(filenames);
A couple ways. You can either refine your dir call, aa = dir(ftp_client,'First_folder/Second_folder/*.jpeg') for example, or use your own filter (regexp is one option) on your filenames to return the indices of what you want.
As a general aside, if you're utilizing this program on different operating systems, I would recommend utilizing fullfile (or filesep at the very least) to build your full pathnames to ensure the right separator is used. Though I didn't do it in my example above...

How to read this line in MATLAB?

I came across the following lines in MATLAB:
m = dir(fullfile(dataset,'*.png'));
m = {m(~[m.isdir]).name};
I understand that the first line is trying to obtain the .png files from a directory. But, what is the second line trying to perform? isdir seems to determine that an input is a directory. That's what I new for that part. But, what is the line trying to perform?
Thanks.
The second line is getting all files that are not a directory and then getting the respective names and storing them into a cell array
m.isdir indicates if it is a folder or not
returns 1 if it is, 0 if not.
~[m.isdir] will indicate which of the values returned from isdir was a 0.
m(~[m.isdir]) grabs all objects in m determined by the logical indexing done above
m(~[m.isdir]).name gets the names of all of them
{m(~[m.isdir]).name} stores them all in cell array
Hopefully this step by step walkthrough helps.
While I am not sure why the second line is necessary because the fullfile(dataset,'*.png') should only return paths that end in .png, which will not be a folder, I guess it is good to check.

Writing a script for reading many .csv files with similar filenames

I have several .csv files with similar filenames except a numeric month (i.e. 03_data.csv, 04_data.csv, 05_data.csv, etc.) that I'd like to read into R.
I have two questions:
Is there a function in R similar to
MATLAB's varname and assignin that
will let me create/declare a variable name
within a function or loop that will allow me to
read the respective .csv file - i.e.
03_data.csv into 03_data data.frame,
etc.? I want to write a quick loop to
do this because the filenames are
similar.
As an alternative, is it better to
create one dataframe with the first
file and then append the rest using a
for loop? How would I do that?
You could look at this related question. You can create the file names easily with a paste command:
file.names <- paste(sprintf("%02d",1:10), "_data.csv", sep="")
Once you have your file names (whether by creating them or by reading them from the directory as in the other question), you can import them quickly with an lapply:
import.list <- lapply(file.names, read.csv)
Lastly, to combine the list into one dataframe, the easiest approach is to use the reshape function below:
library(reshape)
data <- merge_recurse(import.list)
It is also very easy to read the content of a directory including use of regular expressions to skip focus on certain names only, e.g.
filestoread <- list.files(someDir, pattern="\\.csv$", full.names=TRUE)
returns all (fully-formed, including full path) files in the given directory someDir that end on ".csv". You can get fancier with better regular expressions which are documented in many places.
Once you have your list of files, it is straightforward to read them all using apply or lapply or a loop.

In Matlab, how do I create a CSV file from a subset of the lines in a text file?

I need to open a text file and convert it into a CSV file in Matlab. The first 3 lines of the text file are sentences that need to be omitted. The next 28 lines are numbers that need to make up the first column of the CSV, and then the next 28 lines need to make up the second column.
The text file is called datanal.txt and the output file can be named anything. Any help would be appreciated.
Don't have Matlab now to test, but try this. Your input file should be in Matlab's current directory, or put the full path to the file name.
A = csvread('datanal.txt',3,0);
A = reshape(A,28,2);
csvwrite('output.csv',A)
well you can add #'s in front of the first 3 lines then use load and a reshape. Did you need a fully automated script or is there only one file? If you're familiar with matlab at all there are a bunch of ways to turn that large column vector into a matrix.