I'm trying to append many files in Stata using loop. I've tried this answer: Stata: looping and appending
And below is my code.
clear
local pathdir "Data\RawData\edd"
local files: dir "`pathdir'" files "test_*.dta"
save "`pathdir'/master.dta", emptyok replace
foreach file in `files' {
use "`pathdir'/`file'", clear
append using "`pathdir'/master.dta"
save "`pathdir'/master.dta", replace
}
It just gives me empty space. I'm not really sure what to do. Why doesn't this work? Thank you for any help.
Let me just add that looping to append files is not necessary.
clear
local pathdir "Data\RawData\edd"
local files: dir "`pathdir'" files "test_*.dta"
append using `files'
save "whatever.dta"
Related
In the below code I am trying to sort files based on a string within the name. I've been piecing this together with google searches and community help (I'm very new at matlab). Right now I'm getting two odd errors. First, when I try and make a folder, it creates some file (highlighted filein picture that I can't open and the wav files that should have been moved to the folder disappear.
I'm also having an issue where the code renames the first two data files moved to "01" and "01 (1)" and I have no idea why.
DirIn = 'C:\Folder\Experiment' %set incoming directory
eval(['filelist=dir(''' DirIn '/*.wav'')']) %get file list
for i = 1:length(filelist);
Filename = filelist(i).name
name = strsplit(Filename, '_');
newStr = extractBetween(name,7,8);
if strcmp(newStr,'01')
DirOut = fullfile(DirIn, '01');
mkdir DirIn DirOut
movefile(fullfile(filelist(i).folder, filelist(i).name), DirOut);
end
end
This should work:
DirIn = 'C:\Folder\Experiment'; %set incoming directory
filelist=dir(fullfile(DirIn, '*.wav')); %get file list
DirOut = fullfile(DirIn, '01');
for i = 1:length(filelist);
Filename = filelist(i).name
newStr = Filename(7:8);
if strcmp(newStr,'01')
if ~exist(DirOut)
mkdir(DirOut)
end
movefile(fullfile(filelist(i).folder, filelist(i).name), DirOut);
end
end
Firstly, you don't need eval to get the file list. eval impact performance significantly. The below is what you should have done:
filelist=dir(fullfile(DirIn, '*.wav'));
You don't need strsplit or extractBetween since you only intend to extract a part of the string by indexing, i.e. the 7th and 8th characters, you may do this:
newStr = Filename(7:8);
To use variable as an input, you need to use mkdir as a function rather than console command:
mkdir(DirOut)
Lastly, a bit of optimisation. Since DirOut is constant, you can take it outside the loop. You may also want to check if DirOut has already been created to avoid the warning message and overhead in mkdir.
There is no issue with movefile.
Couple things go wrong, first, it is not recommended to use eval. In this case you can just create a character array to pass to dir as follows:
filelist = dir([DirIn '/*.wav'])
Then, you have a strplit that appears to do nothing, since it looks like your files don't have '_' in them, so name will just return Filename. But that is not the issue, since you are using extractBetween on the Filename.
The following does not what you think it does,
mkdir DirIn DirOut
will create two directories named DirIn and DirOut in the current working directory of Matlab. To create the directories you want, use:
mkdir(DirOut)
Since the output directory did not exist before, I suspect Matlab moved the file to the input directory, and renamed it to 01, if you manually add the extension .wav it should be one of the original files.
I am really a newbie in matlab programming. I have a problem in coding to import multiple csv files into one from certain folder:
This is my code:
%% Importing multiple CSV files
myDir = uigetdir; %gets directory
myFiles = dir(fullfile(myDir,'*.csv')); %gets all csv files in struct
for k = 1:length(myFiles)
data{k} = csvread(myFiles{k});
end
I use the code uigetdir in order to be able to select data from any folder, because I try to make an automation program so it would be flexible to use by others. The code that I run only look for the directory and shows the list, but not for merging the csv files into one and read it in "import data". I want it to be merged and read as one file.
My merged file should look like this with semicolon delimited and consist of 47 csv files merged together (this picture is one of the csv file I have):
my merged file
I have been working for it a whole day but I find always error code. Please help me :(. Thank you very much in advance for your help.
As the error message states, you're attempting to reference myFiles as a cell array when it is not. The output of dir is a structure, which cannot be indexed like a cell array.
You want to do something like the following:
for k = 1:numel(myFiles)
filepath = fullfile(myFiles(k).folder, myFiles(k).name);
data{k} = csvread(filepath);
end
I have a foreach loop that gets a list of objects in Folder4 after trimming the full path where the objects reside.
Here is sample code:
$row.Path = $path.InnerText.Replace("/Folder1/Folder2/folder3/folder4/","")
Sample Output:
usp_StoredProcedurename.prc,
fn_FunctionName.udf
File.sql
The last thing I need to do is to remove any extension, ie .prc, .pdf, .udf, .sql, etc
Here is the coplete for each:
You are probably looking for the static GetFileNameWithoutExtension method. To use it, you have to pass a single file or path to it:
[System.Io.Path]::GetFileNameWithoutExtension("usp_StoredProcedurename.prc")
Depending on the actual output of $row.Path you could split the path and join them back later if you want.
Alternative, you could use a regex to remove the file extensions for alle files within your string at once:
$row.Path -replace '\..*'
Be aware that regex will remove everything after a dot.
I want to read multiple files from a folder but this code does not work properly:
direction=dir('data');
for i=3:length(direction)
Fold_name=strcat('data\',direction(i).name);
filename = fullfile(Fold_name);
fileid= fopen(filename);
data = fread (fileid)';
end
I modified your algorithm to make it easier
Just use this form :
folder='address\datafolder\' ( provide your folder address where data is located)
then:
filenames=dir([folder,'*.txt']); ( whatever your data format is, you can specify it in case you have other files you do not want to import, in this example, i used .txt format files)
for k = 1 : numel(filenames)
Do your code
end
It should work. It's a much more efficient method, as it can apply to any folder without you worrying about names, number order etc... Unless you want to specify certain files with the same format within the folder. I would recommend you to use a separate folder to put your files in.
In case of getting access to all the files after reading:
direction=dir('data');
for i=3:length(direction)
Fold_name=strcat('data\',direction(i).name);
filename = fullfile(Fold_name);
fileid(i)= fopen(filename);
data{i-2} = fread (fileid(i))';
end
I'm trying to process a list of files that start with the same string, but only the .mat files. In my folder I have log files with names such as:
CADS3P5Ph1_LKS_20141210_EVAL_103443_001.avi
CADS3P5Ph1_LKS_20141210_EVAL_103443_001_MeasData.mat
CADS3P5Ph1_LKS_20141210_EVAL_103443_002.avi
CADS3P5Ph1_LKS_20141210_EVAL_103443_002_MeasData.mat
CADS3P5Ph1_LKS_20141210_EVAL_103443_003.avi
CADS3P5Ph1_LKS_20141210_EVAL_103443_003_MeasData.mat
CADS3P5Ph1_LKS_20141210_EVAL_104236_001.avi
CADS3P5Ph1_LKS_20141210_EVAL_104236_001_MeasData.mat
I only need to process the files that have the same timestamp (e.g. 103443_xxx)
I made a variable looking with a wildcard
filename = CADS3P5Ph1_LKS_20141210_EVAL_103443_001_MeasData.mat
general_name = filename(1:end - 17);
general_name = strcat(general_name,'*','');
So when I do dir(general_name), it finds all the files that start with "CADS3P5Ph1_LKS_20141210_EVAL_103443",
How do I only get the .mat files, and not the .avi files
I tried
dir(general_name && *.mat)
Is there a way to make something like this work?
Thanks!
Using strcat with general_name and the wildcard character for .mat extensions should work:
dir(strcat(general_name,'*.mat'))