Problems importing data in Stata using "file" and a local macro - import

I'm attempting to import data from a number of log files. Here's how I'm approaching it:
I have the log files in a separate directory and I generate a txt file with the list of filenames in that directory. I then read through that txt file and subsequently use the filenames in a loop with an import command to bring the data in. This process worked when I last run the do file about 6 months ago and now it's not working.
The basics of the code look like this:
cd "filepath\logs\"
! dir cpuusage*.log /a-d /b /o:-d >"filepath\filelist.txt"
file open myfile using "filepath\filelist.txt",read
file read myfile line
import delimited using `line', delim(" ") varnames(nonames)
The result of that import command is (0 vars, 0 obs) despite the fact that filelist.txt has a list of 14 filenames.
I'm a novice so I'm really hoping there's something simple and obvious that I'm overlooking. I still don't understand why this exact method worked six months ago... Any thoughts?

I think you can use fs for this:
cd "filepath\logs\"
fs cpuusage*.log
foreach f in `r(files)' {
import delimited using `f', delim(" ") varnames(nonames)
}
It's hard to help more without knowing what your log files look like. I would suggest trying to import one using the menu to get the syntax right.

Related

How to recode missing genotype code is " '-' " in the ped file of plink

I'm trying to impute genotype data from the public reference panels but my files fail the file sanity check on Sanger Imputation server and it gives the following error:
failed sanity check :
of Non-ACGTN alternate allele at 1:4635556 .. REF_SEQ:'(null)' vs VCF:'-'
I have tried fixing this in the plink with the following command ./plink --bfile chr1 --recode vcf --out chr1_vcf --missing-genotype -
but then it gives error Underscore(s) present in sample IDs.
--recode vcf to chr1_vcf.vcf ... done.
but I still see '_' in the new coded file.
I would appreciate any help, suggestions and comments.
Thanks
Jasdeep
You will have to replace _ with a different character in your PLINK files before running your code.
See below from PLINK manual
When using --recode vcf, sample IDs are formed by merging the FID and IID and placing an underscore between them. When the FID or IID already contains an underscore, this may make it difficult to reconstruct them from the VCF file; you may want to replace underscores with a different character in PLINK files (Unix tr is handy here).

Unzipping files in SAS using 7zip

I´m currently trying to unzip an excel file using 7zip in SAS.
I´ve done some looking around and I´ve managed to put this together although I get the error message "7-Zip: Cannot find archive"
%let UNZIP = C:\Users\maz\Outputfile;
%let CDRIVE = C:\Users\maz\Zip File\TodayFile.zip;
data _null_;
X "cd C:\Program Files\7-Zip";
X "7zG e &CDRIVE. -o&UNZIP.";
run;
Doing some research tells me the folder does not exist, but I know it does. Also, some sources use 7za but I only have 7zG. Any ideas on what to look at next or what is going on?
This is very likely due to the space in 'Zip File'. Try putting quotes around the path name. You can use a double double-quote in a string to represent a single double-quote(!), like this:
X "7zG e ""&CDRIVE"" -o&UNZIP";
X cd "C:\Program Files\7-Zip";
Not really a SAS question. You need to follow the rules of the OS for a path with blanks.

Error code in importing multiple csv files from certain folder using matlab

I am really a newbie in matlab programming. I have a problem in coding to import multiple csv files into one from certain folder:
This is my code:
%% Importing multiple CSV files
myDir = uigetdir; %gets directory
myFiles = dir(fullfile(myDir,'*.csv')); %gets all csv files in struct
for k = 1:length(myFiles)
data{k} = csvread(myFiles{k});
end
I use the code uigetdir in order to be able to select data from any folder, because I try to make an automation program so it would be flexible to use by others. The code that I run only look for the directory and shows the list, but not for merging the csv files into one and read it in "import data". I want it to be merged and read as one file.
My merged file should look like this with semicolon delimited and consist of 47 csv files merged together (this picture is one of the csv file I have):
my merged file
I have been working for it a whole day but I find always error code. Please help me :(. Thank you very much in advance for your help.
As the error message states, you're attempting to reference myFiles as a cell array when it is not. The output of dir is a structure, which cannot be indexed like a cell array.
You want to do something like the following:
for k = 1:numel(myFiles)
filepath = fullfile(myFiles(k).folder, myFiles(k).name);
data{k} = csvread(filepath);
end

Seperate and process odd named text files from an even named text files in Python

I am new to programming & python and is trying to write a program to process astronomical data.I have a huge list of files naming like ww_12m_no0021.spc, ww_12m_no0022.spc and so on. I want to move all the odd numbered files and even numbered files in two seperate folders.
import shutil
import os
for file in os.listdir("/Users/asifrasha/Desktop/python_test/input"):
if os.path.splitext(file) [1] == ".spc":
print file
shutil.copy(file, os.path.join("/Users/asifrasha/Desktop/python_test/output",file))
which is actually copying all the spc file to a different folder. I am struggling a bit on how I can only copy the odd number files (no0021, no0023…) to a seperate folder. Any help or suggestions will be much appreciated!!!
import os
import shutil
# Modify these to your need
odd_dir = "/Users/asifrasha/Desktop/python_test/output/odd"
even_dir = "/Users/asifrasha/Desktop/python_test/output/even"
for filename in os.listdir("/Users/asifrasha/Desktop/python_test/input"):
basename, extenstion = os.path.splitext(filename)
if extenstion == ".spc":
num = basename[-4:] # Get the numbers (i.e. the last 4 characters)
num = int(num, 10) # Convert to int (base 10)
if num % 2: # Odd
dest_dir = odd_dir
else: # Even
dest_dir = even_dir
dest = os.path.join(dest_dir, filename)
shutil.copy(filename, dest)
Obviously you can simplify it a bit; I'm just trying to be as clear as possible.
Assuming your files are named ww_12m_no followed by the number:
if int(os.splitext(file)[0][9:])%2==1:
#file is oddly numbered, go ahead and copy...
If the length of the first half of the name changes, I would use regex... I didn't test the code, but that's the gist of it. I'm not sure this question belongs here though...

Batch script to show multiple filenames in one call

Im trying to create a batch script to call a .exe to carry out analysis on multiple data files in a same folder. The syntax should be like:
"D:\Softwares\Analyzer.exe" [DataFile1].dat [DataFile2].dat ... [DataFileN].dat AnalysisFile.pdo"
Currently I have tried to use a FOR loop to scan each *.dat file in a specified folder. (I don't know how many data files in that folder, so I cannot type the filenames directly in the command line)
For example:
#ECHO OFF
FOR /r %%i in (*.dat) DO (
"D:\Softwares\Analyzer.exe" %%~ni.dat TestAnalysis.pdo
)
PAUSE
However, the analysis is carried out on seperate datafiles, and the .exe file will pop-up and open every time when a new .dat file is detected. Is there any way I could use *.dat or any other methods to represent [DataFile1].dat [DataFile2].dat ... [DataFileN].dat in one line seperated by a space (not a new line)?
I have also tried to use #tilte, which does not work as well. Since the .exe window keep pop-up whenever a new .dat file is detected and I have to close each of them in order to continue to next .dat file.
In general, I would like to do an automatic scan in a folder, get the names of the datafiles, and write a command line to call these .dat files in one line.
Any ideas/helps appreciated!!!
try this, remove the word echo if the output is OK:
#echo off &setlocal enabledelayedexpansion
set "line="
for %%i in (*.dat) do set "line=!line! "%%i""
echo "D:\Softwares\Analyzer.exe" %line% TestAnalysis.pdo
The code doesn't work with *dat files with exclams ! in the file name. This may be fixed if needed.