Trouble Importing Zip Files with A Nested Do-Loop in SAS - import

I am attempting to import 104 zip files, each with approximately 30 excel files. I have created an excel file with all of the names of the zip files that I want to import (NYISO_Names). The first do-loop looks through all of the names of the zip files. Next, I created a list of the contents of each zip files. The second do-loop cycles through the contents.
So far, the code imports the first zip file (30 excel workbooks); however, it never moves on to the remaining zip files. Is something wrong with how the do-loops are nested?
%macro test;
%do i=1 %to 6;
%put ---&i.;
%let Iteration = &i.;
data abs;
set NYISO_Names;
if _n_ =&i. then call symput("FileName", strip(FileName));
if _n_ = &i. then call symput("Path", strip(Path));
run;
filename inzip zip "G:Desktop\&FileName.";
/* Read the "members" (files) from the ZIP file */
data Contents_&i.(keep=memname isFolder sheet);
length memname $200 isFolder 8;
fid=dopen("inzip");
if fid=0 then
stop;
memcount=dnum(fid);
do i=1 to memcount;
memname=dread(fid,i);
sheet = substr(memname, 1, length(memname)-4);
/* check for trailing / in folder name */
isFolder = (first(reverse(trim(memname)))='/');
output;
end;
rc=dclose(fid);
run;
DATA _NULL_; IF 0 THEN SET Contents_1 NOBS=X; CALL SYMPUT('RECCOUNT',X); STOP; RUN;
%do i = 1 %to &RECCOUNT;
data abs2;
set Contents_&Iteration.;
if _n_ =&i. then call symput("memname", strip(memname));
if _n_ =&i. then call symput("sheet", strip(sheet));
run;
/* identify a temp folder in the WORK directory */
filename xl "%sysfunc(getoption(work))/&memname." ;
/* hat tip: "data _null_" on SAS-L */
data _null_;
/* using member syntax here */
infile inzip(&memname.)
lrecl=256 recfm=F length=length eof=eof ;
file xl lrecl=256 recfm=N;
input;
put _infile_ $varying256. length;
return;
eof:
stop;
run;
proc import datafile=xl dbms=csv out=Test&i. replace;
run;
%end;
%end;
%mend;
%test();
Any help would be greatly appreciated.

Your nested %do loops are using the same index variable i
Try using distinct variables with semantic meaning to the problem at hand
%macro test;
…
%do iterationIndex = 1 %to 6;
…
%do referenceIndex = 1 %to &REFCOUNT;
…
%end;
%end;
…
%mend;
%test;

Related

Iterative %DO LOOP issue in SAS macro for automated emailing of site metrics

I have a dataset titled 'SampleReport' of site specific reporting metrics and am looking to email each site their own metrics in an attached PDF. The PDFs generate and are emailed out to their respective sites, however when trying to add ODS text to the PDF, I'm having issues defining my other macros (currently only 'email' is retained as a macro variable but I also want 'site', 'Site_Contact' and 'mcnt_enrolled').
Below is the test dataset and my code; I'm new to SAS and would appreciate any help/suggestions:
site site_enrolled_total mcnt_enrolled Site_Contact Email Expected_Enrolled NetworkAvg Overall_Study_Enrollment Overall_Enrollment_to_Date
site_a 32 7 Homer Simpson hsimp#gmail.com 40 5 115 1518
site_b 36 8 Jim Schoe jimmy2schoes#aol.com 40 4 115 1518
site_ c 20 2 Hank Hill propaneking#yahoo.com 36 7 115 1518
site_d 27 7 Lisa Simpson lisasimpson#gmail.com 36 5 115 1518
And here's my code:
%macro getemail(DATA);
ods _all_ close;
%let outdir = %sysfunc(getoption(WORK));
OPTIONS NODATE NONUMBER;
ODS GRAPHICS / RESET=ALL;
** get list of unique emails;
proc sql noprint;
select distinct(email) into :wantemail separated by '~'
from samplereport
order by email;
quit;
%put &=wantemail;
**count number of separators(i.e. '~') and count number of unique emails;
%let cntsep1 = %sysfunc(count(&wantemail,~));
%let cntemail = %eval(&cntsep1+1);
** start do loop to cycle thru list of emails;
** and create the site macro var for each loop;
%do i = 1 %to &cntemail;
%let email = %scan(&wantemail,&i,~);
%put This is for by group where email = &email;
**use site macro variable for file name and WHERE and TITLE;
ODS PDF (id=dagwood)
FILE="&outdir.\PDF_&email..PDF" STARTPAGE=NO STYLE=Journal gtitle;
title j=center bold "Metrics for January 2020";
ods text= "Email: &email. ";
ods text= "Site: &site. ";
ods text= "Site Contact: &Site_Contact.";
ods text= "Enrolled this Month: &mcnt_enrolled. ";
run;
ods layout gridded columns=2;
ods graphics / width=300 height=300;
ods region;
ods pdf(id=dagwood) style=statdoc;
PROC SGPLOT DATA=work.samplereport;
where email = "&email";
Title "Enrollment Metrics for Your Site";
vbar site / response=Expected_Enrolled stat=sum DATALABEL LEGENDLABEL="Expected Enrollment for Your Site"
discreteoffset=-0.5 barwidth=0.2 fillattrs=graphdata2;
yaxis grid display=(nolabel);
xaxis display=(noline NOTICKS NOVALUES nolabel);
vbar site / response=site_enrolled_total stat=sum DATALABEL LEGENDLABEL="Enrolled for Your Site to Date"
discreteoffset=-0.2 barwidth=0.2 fillattrs=graphdata1;
yaxis grid display=(nolabel);
xaxis display=(noline NOTICKS NOVALUES nolabel);
vbar site / response=mcnt_enrolled stat=sum DATALABEL LEGENDLABEL="Enrolled this Month"
discreteoffset=0.2 barwidth=0.2 fillattrs=graphdata3;
yaxis grid display=(nolabel);
xaxis display=(noline NOTICKS NOVALUES nolabel);
vbar site / response=NetworkAvg stat=sum DATALABEL LEGENDLABEL="Your Network Avg Enrollment this Month"
discreteoffset=0.5 barwidth=0.2 fillattrs=graphdata4;
yaxis grid display=(nolabel);
xaxis display=(noline NOTICKS NOVALUES nolabel);
RUN;
ods region;
ods pdf(id=dagwood) style=statdoc;
goptions reset=all;
proc sgplot data=work.samplereport;
where email = "&email";
Title "Study-Wide Enrollment to Date and Goal";
vbar site / response=Overall_Study_Enrollment stat=sum DATALABEL LEGENDLABEL="Enrollment for All Sites"
discreteoffset=-0.3 barwidth=0.4 fillattrs=graphdata5;
yaxis grid display=(nolabel);
xaxis display=(noline NOTICKS NOVALUES nolabel);
vbar site / response=Overall_Expected_Study stat=sum DATALABEL LEGENDLABEL="Expected Enrollment Goal"
discreteoffset=0.3 barwidth=0.4 fillattrs=graphdata6;
yaxis grid display=(nolabel);
xaxis display=(noline NOTICKS NOVALUES nolabel);
run;
ODS LAYOUT END;
%end;
* get emails from DATA;
proc sql noprint;
select EMAIL into :EMAIL1- from &DATA;
select count(*) into :EMAILN from &DATA;
* cycle through emails;
%do I=1 %to &EMAILN;
filename temp email to="&&EMAIL&I"
attach="&outdir.\PDF_&&email&I...PDF"
from="myemail#gmail.com"
type="text/html"
subject="Site Metrics";
ods html file=temp;
run;
ods html close;
%end;
%mend getemail;
%getemail(samplereport);
The email %DO loop statement is not properly terminated.
* cycle through emails;
%do I=1 %to &EMAILN
Should be
* cycle through emails;
%do I=1 %to &EMAILN;
And maybe the ODS HTML CLOSE should be outside the loop.

file read using fgetc adds FF at the end

I'm reading a file using fgetc. File reading starts at an offset.At the end I see 8'hFF getting appended at the end of file.I'm expecting 6 bytes in the file but see 7 in them.I'm not sure why this is happening. Any ideas?
Below is my code:
module file_read();
integer fd,fd1,file_char,status;
logic [7:0] captured_data;
initial begin
fd = $fopen("input_file", "rb");
fd1 =$fopen("write_file","w");
status=$fseek(fd,1872,0);
assert (status);
// while ($fgetc(fd) != `EOF) begin
while (!$feof(fd)) begin
file_char=$fgetc(fd);
$display("file char is %h",file_char);
end
end // initial begin
Below are the file contents(in hex):
last line of input_file(total file size =1878):
0000750: 0000 1567 d48d ...g..
write_file:
0000000: 0000 1567 d48d ff ...g...
Thanks!
The reason you are getting the extra 'hff at the end of the written file is due to how $feof (or generally the foef C function) works. Simply put, it doesn't check if the next read is off the end of the file but if the previous read read off the end of the file. So, if you are reading character by character (byte by byte) using $fgetc, $feof will only return true once $fgetc reads off the end of the file, and returns EOF itself (ie, -1 or 'hff if converted to logic [7:0]). You should be checking for this error condition each time you read a byte, something like this:
integer fd, ch;
...
fd = $fopen("file.bin", "rb");
...
while ((ch = $fgetc(fd)) != -1) begin
$display(ch);
end

How to adjust a variable via dividing it by the sum of observations in previous days?

I want to write a code to run the formula which is shown below:
Adjusted volume for each name and date = IntradayVolume / ( dailyvolume for five last days)
And I use codes which are shown below:
*Division for calculating adjusted volume;
proc sort data=sampledata_sumvolso02 out=sampledata_sumvolso02;
by TRD_STCK_CD TRD_EVENT_DT;
run;
*Adjusted intraday volume for days of a week;
data sampledata_adjvol02;
do until(last.TRD_STCK_CD);
do until(last.TRD_EVENT_DT);
set sampledata_sumvolso02;
by TRD_STCK_CD TRD_EVENT_DT;
if first.TRD_STCK_CD then
n=0;
if first.TRD_EVENT_DT then
n+1;
if n>7 then
do;
if not missing(IntradayVolume) then
adjusted_volume=divide(IntradayVolume,temp);
else call missing(adjusted_volume);
end;
if last.TRD_EVENT_DT then
temp=dailyvolume;
output;
end;
end;
drop temp n;
run;
Here is a sample of my data:
data WORK.SAMPLEDATA_SUMVOLSO02;
infile datalines dsd truncover;
input TRD_STCK_CD:$15. TRD_PR:32. TRD_TUROVR:14. TRD_EVENT_DT:DATE9. TRD_EVENT_TM:TIME5. TRD_EVENT_ROUNDED:32. TRD_EVENT_ROUFOR:$5. CountedVOLUME:32. DailyVolume:32. IntradayVolume:32.;
format TRD_TUROVR 14. TRD_EVENT_DT DATE9. TRD_EVENT_TM TIME5.;
label TRD_STCK_CD="TRD_STCK_CD" TRD_PR="TRD_PR" TRD_TUROVR="TRD_TUROVR" TRD_EVENT_DT="TRD_EVENT_DT";
datalines4;
BALI1,850,9260,24MAR2008,9:14,34200,9:30,7871000,,
BALI1,850,2000,23MAR2008,9:15,34200,9:30,1700000,,
BALI1,850,10000,22MAR2008,9:15,34200,9:30,8500000,,
BALI1,850,6000,21MAR2008,9:15,34200,9:30,5100000,,
BALI1,850,10000,20MAR2008,9:29,34200,9:30,8500000,31671000,31671000
BANK1,1164,10729,20MAR2008,9:38,36000,10:00,12488556,,12488556
BANK1,1148,2000,21MAR2008,11:24,41400,11:30,2296000,,
BANK1,1147,1575,22MAR2008,11:24,41400,11:30,1806525,16591081,4102525
BHMN1,1013,1500,14MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,15MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,16MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,17MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,18MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,19MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1000,20MAR2008,9:00,34200,9:30,1013000,,
BHMN1,1013,450,21MAR2008,9:00,34200,9:30,455850,,
BHMN1,1013,1500,22MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,23MAR2008,9:00,34200,9:30,1519500,,
BHMN1,1013,1500,24MAR2008,9:04,34200,9:30,1519500,,
BHMN1,1013,1500,25MAR2008,9:04,34200,9:30,1519500,,
BHMN1,1013,601,26MAR2008,9:05,34200,9:30,608813,,
BHMN1,1013,697,27MAR2008,9:06,34200,9:30,706061,,
BHMN1,1013,1500,28MAR2008,9:08,34200,9:30,1519500,,
BHMN1,1013,1500,29MAR2008,9:08,34200,9:30,1519500,,
BHMN1,1013,1500,12MAR2008,9:09,34200,9:30,1519500,,
BHMN1,1013,1500,13MAR2008,9:16,34200,9:30,1519500,,
BHMN1,1013,1500,14MAR2008,9:22,34200,9:30,1519500,,25576224
BHMN1,1013,1500,15MAR2008,9:41,36000,10:00,1519500,,1519500
BHMN1,1013,1500,16MAR2008,10:13,37800,10:30,1519500,,
BHMN1,1013,1500,17MAR2008,10:13,37800,10:30,1519500,,3039000
BHMN1,1013,1500,18MAR2008,11:25,41400,11:30,1519500,,1519500
BHMN1,1013,1500,19MAR2008,11:51,43200,12:00,1519500,33173724,1519500
CHML1,12000,745,20MAR2008,9:00,34200,9:30,8940000,,
CHML1,12000,82,21MAR2008,9:00,34200,9:30,984000,,
CHML1,11998,206,22MAR2008,9:07,34200,9:30,2471588,,
CHML1,11998,414,23MAR2008,9:09,34200,9:30,4967172,,
CHML1,11998,348,24MAR2008,9:12,34200,9:30,4175304,,
CHML1,11996,82,25MAR2008,9:15,34200,9:30,983672,,
CHML1,11996,1240,26MAR2008,9:16,34200,9:30,14875040,,
CHML1,11996,414,27MAR2008,9:17,34200,9:30,4966344,,
CHML1,11970,1000,28MAR2008,9:21,34200,9:30,11970000,,
CHML1,11964,158,29MAR2008,9:22,34200,9:30,1890312,,
CHML1,11990,1946,20MAR2008,9:23,34200,9:30,23332540,,79555972
CHML1,11851,500,19MAR2008,10:42,39600,11:00,5925500,,
CHML1,11852,1054,18MAR2008,10:44,39600,11:00,12492008,,
CHML1,11900,2000,17MAR2008,10:56,39600,11:00,23800000,,42217508
CHML1,11913,244,16MAR2008,11:08,41400,11:30,2906772,,
CHML1,11913,56,15MAR2008,11:09,41400,11:30,667128,,
CHML1,11968,944,14MAR2008,11:13,41400,11:30,11297792,,
CHML1,11968,2056,13MAR2008,11:27,41400,11:30,24606208,,
CHML1,11987,380,14MAR2008,11:29,41400,11:30,4555060,,44032960
CHML1,11988,620,11MAR2008,11:30,43200,12:00,7432560,,
CHML1,11981,1663,12MAR2008,11:52,43200,12:00,19924403,,
CHML1,11981,765,24MAR2008,11:52,43200,12:00,9165465,202328868,36522428
CRBN1,1486,1700,22MAR2008,9:00,34200,9:30,2526200,,
CRBN1,1485,3353,24MAR2008,9:21,34200,9:30,4979205,7505405,7505405
DADE1,1685,2972,24MAR2008,10:04,37800,10:30,5007820,,5007820
DADE1,1632,2000,20MAR2008,11:49,43200,12:00,3264000,,
DADE1,1631,5000,21MAR2008,11:49,43200,12:00,8155000,,
DADE1,1630,3000,22MAR2008,11:49,43200,12:00,4890000,,
DADE1,1630,2000,23MAR2008,11:50,43200,12:00,3260000,,
DADE1,1630,8000,24MAR2008,11:50,43200,12:00,13040000,37616820,32609000
DFRB1,7450,124,24MAR2008,10:54,39600,11:00,923800,,923800
DFRB1,7450,132,24MAR2008,11:16,41400,11:30,983400,1907200,983400
DJBR1,7491,3000,24MAR2008,10:06,37800,10:30,22473000,22473000,22473000
DRZK1,14890,72,24MAR2008,11:58,43200,12:00,1072080,1072080,1072080
FIBR1,2846,100,24MAR2008,11:17,41400,11:30,284600,284600,284600
FKAS1,3584,5173,20MAR2008,9:15,34200,9:30,18540032,,
FKAS1,3584,3000,21MAR2008,9:15,34200,9:30,10752000,,
FKAS1,3585,1827,22MAR2008,9:15,34200,9:30,6549795,,35841827
FKAS1,3585,183,23MAR2008,9:35,36000,10:00,656055,,
FKAS1,3585,3200,24MAR2008,9:35,36000,10:00,11472000,,12128055
FKAS1,3539,2000,25MAR2008,11:02,41400,11:30,7078000,,
FKAS1,3538,8000,26MAR2008,11:02,41400,11:30,28304000,,
FKAS1,3538,2000,27MAR2008,11:02,41400,11:30,7076000,,
FKAS1,3537,8000,28MAR2008,11:02,41400,11:30,28296000,,
FKAS1,3537,2000,29MAR2008,11:02,41400,11:30,7074000,,
FKAS1,3535,8000,20MAR2008,11:02,41400,11:30,28280000,,
FKAS1,3535,2000,19MAR2008,11:03,41400,11:30,7070000,,
FKAS1,3535,6617,18MAR2008,11:03,41400,11:30,23391095,,
FKAS1,3533,100,17MAR2008,11:03,41400,11:30,353300,,
FKAS1,3533,1283,16MAR2008,11:07,41400,11:30,4532839,,
FKAS1,3533,2763,15MAR2008,11:08,41400,11:30,9761679,,
FKAS1,3529,10000,14MAR2008,11:15,41400,11:30,35290000,,
FKAS1,3516,10000,13MAR2008,11:17,41400,11:30,35160000,,
FKAS1,3516,10000,12MAR2008,11:19,41400,11:30,35160000,,
FKAS1,3516,39,11MAR2008,11:19,41400,11:30,137124,,
FKAS1,3515,2000,19MAR2008,11:25,41400,11:30,7030000,,
FKAS1,3515,198,24MAR2008,11:25,41400,11:30,695970,,
FKAS1,3515,9802,20MAR2008,11:25,41400,11:30,34454030,,299144037
FKAS1,3515,4000,21MAR2008,11:48,43200,12:00,14060000,,
FKAS1,3513,6000,22MAR2008,11:52,43200,12:00,21078000,,
FKAS1,3514,2000,23MAR2008,11:52,43200,12:00,7028000,,
FKAS1,3514,1085,24MAR2008,11:52,43200,12:00,3812690,,
FKAS1,3500,1000,25MAR2008,11:56,43200,12:00,3500000,396592609,49478690
FOLD1,3198,390,24MAR2008,9:09,34200,9:30,1247220,,1247220
FOLD1,3197,2000,27MAR2008,10:31,39600,11:00,6394000,,
FOLD1,3191,2000,28MAR2008,10:32,39600,11:00,6382000,,
FOLD1,3190,4000,20MAR2008,10:33,39600,11:00,12760000,,
FOLD1,3190,542,18MAR2008,10:33,39600,11:00,1728980,,
;;;;
Now, I want to divide Intradayvolume variable by aggregated Dailyvolume variable in a week before today. It is important to specify for SAS to diagnoses days of a week and adapts it with the date variable in my dataset. Because the format of the date variable of my dataset is ddmmyy. I mean SAS should find which days of a week are today and aggregates Dailyvolume variable of a week before today as the denominator.
How can I run that formula?
Thanks in advance.
I'm not sure exactly what you're doing there, but one approach might be to create a temporary array and index it with the date. Then look up in that index. You'd have some issues with that simplistic of a method with holidays and such, but it should give you at least an approximately right answer.
For example (again, I don't entirely understand your variables or what you're doing, so this is an approximate idea):
*Adjusted intraday volume for days of a week;
data sampledata_adjvol02;
array prevdat[32767] _temporary_;
do until(last.TRD_STCK_CD);
do until(last.TRD_EVENT_DT);
set sampledata_sumvolso02;
by TRD_STCK_CD TRD_EVENT_DT;
if first.TRD_STCK_CD then
n=0;
if first.TRD_EVENT_DT then
n+1;
if n>7 then
do;
if not missing(IntradayVolume) then
adjusted_volume=divide(IntradayVolume,prevdat[TRD_EVENT_DT-7]);
else call missing(adjusted_volume);
end;
if last.TRD_EVENT_DT then
temp=dailyvolume;
output;
prevdat[TRD_EVENT_DT] = sum(prevdat[TRD_EVENT_DT],countedVolume);
end;
end;
call missing(of prevdat[*]);
drop temp n;
run;
data sampledata_adjvol (drop=cnt dv_:);
set Sampledata_SumVolSo;
by TRD_STCK_CD TRD_EVENT_DT;
retain dv_1 dv_2 dv_3 dv_4 cnt 0 dv_sum;
if first.TRD_STCK_CD then do;
cnt=0;
dv_1=0;
dv_2=0;
dv_3=0;
dv_4=0;
dv_sum=0;
end;
adjusted_volume_5=intradayvolume/dv_sum;
if last.TRD_EVENT_DT then do;
cnt=cnt+1;
if cnt=1 then dv_4=dailyvolume;
else if cnt=2 then dv_3=dailyvolume;
else if cnt=3 then dv_2=dailyvolume;
else if cnt=4 then dv_1=dailyvolume;
else do;
dv_sum=dailyvolume+dv_1+dv_2+dv_3+dv_4;
dv_4=dv_3;
dv_3=dv_2;
dv_2=dv_1;
dv_1=dailyvolume;
end;
end;
run;

Error: required operator not found in Macro: SAS

I'm working on this macro (see below) and this statement is not resolving : %if (&var_len > &&max&i) %then %let max&i=&var_len; and the error that is returned is: ERROR: Required operator not found in expression: (&var_len > &&max&i)
ERROR: The macro PRCS_FREQ will stop executing
Does anyone have any idea on how I can improve this macro and to get this statement to resolve?
Macro:
%macro prcs_freq;
%do i=1 %to &n_of_var;
%global end&i f_line_count&i f_var&i;
%let max&i=1;
%let k=%eval(&i+1);
%if (&i < &n_of_var) %then %let end&i=%eval(&&cum&k-1);
%else %let end&i=&n_of_line;
%let f=%eval(&&cum&i+1); /* line number for "Frequency" */
%let freq_pos&i=%index(%bquote(&&line&f),Frequency); /* column position of "Frequency" */
%let f_var&i=%upcase(%scan(%bquote(&&line&f),1,%str( )));
%let var_len=%length(&&f_var&i);
%let max&i=&var_len;
%global &&f_var&i;
%let &&f_var&i=&i; /* for Frequency */
%do j=&f+2 %to &&end&i;
%let var_len=%length(%substr(%bquote(&&line&j),1,&&freq_pos&i-1));
%if (&var_len > &&max&i) %then %let max&i=&var_len;
%put *** f=&f i=&i j=&j var_len=&var_len freq_pos&i=&&freq_pos&i
max&i=&&max&i f_var&i=&&f_var&i;
%end; /* %do j= */
%let f_line_count&i=%eval(&&end&i-&&cum&i+1);
%end; /* %do i= */
data _null_;
%do i=1 %to &n_of_var;
file "&wk_dir/&&f_var&i...txt";
%do j=&&cum&i %to &&end&i;
put "%substr(%bquote(&&line&j),1,&&max&i+5)%substr(%bquote (&&line&j),&&freq_pos&i)";
%end;
%end;
run;
/* test */
%put =====;
%put ===== Frequency;
%put =====;
%do i=1 %to &n_of_var;
%put f_var&i=&&f_var&i cum&i=&&cum&i end&i=&&end&i f_line_count&i=&&f_line_count&i &&f_var&i=&&&&&&f_var&i;
%end;
%mend prcs_freq;
The symbolgen log shows that your macro variable &VAR_LEN resolves to: 10) . That unmatched parenthesis causes the problem, e.g.:
188 %macro ck;
189 %if 10) > 0 %then %put true;
190 %mend;
191 %ck;
ERROR: Required operator not found in expression: 10) > 0
ERROR: The macro CK will stop executing
The open parenthesis is the missing operator referred to in the error. I think the source of the extra unmatched parenthesis is this line:
%let var_len=%length(%substr(%bquote(&&line&j),1,&&freq_pos&i-1));
I think &&line&j is resolving to a value that has an unmatched close parenthesis. %BQUOTE would quote (mask) the parenthesis. But %substr() would unquote it. And then the %length function would see the close parenthesis as the end of the call to %length. Then the close parenthesis you mean to be the end of the call to %length becomes the extra parenthesis. If you change to %qsubstr() it should quote the parenthesis, and work appropriately.
Below is a test macro, illustrating what I think is the problem:
%macro test(line= ) ;
%put &=line ;
%let bad_var_len =%length( %substr(%bquote(&line),1,12)) ;
%let good_var_len=%length(%qsubstr(%bquote(&line),1,12)) ;
%put &=bad_var_len ;
%put &=good_var_len ;
%if &bad_var_len>0 %then %put true ;
%mend test;
Here is the log from running it:
219 %test(line=%str(Hi mom%) how are you?))
LINE=Hi mom) how are you?
BAD_VAR_LEN=6 how )
GOOD_VAR_LEN=12
ERROR: Required operator not found in expression: &bad_var_len>0
ERROR: The macro TEST will stop executing.
Doing complex calculations in macro code is frustrating and hard to debug. Since your macro is already generating a data step I suggest that you just replace a lot of the macro calculations with data step code. You can use CALL SYMPUTX() to generate any new macro variables that you need for later steps of the macro.

Octave: Load all files from specific directory

I used to have Matlab and loaded all txt-files from directory "C:\folder\" into Matlab with the following code:
myFolder = 'C:\folder\';
filepattern = fullfile(myFolder, '*.txt');
files = dir(filepattern);
for i=1:length(files)
eval(['load ' myFolder,files(i).name ' -ascii']);
end
If C:\folder\ contains A.txt, B.txt, C.txt, I would then have matrices A, B and C in the workspace.
The code doesn't work in octave, maybe because of "fullfile"?. Anyway, with the following code I get matrices with the names C__folder_A, C__folder_B, C__folder_C. However, I need matrices called A, B, C.
myFolder = 'C:\folder\';
files = dir(myFolder);
for i=3:length(files)
eval(['load ' myFolder,files(i).name ' -ascii']);
end
Can you help me?
Thanks,
Martin
PS: The loop starts with 3 because files(1).name = . and files(2).name = ..
EDIT:
I have just found a solution. It's not elegant, but it works.
I just add the path in which the files are with "addpath", then I don't have to give the full name of the directory in the loop.
myFolder = 'C:\folder\';
addpath(myFolder)
files = dir(myFolder);
for i=3:length(files)
eval(['load ' files(i).name ' -ascii']);
end
It's usually bad design if you load files to variables which name is generated dynamically and you should load them to a cell array instead but this should work:
files = glob('C:\folder\*.txt')
for i=1:numel(files)
[~, name] = fileparts (files{i});
eval(sprintf('%s = load("%s", "-ascii");', name, files{i}));
endfor
The function scanFiles searches file names with extensions in the current dirrectory (initialPath) and subdirectories recursively. The parameter fileHandler is a function that you can use to process populated file structure (i.e. read text, load image, etc.)
Source
function scanFiles(initialPath, extensions, fileHandler)
persistent total = 0;
persistent depth = 0; depth++;
initialDir = dir(initialPath);
printf('Scanning the directory %s ...\n', initialPath);
for idx = 1 : length(initialDir)
curDir = initialDir(idx);
curPath = strcat(curDir.folder, '\', curDir.name);
if regexp(curDir.name, "(?!(\\.\\.?)).*") * curDir.isdir
scanFiles(curPath, extensions, fileHandler);
elseif regexp(curDir.name, cstrcat("\\.(?i:)(?:", extensions, ")$"))
total++;
file = struct("name",curDir.name,
"path",curPath,
"parent",regexp(curDir.folder,'[^\\\/]*$','match'),
"bytes",curDir.bytes);
fileHandler(file);
endif
end
if!(--depth)
printf('Total number of files:%d\n', total);
total=0;
endif
endfunction
Usage
# txt
# textFileHandlerFunc=#(file)fprintf('%s',fileread(file.path));
# scanFiles("E:\\Examples\\project\\", "txt", textFileHandlerFunc);
# images
# imageFileHandlerFunc=#(file)imread(file.path);
# scanFiles("E:\\Examples\\project\\datasets\\", "jpg|png", imageFileHandlerFunc);
# list files
fileHandlerFunc=#(file)fprintf('path=%s\nname=%s\nsize=%d bytes\nparent=%s\n\n',
file.path,file.name,file.bytes,file.parent);
scanFiles("E:\\Examples\\project\\", "txt", fileHandlerFunc);