I am looking to have a programme which cleans up some messy data I have, I am looking to do this for both the assets and liabilities side of the project i'm working on.
My question is there a way to use a do loop to use the cleaning up data to first clean up the assets then liabilities. something like this:
%do %I = Asset %to Liability;
%assetorliability= I ;
proc sort data = &assetorliability;
by price;
run;
data want&assetorliability;
set &assetorliability;
if _N_ < 50000;
run;
the actual script is quite long so a singular macro may not be the ideal solution but this loop would be great.
TIA.
EDIT : the programme includes some macros and the errors received are as follows:
%let list =Asset Liability;
%do i=1 %to %sysfunc(countw(&list,%str( )));
%let next=%scan(&list,&i,%str( ));
%Balance;
%end;
in the macro the data steps are named with a balance&list to allow for each scenario. the errors are:
13221 %let list =Asset Liability;
13222 %do i=1 %to %sysfunc(countw(&list,%str( )));
ERROR: The %DO statement is not valid in open code.
13223
13224 %let next=%scan(&list,&i,%str( ));
WARNING: Apparent symbolic reference I not resolved.
WARNING: Apparent symbolic reference I not resolved.
ERROR: A character operand was found in the %EVAL function or %IF condition where a numeric
operand is required. The condition was: &i
ERROR: Argument 2 to macro function %SCAN is not a number.
ERROR: The %END statement is not valid in open code.
The macro %do statement is not as flexible as the data step do statement. To loop over a list of values you would want to put the list into a macro variable and use an index variable in your %do loop.
Note that macro logic needs to be inside of a macro. You cannot use it in "open" code.
%macro do_over(list);
%local i next;
%do i=1 %to %sysfunc(countw(&list,%str( )));
%let next=%scan(&list,&i,%str( ));
proc sort data = &next ;
by price;
run;
data want&next ;
...
%end;
%mend do_over ;
%do_over(Asset Liability)
Related
I am trying to compare two numberic value in a Macro.
But I keep getting the following message:
ERROR: A character operand was found in the %EVAL function or %IF condition where a numeric operand is required. The condition was: 0.2
ERROR: The %TO value of the %DO I loop is invalid.
ERROR: A character operand was found in the %EVAL function or %IF condition where a numeric operand is required. The condition was: 0.05
ERROR: The %BY value of the %DO I loop is invalid.
ERROR: The macro FAIL will stop executing.
My code is the following:
%macro fail;
%do i=0 %to 0.2 %by 0.05;
data failcrs;
set fail;
if f_p>=input(&i, 8.) then output;
run;
%end;
%mend failcrs;
f_p is a numeric variable.
What is wrong with my code? Please help.
Thank you so much!
Conditional tests in macro code (%if, %until, %while, etc) use %eval() macro function that only does integer arithmetic. This includes the increments and tests done in a %do-%to-%by loop.
To use floating point arithmetic you need to use the %sysvalf() macro function.
You could code your own increments to the loop counter.
%let i=0;
%do %while( %sysevalf( &I <= 0.2 ) );
...
%let i=%sysevalf(&i + 0.05);
%end;
Or make the loop counter an integer and use another macro variable to hold the fraction.
%do j=0 %to 20 %by 5 ;
%let i=%sysevalf(&j/100);
...
%end;
You have a couple of issues.
Macro loops work better with integers, but an easy workaround is a %DO %UNTIL loop instead.
Name on %MEND is different than on %MACRO
Invalid values for %DO %I loop.
Non-unique data set name, which means the output overwrites itself.
*fake data to work with;
data fail;
do f_p=0 to 0.2 by 0.01;
output;
end;
run;
%macro fail;
%let i=0;
%do %until(&i = 0.2); /*2*/
data failcrs_%sysevalf(&i*100); /*3*/
set fail;
if f_p>=&i then output;
run;
%let i = %sysevalf(&i + 0.05);
%end;
%mend fail; /*3*/
*test macro;
%fail;
The numbers in the comments align with the issues identified.
Try using best32. But why do you want to loop, when your dataset is overwritten for each loop. Please check log for each of step below. As at #Reeza in comments explains below you even do not an input statement
Options mprint;
/* do this*/
%macro fail;
%let i =15;
data failcrs;
set sashelp.class;
if age lt input(&i, best32.) then output;
run;
%mend fail;
%fail
/* dataset overwritten every time to finally pick up 15 as value check in the log*/
%macro fail1;
%do i = 1 %to 15;
data failcrs1;
set sashelp.class;
if age lt input(i, best32.) then output;
run;
%end;
%mend fail1;
%fail1
%macro fail;
%let i=0;
%do %until(&i = 0.2);
data failcrs;
set crse_grade_dist_fail;
if f_p>=&i then output;
run;
proc sql;
create table count_failclass as
select strm, count(class_nbr) as numfclass_%sysevalf(&i*100)
from failcrs
group by strm;
quit;
proc sql;
create table failfaculty as
select strm, instructor_id, instructor, count(class_nbr) as numfclass
from failcrs
group by strm, instructor_id, instructor;
quit;
proc sql;
create table count_failfaculty as
select strm, count(instructor) as numffaculty_%sysevalf(&i*100)
from failfaculty
group by strm;
quit;
data count_class_faculty;
set count_class_faculty;
set count_failclass;
set count_failfaculty;
run;
%let i = %sysevalf(&i + 0.05);
%end;
%mend fail;
Good thing is my data doesn't have f_p=0, all of them is greater than zero. Because I only want to count failed courses.
Documentation is written to be read, a simple search for "SAS 9.4 Macro Do" should explain it all -- start, stop and by are integers -- integers in the sense that whatever macro source expression in their place evaluates implicitly or explicitly to an integer at need time.
The macro you coded is a little strange. It will generate multiple data steps that all overwrite the same dataset. You might want to concentrate on not writing macro code first, and move to it when the need to have repetitive boilerplate code submitted. Writing good macro code means you have to think in terms of "will this generate appropriate source code and what side effect will these macro statements have in their resolution scope"
%DO, Iterative Statement
Syntax
%DO macro-variable=start %TO stop <%BY increment>;
text and macro language statements
%END;
Required Arguments
macro-variable
names a macro variable or a text expression that generates a macro
variable name. Its value functions as an index that determines the
number of times the %DO loop iterates. If the macro variable specified
as the index does not exist, the macro processor creates it in the
local symbol table.
You can change the value of the
index variable during processing. For example, using conditional
processing to set the value of the index variable beyond the stop
value when a certain condition is met ends processing of the loop.
startstop
specify integers or macro expressions that generate integers
to control the number of times the portion of the macro between the
iterative %DO and %END statements is processed.
The first time the
%DO group iterates, macro-variable is equal to start. As processing
continues, the value of macro-variable changes by the value of
increment until the value of macro-variable is outside the range of
integers included by start and stop.
increment
specifies an integer
(other than 0) or a macro expression that generates an integer to be
added to the value of the index variable in each iteration of the
loop. By default, increment is 1. Increment is evaluated before the
first iteration of the loop. Therefore, you cannot change it as the
loop iterates.
I'm new to SAS Macro programming and need to enable the following macro to be able to handle and process multiple values for its macro parameters.Hello,
data have;
input name $ ACCOUNT_ID $ cust_id;
cards;
ARTHUR CC1234 1234
TOM eil1235 1235
MIKEZ tb1236 1236
MATT mb1237 1237
LIZ TB1238 1238
PIZ VB1239 1239
TAN MB1240 1240
PANDA . 1241
;
run;
%MACRO algo (IN_DS=,VAR_LIST=,DATA_TYPE_LIST=,OUT_DS=);
DATA &OUT_DS;
SET &IN_DS;
%If &data_type_LIST = num %then
&var_LIST=sum(&VAR_LIST,2);
%else &var_LIST=cats(&var_LIST,'re');;
run;
%mend;
%algo(IN_DS=HAVE,VAR_LIST=CUST_ID,DATA_TYPE_LIST=num,OUT_DS=out1);`
I now need to enable this macro to be able to pass multiple values for the macro parameters. Something like this :
%algo(IN_DS=HAVE,VAR_LIST='CUST_ID,ACCT_ID',DATA_TYPE_LIST='num,char',OUT_DS=out1);
Can someone help me enable this functionality in the macro code.
The parameter argument should be macro quoted with %STR() in the macro invocation.
Try
%algo
( IN_DS=HAVE
, VAR_LIST= %STR (CUST_ID, ACCT_ID)
, DATA_TYPE_LIST=num
, OUT_DS=out1
);
Macro quoting is different than DATA step quoting used for character literals.
Make sure the macro can handle multiple values. In general it is not a good idea to use comma as the delimiter in your list of values when calling a macro.
Usually space is the best delimiter since then you can use the macro value directly in the generated code. For example if your variables are all of the same type you can just use data step ARRAY.
%MACRO algo (IN_DS=,VAR_LIST=,DATA_TYPE_LIST=,OUT_DS=);
DATA &OUT_DS;
SET &IN_DS;
array list &var_list ;
do _n_=1 to dim(list);
%if &data_type_LIST = num %then %do ;
list(_n_)=sum(list(_n_),2);
%end;
%else %do;
list(_n_)=cats(list(_n_),'re');
%end;
end;
run;
%mend algo;
If your variables are NOT all of the same type then you need to generate a separate statement for each variable. In that case you can use a different delimiter if you want, like a pipe character, that is easier to use as delimiter in calls to macro functions like %scan().
%MACRO algo (IN_DS=,VAR_LIST=,DATA_TYPE_LIST=,OUT_DS=);
%local i var;
DATA &OUT_DS;
SET &IN_DS;
%do i=1 %to %sysfunc(countw(&var_list,|));
%let var=%scan(&var_list,&i,|);
%if %scan(&data_type_LIST,&i,|) = num %then %do ;
&var=sum(&var,2);
%end;
%else %do;
&var=cats(&var,'re');
%end;
%end;
run;
%mend algo;
%algo(IN_DS=HAVE,VAR_LIST=CUST_ID|ACCT_ID,DATA_TYPE_LIST=num|char,OUT_DS=out1);
If you want to pass a list of variables and then use that list in the code you posted above, my suggestion would be to treat the &var_list as a list, and use scan to determine how many variables there are, and then loop through the list and execute the code accordingly.
I am trying to rename the variables based on one array elements in the folloing way,
%let var= class name gender;
data want;
set have;
%global noof;
array point(*)$ %str(&var) ;
a=dim(point);
call symputx('noof',a);
array newvar(&noof);
do i=1 to &noof;
newvar(i)=translate(point(i),',','.');
end;
drop &var;
do i=1 to &noof;
rename newvar(i)=vname(point(i));
end;
run;
I want to rename the new variables to the first array elemets.
LOG:
rename newvar(i)=vname(point(i));
-
22
76
ERROR 22-322: Syntax error, expecting one of the following: -, :, =.
ERROR 76-322: Syntax error, statement will be ignored.
Unfortunately, the value on the RHS of the RENAME statement must be a literal. The statement is evaluated at compile time, not run time.
Try this:
%let var= class name gender;
%macro translate(datain,dataout,vars);
%local n i var;
%let n=%sysfunc(countw(&vars));
data &dataout(rename=(
%do i=1 %to &n;
%let var = %scan(&vars,&i);
newvar&i = &var
%end;
));
set &datain;
array invars(&n) $ &vars ;
array newvar(&n) $;
do i=1 to &n;
newvar(i)=translate(invars(i),',','.');
end;
drop &vars i;
run;
%mend;
data test;
class = "1,2,3";
name= "Dom,Pazzula";
gender="M";
run;
%translate(test,out,&var);
You can run into issues if the length of these character variables are too large. The new variables might be truncated. You will have to modify this to add a length statement.
I'm trying to run some code which will hopefully concatenate multiple months or years worth of data. I am trying to figure out when a field was populated with a value. I.e. there is field XYZ in my data set and it is populated with value A in November 2016. If I run my code from Jan - Dec I would like a new field populated with the date that SAS encounters a non-blank value in that field.
Here's my code:
options mprint symbolgen source mlogic merror syntaxcheck ;
%macro append_monthly(iStart_date=, iEnd_date=);
%local tmp_date i;
%let tmp_date = %sysfunc(intnx(month,&iStart_date,0,beginning)) ;
%do %while (&tmp_date le &iEnd_date);
%let i = %sysfunc(sum(&tmp_date),yymmn4.);
%put &i.;
%let tmp_date = %sysfunc(intnx(month,&tmp_date,1,beginning)) ;
libname note "my.qualifiers.fords.note&i." disp=shr;
data new ;
set note.file ;
%if ln_note_crbur_date_delinq ne '' %then spc_cmt_date = &i.;
run;
%end;
%mend;
%append_monthly(iStart_date=%sysfunc(mdy(5,1,2016)), iEnd_date=%sysfunc(mdy(10,1,2016)) );
LIBNAME _ALL_ CLEAR;
Here's a sample from log with errors :
SYMBOLGEN: Macro variable TMP_DATE resolves to 20606
SYMBOLGEN: Macro variable IEND_DATE resolves to 20728
MLOGIC(APPEND_MONTHLY): %DO %WHILE(&tmp_date le &iEnd_date) condition is TRUE; loop will iterate again.
MLOGIC(APPEND_MONTHLY): %LET (variable name is I)
SYMBOLGEN: Macro variable TMP_DATE resolves to 20606
MLOGIC(APPEND_MONTHLY): %PUT &i.
SYMBOLGEN: Macro variable I resolves to 1606
1606
MLOGIC(APPEND_MONTHLY): %LET (variable name is TMP_DATE)
SYMBOLGEN: Macro variable TMP_DATE resolves to 20606
MPRINT(APPEND_MONTHLY): spc_cmt_date = 1605 run;
SYMBOLGEN: Macro variable I resolves to 1606
MPRINT(APPEND_MONTHLY): libname note "my.qualifiers.fords.note1606" disp=shr;
ERROR: Unable to clear or re-assign the library NOTE because it is still in use.
ERROR: Error in the LIBNAME statement.
NOTE: The SAS System stopped processing this step because of errors.
WARNING: The data set WORK.NEW may be incomplete. When this step was stopped there were 0 observations and 622 variables.
WARNING: Data set WORK.NEW was not replaced because this step was stopped.
NOTE: The DATA statement used 0.01 CPU seconds and 49483K.
NOTE: The address space has used a maximum of 4292K below the line and 240388K above the line.
I can't figure out why this isn't working. Maybe this could work using Proc append.
Basically, I just want my output with a field that returns a date in the form of YYMM for when field ln_note_crbur_date_delinq was non-blank.
Any help would be greatly appreciated
I'd guess the reason for your error is that the handle is not being cleared on your source file before the next libname statement tries to re-assign.
An easy fix would be to use a different alias (libref) each time, as follows:
libname note&i "my.qualifiers.fords.note&i." disp=shr;
Then adjust your data step like so:
data new ;
set note&i..file ;
The next part appears to be confusion between macro logic and data step. Simply remove the % symbols as follows:
if ln_note_crbur_date_delinq ne '' then spc_cmt_date = &i.;
Finally, add a proc append before the %end as follows:
proc append base=work.final data=new; run;
If work.final does not exist, it will be created in the same format as new.
EDIT:
following discussion in comments, here is a revised approach:
%macro append_monthly(iStart_date=, iEnd_date=);
%local tmp_date i set_statement;
%let tmp_date = %sysfunc(intnx(month,&iStart_date,0,beginning)) ;
%do %while (&tmp_date le &iEnd_date);
%let i = %sysfunc(sum(&tmp_date),yymmn4.);
%let tmp_date = %sysfunc(intnx(month,&tmp_date,1,beginning)) ;
%let set_statement=&set_statement &i..file;
libname note&i "my.qualifiers.fords.note&i." disp=shr;
%end;
data new ;
set &set_statement;
if ln_note_crbur_date_delinq ne '' then spc_cmt_date = &i.;
run;
%mend;
%append_monthly(iStart_date=%sysfunc(mdy(5,1,2016)), iEnd_date=%sysfunc(mdy(10,1,2016)) );
LIBNAME _ALL_ CLEAR;
I started to learn %macro in SAS and now I'm trying to implement simple bootstrap with histogram as an output.
/*Create K data sets(vectors)*/
%macro datasets(K);
%do i=1 %to &K;
data indata&i;
%do j = 1 %to 50;
x=(rand('normal',2,9));
output;
%end;
run;
%end;
%mend datasets;
%datasets(3);
/*Bootstrap and hist*/
%macro boot (data,res);
%do i=1 %to &res;
%let x = (sample(&data,50));
%let m = (mean(&x));
%end;
proc iml;
read &m into A;
create DataM from A;
append from A;
close Data1;
quit;
proc univariate data=Data1;
histogram m;
run;
%mend boot;
%boot(Indata1,100);
It doesn't work and I can't understand why. Can you point me the mistake?
Use PROC SURVEYSELECT to generate bootstrap samples then do bootstrap analysis by Replication (a variable created by SURVEYSELECT). Your macro idea will be far too slow.
As mentioned use Proc SurveySelect and Proc Means. You can select all 100 samples in one Proc SurveySelect and then apply Proc Means with a BY statement to calculate the means in one step. Macro's don't add anything to the solution here.
I'm posting both solutions - the macro solution does take longer as well.
*Without macro;
proc surveyselect data=indata1 out=rsample method=srs n=50 reps=100;
run;
proc means data=rsample noprint;
by replicate;
var x;
output out=Data1 mean(x)=m;
run;
proc univariate data=Data1;
histogram m;
run;
*Macro solution;
%macro boot(data, res);
%do i=1 %to &res;
%*Currently pulls the same sample every time but you can fix that part;
proc surveyselect data=&data out=x method=srs n=50 reps=1 seed=343434;
run;
proc means data=x noprint;
var x;
output out=m mean(x)=m;
run;
proc append base=DataM data=m;
run;
%end;
%mend;
%boot(Indata1,10);
Perhaps it will help if we outline some of the ways that the posted macro code does NOT work. If nothing else then as examples of things to avoid.
If the first macro , %datasets(), you are using a macro %DO loop where you should use a normal data step DO loop. Also make sure to define your local macro variables as local. This will prevent the macro from modifying the value of an existing macro variable with the same name.
/*Create K data sets(vectors)*/
%macro datasets(K);
%local i ;
%do i=1 %to &K;
data indata&i;
do j = 1 to 50;
x=(rand('normal',2,9));
output;
end;
drop j;
run;
%end;
%mend datasets;
In the second macro you have a %DO loop that does nothing.
%do i=1 %to &res;
%let x = (sample(&data,50));
%let m = (mean(&x));
%end;
You repeat the exact same %LET statements multiple times. The result does not change since the loop variable i is not referenced at all. If you called the macro with data=indata1 then the result of the two statements will be that X=(sample(indata1,50)) and that M=(mean((sample(indata1,50)))). I think that perhaps you intended that the strings sample and mean might take some action, but since they have no macro triggers (& or %) they are just streams of characters to the macro processor.
I am not an expert on IML, but those statements also do not look like they are doing much.