I cannot run my PROC SQL function by calling the macro in my Data step.
The SQL function alone works, but I need to let it run for every Security Group.
%macro adding;
proc sql;
insert into have (Time,seconds) values
("9:10:00"t,33000);
insert into have (Time,seconds) values
("16:50:00"t,60600);
quit;
%mend;
data have;
set have;
by security;
if first.security then %adding;
if seconds=33000 then date=lag(date);
if seconds=60600 then date=lag(date);
run;
The error is:
1 proc sql; insert into have (Time,seconds)
values
---- ------
180 180 180 180 1 ! ("9:10:00"t,33000); insert into have (Time,seconds) values 1 !
("16:50:00"t,60600); quit; ERROR 180-322: Statement is
not valid or it is used out of proper order.
I don't know what to change that I can use it...
Thankful for any help! Best
Use call execute to call the macro.
If first.security then call execute('%adding');
However, the macro will run AFTER the data step, not during.
Also, trying to change the data in place that many ways could lead to difficulties in debugging. Your DATA, SET, and SQL all reference the same data set.
If you're trying to change the data in your proc and add records you may want to consider using explicit OUTPUT statements within the data step itself. You could use a macro to generate these statements if desired.
If first.security then do;
Time=...;
Seconds=....;
Output;
Time=....;
Seconds=....;
Output;
End;
*rest of SAS code;
Output; Add an explicit output if required;
Run;
You also shouldn't be calculating a lagged value conditionally, as the lag is a queue. You'll get unexpected behaviour. I haven't highlighted all the issues with your process but this should be enough to help you find the rest.
Related
New here, so if I did something wrong, I apologize. I'm new user of SAS as well.
I created a macro that calls first a proc sql that creates a certain table that I want to pass it to another macro (inside the first macro).
%Macro Mc_Copy_Table (TABLE_NAME);
proc sql;
create table &TABLE_NAME as
select *
from OR_IN.&TABLE_NAME;
connect using OR_OUT;
execute (truncate table &TABLE_NAME) By OR_OUT;
disconnect from OR_OUT;
quit;
%MC_obsnvars(&TABLE_NAME);
%put &Nobs;
%if &Nobs > 100000 %then
%do; /* use of the sql loader */
proc append base = OR_OU. &TABLE_NAME (&BULKLOAD_OPTION)
data = &TABLE_NAME;
run;
%end;
%else
%do;
proc append base = OR_OU. &TABLE_NAME (Insertbuff=10000)
data = &TABLE_NAME;
run;
%end;
%Mend Mc_Copy_Table;
The Mc_Obsnvars macro use the attrn function to get the number of observations from the given dataset (it opens the dataset first). Depending on the number of observations, my program either use the sqlloader or not. OR_IN and OR_OUT are libnames (oracle engine).
When The macro Mc_Copy_Table is executed, with let's say TABLE1 as argument, the Mc_Obsnvars is executed first which tries to open TABLE1 which doesn't exist yet. The proc sql is executed afterwards.
Why the macro is executed before the proc sql ? and is there any way to have the proc sql be executed first ? putting the proc sql part in a macro doesn't solve the problem. Thanks :)
I think you have a syntax issue, as Quentin alludes to in his comment. This works OK for me:
%macro copy_table(intable, outtable);
proc sql noprint;
create table &outtable as
select * from &intable;
%count_obs(&outtable);
%put NOBS:&nobs;
quit;
%mend;
%macro count_obs(table);
%global nobs;
select count(*) into :nobs trimmed from &table;
%mend;
data test;
do i=1 to 10;
output;
end;
run;
%copy_table(test,test2);
Note however, you don't have to do the count. There is an automatic variable from PROC SQL called &sqlobs with the number of records returned from the last query.
So this gives you what you are looking for, I think:
%macro copy_table(intable, outtable);
proc sql noprint;
create table &outtable as
select * from &intable
where i < 5;
%let nobs=&sqlobs;
%put NOBS:&nobs;
quit;
%mend;
%copy_table(test,test2);
I need to break down this SAS macro that adds suffixes to some number of variables into pseudocode, but there are some parts of it I don't fully understand.
%macro add_suffix(lib,dsn, suffix);
options pageno=1 nodate;
OPTIONS OBS= 1;
DATA GRIDWORK.TMP;
SET &lib..&dsn.;
RUN;
proc sql noprint;
select nvar into :num_vars
from dictionary.tables
where libname="GRIDWORK" and
memname="TMP";
select distinct(name) into :var1-
:var%TRIM(%LEFT(&num_vars))
from dictionary.columns
where libname="GRIDWORK" and
memname="TMP";
quit;
run;
OPTIONS OBS= MAX;
proc datasets library=&LIB;
modify &DSN;
rename
%do i=1 %to &num_vars;
&&var&i=&&var&i..&suffix
%end;
;
quit;
run;
proc datasets library=&LIB;
modify &DSN;
rename pers_gen_key&suffix = pers_gen_key;
quit;
run;
proc sql;
drop table gridwork.tmp;
quit;
%mend add_suffix;
1) In this part of the code:
DATA GRIDWORK.TMP;
SET &lib..&dsn.;
RUN;
How can you have setting a dataset equal to two values? Is it setting GRIDWORK.TMP to the concatenation of &lib and &dsn? What exactly do the multiple periods mean here?
2) I understand that this section is storing variables in an array:
proc sql noprint;
select nvar into :num_vars
from dictionary.tables
where libname="GRIDWORK" and
memname="TMP";
select distinct(name) into :var1-
:var%TRIM(%LEFT(&num_vars))
from dictionary.columns
where libname="GRIDWORK" and
memname="TMP";
quit;
How exactly do dictionary.tables and dictionary.columns work, and how do they differ from eachother in this context? Here is the documentation, I read through it but am still having trouble understanding what exactly is going on in this section of the code.
3) Towards the end of the macro we have:
OPTIONS OBS= MAX;
proc datasets library=&LIB;
modify &DSN;
rename
%do i=1 %to &num_vars;
&&var&i=&&var&i..&suffix
%end;
;
quit;
run;
Here is the documentation for the proc datasets procedure. It says it names the library that the procedure processes. Does this mean that &dsn is part of the &lib library? I guess I am unsure of how libraries work in SAS. Are they built in, or user-defined? Why are they necessary, couldn't we just modify &DSN on its own?
SAS has two level references, library name and then data set name. The first macro variable points to the library and the second to the data set name. A period tells the macro processor where the macro variable ends and the second period is used to separate the libname from the data set name.
Its not storing in arrays, its creating macro variables. The Dictionary tables are metadata about your tables. I would recommend actually looking at them. The difference between the tables is that TABLES has information on your dataset and COLUMNS has information on the variables in each table.
A library is simply a directory/folder where SAS datasets are stored. This allows SAS to reference different directories to save files, and allows users to implement organizational systems on their data. &dsn is a data set in the &lib folder.
I highly recommend you look into the %put statement and place it in various parts of the code to see exactly what the code is doing.
I'm trying to convert a string column which is in sybase in the below format into SAS date.
The sybase table has string values like this
2015-04-23 04:04:46.517
2015-04-22 04:04:35.162
2015-04-21 04:04:43.646
I need to get the max of these values and store it in a max_tmsp variable and get the records where last_updt_tmsp > max_tmsp.
I referred to this link and tried to write some code but it is not working.
All this code is in Precode before the job starts.
proc sql noprint;
SELECT
select max(input(PROPERTY_VAL, MDYAMPMw.d)) into :last_updt_tmsp
from sybase_lib.prop_vals where property_key='last.update.date';
quit;
format &last_updt_tmsp. DATETIME18.;
data _null_;
call symput('lst_cre_dttm',"'"||"&last_updt_tmsp."||"'dt");
run;
%put lst_cre_dttm=&lst_cre_dttm
you can do this in a data step, try the following:
data datetime;
format new_date datetime24.3;
a="2015-04-23 04:04:46.517";
new_date=input(a, anydtdtm24.);
run;
Using proc sql you can try:
proc sql;
select max(input(a,anydtdtm24.)) format datetime24.3 into: max_date
from table1;
quit;
%put &max_date;
the point to remember is max of a character variable will not give you consistent results as compared to max of a numeric variable. You want the latter.
Putting together answer and comment about the answer
Data HAVE;
length PROPERTY_VAL $23;
Input PROPERTY_VAL $ 1-23;
Datalines;
2015-04-23 04:04:46.517
2015-04-22 04:04:35.162
2015-04-21 04:04:43.646
;
Run;
proc sql noprint;
select max(input(PROPERTY_VAL, anydtdtm24.)),
max(input(PROPERTY_VAL, anydtdtm24.)) format=datetime22.3
into :last_updt_tmsp, :last_updt_tmsp_f
from HAVE;
quit;
%Put LAST_UPDT_TMSP: &last_updt_tmsp (&last_updt_tmsp_f);
DCR and Carolina thank you very much. It worked. Can you please tell me the difference between max(input(PROPERTY_VAL, anydtdtm24.)) format=datetime22.3 and max(input(a,anydtdtm24.)) format datetime24.3 .
DCR i will keep your point in mind and let my team know and see if we can switch to the last_updt_dt column
Also is it possible to see what value is stored in &last_updt_tmsp in the log
I took carolina explanation though i'm sure DCR yours works too .. I did not change the bottom part
proc sql noprint;
SELECT
select max(input(PROPERTY_VAL, anydtdtm24.)) format=datetime22.3 into :last_updt_tmsp
from sybase_lib.prop_vals where property_key='last.update.date';
quit;
format &last_updt_tmsp. DATETIME18.;
data null;
call symput('lst_cre_dttm',"'"||"&last_updt_tmsp."||"'dt");
run;
%put lst_cre_dttm=&lst_cre_dttm
I am inserting a lot of measurement data from different sources in a postgres database. The data are a measured value and an uncertainty (and a lot of auxilliary data) The problem is that in some cases I get an absolute error value eg 123 +/- 33, in other cases, I get a relative error as a percentage of the measured value, eg 123 +/- 10%. I would like to store all the measurements with absolute error, i.e the latter should be stored as 123 +/- 12.3 - (at this point, I don't care too much about the valid number of digits)
My idea is to use a trigger to do this. Basically, if the error is numeric, store it as is, if it is non-numeric, check if the last character is '%', in that case, multiply it with the measured value, divide by 100 and store the resulting value. I got an isnumeric-function from here: isnumeric() with PostgreSQL which works fine. But when I try to make this into a trigger, it seems as if the input is checked for validity even before the trigger fires, so that the insert is abortet before I get any possibility to do anything with the values.
my triggerfunction: (need to do the calculation, just setting the error to 0 here...
create function my_trigger_function()
returns trigger as'
begin
if not isnumeric(new.err) THEN
new.err=0;
end if;
return new;
end' language 'plpgsql';
then I connect it to the table:
create trigger test_trigger
before insert on test
for each row
execute procedure my_trigger_function();
Doing this, I would expect to get val=123 and err=0 for the following insert
insert into test(val,err) values(123,'10%');
but the insert fails with "invalid input syntax for type numeric" which then must be triggered before my trigger gets any possibility to see the data (or I have misunderstood something basic). Is it possible to make the new.err data-type agnostic or can I run the trigger even earlier or is what I want to do just plain impossible?
It's not possible with a trigger because the SQL parser fails before.
When the trigger is launched, the NEW.* columns already have their definitive types matching the destination columns.
The closest alternative is to provide a function converting from text to numeric implementing your custom syntax rules and apply it in the VALUES clause:
insert into test(val,err) values(123, custom_convert('10%'));
Daniel answered on my original question - and I found out I had to think otherwise. His proposal for how to do it may work for others, but the way my system interfaces to the database by fetching table and column names directly from the database, it would not work well.
Instead I added a boolean field relerr to the measurement table
alter table measure add relerr boolean default false;
Then I made a trigger that checks if relerr is true - indicating that I am trying to store a relative error, if so, it recalculates the error column (called prec for precision)
CREATE FUNCTION calc_fromrel_error()
RETURNS trigger as'
IF NEW.relerr THEN
NEW.prec=NEW.prec*NEW.value/100;
NEW.relerr=FALSE;
END IF;
return NEW;
END' language 'plpgsql';
and then
create trigger meas_calc_relerr_trigger
before update on measure
for each row
execute procedure calc_fromrel_error();
voila, by doing a
INSERT into measure (value,prec,relerr) values(220,10,true);
I get the table populated with 220,22,false. Inserted values should normally never be updated, if that for some strange reason should happen, I will be able to calculate the prec column manually.
I am using entity framework with a stored procedure, in which I am generating query dynamically and executing that query. The stored proc query looks like:
Begin
DECLARE #Query nvarchar(MAX)
SET #Query = 'SELECT e.id, e.name, e.add, e.phno from employee'
EXEC sp_executesql #Query
End
In above sql code you can see that i am executing '#Query' variable, and that variable value can be changed dynamically.
I am able to add my stored proc in my edmx file. and then I go to model browser and say Add function import and try to Get column information it does not show anything. but when I execute my stored proc at server it returns all columns with values. Why i am not getting column information at model browser?
The model browser isn't running the stored procedure to then gather the column information from its result - it's trying to grab the column information from the underlying procedure definition using the sys tables.
This procedure, because it's dynamic, will not have an underlying definition and therefore won't be importable into the EDMX like this.
Temporarily change your stored proc to
SELECT TOP 1 e.id, e.name, e.add, e.phno from employee /* ... rest of your proc commented out */
Then add it to the EF and create the function import (it will see the columns).
Then change the proc back to how you had it above.
Try adding SET NOCOUNT ON after BEGIN.... that supresses messages that might cause it to be "confused"