SAS IML use of Mattrib with Macro (symget) in a loop - macros

In an IML proc I have several martices and several vectors with the names of columns:
proc IML;
mydata1 = {1 2 3, 2 3 4};
mydata2 = {1 2, 2 3};
names1 = {'red' 'green' 'blue'};
names2 = {'black' 'white'};
To assign column names to columns in matrices one can copypaste the mattrib statement enough times:
/* mattrib mydata1 colname=names1;*/
/* mattrib mydata2 colname=names2;*/
However, in my case the number of matrices is defined at execution, thus a do loop is needed. The following code
varNumb=2;
do idx=1 to varNumb;
call symputx ('mydataX', cat('mydata',idx));
call symputx ('namesX', cat('names',idx));
mattrib (symget('mydataX')) colname=(symget('namesX'));
end;
print (mydata1[,'red']) (mydata2[,'white']);
quit;
however produces the "Expecting a name" error on the first symget.
Similar question Loop over names in SAS-IML? offers the macro workaround with symget, what produces an error here.
What is the correct way of using mattrib with symget? Is there other way of making a variable from a string than macro?
Any help would be appreciated.
Thanks,
Alex
EDIT1
The problem is in the symget function. The &-sign resolves the name of the matrix contained in the macro variable, the symget only returns the name of the macro.
proc IML;
mydata1 = {1 2 3};
call symputx ('mydataX', 'mydata1');
mydataNew = (symget('mydataX'));
print (&mydataX);
print (symget("mydataX"));
print mydataNew;
quit;
results in
mydata1 :
1 2 3
mydata1
mydataNew :
mydata1
Any ideas?
EDIT2
Function value solves the symget problem in EDIT1
mydataNew = value(symget('mydataX'));
print (&mydataX);
print (value(symget("mydataX")));
print mydataNew;
The mattrib issue but persists.
SOLVED
Thanks Rick, you have opened my eyes to CALL EXECUTE() statement.

When you use CALL SYMPUTX, you should not use quotes for the second argument. Your statement
call symputx ('mydataX', 'mydata1');
assigns the string 'mydata1' to the macro variable.
In general, trying to use macro variables in SAS/IML loops often results in complicated code. See the article Macros and loops in the SAS/IML language for an indication of the issues caused by trying to combine a macro preprocessor with an interactive language. Because the MATTRIB statement expects a literal value for the matrix name, I recomend that you use CALL EXECUTE rather than macro substitution to execute the MATTRIB statement.
You are also having problems because a macro variable is always a scalar string, whereas the column name is a vector of strings. Use the ROWCAT function to concatenate the vector of names into a single string.
The following statements accomplish your objective without using macro variables:
/* Use CALL EXECUTE to set matrix attributes dynamically.
Requires that matrixName and varNames be defined at main scope */
start SetMattrib;
cmd = "mattrib " + matrixName + " colname={" + varNames + "};";
*print cmd; /* for debugging */
call execute(cmd);
finish;
varNumb=2;
do idx=1 to varNumb;
matrixName = cat('mydata',idx);
varNames = rowcat( value(cat('names',idx)) + " " );
run SetMattrib;
end;

Related

Calling macro from within generated function in Julia

I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4

julia metaprogramming and nloops variable evaluation

I am a noob at metaprogramming so maybe I am not understanding this. I thought the purpose of the #nloops macro in Base.Cartesian was to make it possible to code an arbitrary number of nested for loops, in circumstances where the dimension is unknown a priori. In the documentation for the module, the following example is given:
#nloops 3 i A begin
s += #nref 3 A i
end
which evaluates to
for i_3 = 1:size(A,3)
for i_2 = 1:size(A,2)
for i_1 = 1:size(A,1)
s += A[i_1,i_2,i_3]
end
end
end
Here, the number 3 is known a priori. For my purposes, however, and for the purposes that I thought nloops was created, the number of nested levels is not known ahead of time. So I would not be able to hard code the integer 3. Even in the documentation, it is stated:
The (basic) syntax of #nloops is as follows:
The first argument must be an integer (not a variable) specifying the number of loops.
...
If I assign an integer value - say the dimension of an array that is passed to a function - to some variable, the nloops macro no longer works:
b = 3
#nloops b i A begin
s += #nref b A i
end
This returns an error:
ERROR: LoadError: MethodError: no method matching _nloops(::Symbol, ::Symbol, ::Symbol, ::Expr)
Closest candidates are:
_nloops(::Int64, ::Symbol, ::Symbol, ::Expr...) at cartesian.jl:43
...
I don't know how to have nloops evaluate the b variable as an integer rather than a symbol. I have looked at the documentation and tried various iterations of eval and other functions and macros but it is either interpreted as a symbol or an Expr. What is the correct, julian way to write this?
See supplying the number of expressions:
julia> A = rand(4, 4, 3) # 3D array (Array{Int, 3})
A generated function is kinda like a macro, in that the resulting expression is not returned, but compiled and executed on invocation/call, it also sees the type (and their type parameters of course) of the arguments, ie:
inside the generated function, A is Array{T, N}, not the value of the array.
so T is Int and N is 3!
Here inside the quoted expression, N is interpolated into the expression, with the syntax $N, which evaluates to 3:
julia> #generated function mysum(A::Array{T,N}) where {T,N}
quote
s = zero(T)
#nloops $N i A begin
s += #nref $N A i
end
s
end
end
mysum (generic function with 1 method)
julia> mysum(A)
23.2791638775186
You could construct the expression and then evaluate it, ie.:
julia> s = 0; n = 3;
julia> _3loops = quote
#nloops $n i A begin
global s += #nref $n A i
end
end
quote
#nloops 3 i A begin
global s += #nref(3, A, i)
end
end
julia> eval(_3loops)
julia> s
23.2791638775186
I have scrubbed manually the LineNumberNodes from the AST for readability (there is also MacroTools.prettify, that does it for you).
Running this example in the REPL needs to declare s as global inside the loop in Julia 1.0.

Creating a function with variable number of inputs?

I am trying to define the following function in MATLAB:
file = #(var1,var2,var3,var4) ['var1=' num2str(var1) 'var2=' num2str(var2) 'var3=' num2str(var3) 'var4=' num2str(var4)'];
However, I want the function to expand as I add more parameters; if I wanted to add the variable vark, I want the function to be:
file = #(var1,var2,var3,var4,vark) ['var1=' num2str(var1) 'var2=' num2str(var2) 'var3=' num2str(var3) 'var4=' num2str(var4) 'vark=' num2str(vark)'];
Is there a systematic way to do this?
Use fprintf with varargin for this:
f = #(varargin) fprintf('var%i= %i\n', [(1:numel(varargin));[varargin{:}]])
f(5,6,7,88)
var1= 5
var2= 6
var3= 7
var4= 88
The format I've used is: 'var%i= %i\n'. This means it will first write var then %i says it should input an integer. Thereafter it should write = followed by a new number: %i and a newline \n.
It will choose the integer in odd positions for var%i and integers in the even positions for the actual number. Since the linear index in MATLAB goes column for column we place the vector [1 2 3 4 5 ...] on top, and the content of the variable in the second row.
By the way: If you actually want it on the format you specified in the question, skip the \n:
f = #(varargin) fprintf('var%i= %i', [(1:numel(varargin));[varargin{:}]])
f(6,12,3,15,5553)
var1= 6var2= 12var3= 3var4= 15var5= 5553
Also, you can change the second %i to floats (%f), doubles (%d) etc.
If you want to use actual variable names var1, var2, var3, ... in your input then I can only say one thing: Don't! It's a horrible idea. Use cells, structs, or anything else than numbered variable names.
Just to be crytsal clear: Don't use the output from this in MATLAB in combination with eval! eval is evil. The Mathworks actually warns you about this in the official documentation!
How about calling the function as many times as the number of parameters? I wrote this considering the specific form of the character string returned by your function where k is assumed to be the index of the 'kth' variable to be entered. Array var can be the list of your numeric parameters.
file=#(var,i)[strcat('var',num2str(i),'=') num2str(var) ];
var=[2,3,4,5];
str='';
for i=1:length(var);
str=strcat(str,file(var(i),i));
end
If you want a function to accept a flexible number of input arguments, you need varargin.
In case you want the final string to be composed of the names of your variables as in your workspace, I found no way, since you need varargin and then it looks impossible. But if you are fine with having var1, var2 in your string, you can define this function and then use it:
function str = strgen(varargin)
str = '';
for ii = 1:numel(varargin);
str = sprintf('%s var%d = %s', str, ii, num2str(varargin{ii}));
end
str = str(2:end); % to remove the initial blank space
It is also compatible with strings. Testing it:
% A = pi;
% B = 'Hello!';
strgen(A, B)
ans =
var1 = 3.1416 var2 = Hello!

SAS IML pass reference to variable defined in macro

In SAS/IML I try to pass to a user defined module a reference to a variable that is defined in macro. The module changes the variable value.
Since call of the function is in the do-loop I cannot use &-sign. However use of symget does not work. Here is my code.
proc iml;
start funcReference(argOut);
print "funcReference " argOut;
argOut = 5;
finish funcReference;
store module=funcReference;
quit;
proc IML;
mydata1 = {1 2 3};
call symput ('macVar', 'mydata1');
load module=funcReference;
run funcReference(symget('macVar'));
print mydata1;
quit;
The output shows that variable mydata1 have not changed:
argOut
funcReference mydata1
mydata1
1 2 3
Any ideas?
SOLVED
Thanks a lot!
You are sending in a temporary scalar matrix (the result of the SYMGET call). That temporary variable is being updated and promptly vanishes, as explained in the article "Oh, those pesky temporary variables!"
Instead of macro variables (which are text strings), you should use the VALUE and VALSET functions, as described in the article "Indirect assignment: How to create and use matrices named x1, x2,..., xn" You need to send in a real matrix in order for the values to be updated correctly, as follows:
proc IML;
load module=funcReference;
mydata1 = {1 2 3};
call symput('macVar', 'mydata1');
matrixName = symget('macVar'); /* matrix named &mydata1 */
z = value(matrixName); /* z contains data */
run funcReference(z); /* update values in z */
call valset(matrixName, z); /* update data in &mydata1 */
print mydata1;

Loop through values of a SPSS variable inside of a Macro

How can I pass the values of a specific variable to a list-processing loop inside a macro?
Let's say, as an simplified example, I've got a variable foo which contains the values 1,4,12,33 and 51.
DATA LIST FREE / foo (F2) .
BEGIN DATA
1
4
12
33
51
END DATA.
And a macro that does some stuff with those values.
For testing reasons this Macro will just echo those values.
I'd like to find a way to run a routine that works like the following:
DEFINE !testmacro (list !CMDEND)
!DO !i !IN (!list)
ECHO !QUOTE(!i).
!DOEND.
!ENDDEFINE.
!testmacro list = 1 4 12 33 51. * <- = values from foo.
This is a situation where using the Python apis would be a good choice.
I made myself a little bit familiar with Python recently :-)
So this is what I worked out.
If the variable is a numeric:
BEGIN PROGRAM PYTHON.
import spss,spssdata
foolist = [element[0] for element in spssdata.Spssdata('foo').fetchall()]
foostring = " ".join(str(int(i)) for i in foolist)
spss.Submit("!testmacro list = %(foostring)s." %locals())
END PROGRAM.
If the variable is a string:
BEGIN PROGRAM PYTHON.
import spss,spssdata
foolist = [element[0].strip() for element in spssdata.Spssdata('bar').fetchall()]
foostring = " ".join(foolist)
spss.Submit("!testmacro list = %(foostring)s." %locals())
END PROGRAM.
Variants
Duplicates removed and list is orderd
BEGIN PROGRAM PYTHON.
import spss,spssdata
foolist = sorted(set([element[0] for element in spssdata.Spssdata('foo').fetchall()]))
foostring = " ".join(str(int(i)) for i in foolist)
spss.Submit("!testmacro list = %(foostring)s." %locals())
END PROGRAM.
Duplicates removed and items in order of first appearance in the dataset
Here, I use a function which I retrieved from Peter Bengtsson's Homepage (peterbe.com)
BEGIN PROGRAM PYTHON.
import spss,spssdata
def uniquify(seq, idfun=None):
# order preserving
if idfun is None:
def idfun(x): return x
seen = {}
result = []
for item in seq:
marker = idfun(item)
if marker in seen: continue
seen[marker] = 1
result.append(item)
return result
foolist = uniquify([element[0] for element in spssdata.Spssdata('foo').fetchall()])
foostring = " ".join(str(int(i)) for i in foolist)
spss.Submit("!testmacro list = %(foostring)s." %locals())
END PROGRAM.
Non-Python Solution
Not that I recommend it, but there is even a way to do this without Python.
I got the basic Idea from a SPSS programming book, which goes as follows:
Use the WRITE command to create a text file with the wanted command and variable values and include it with the insert command.
DATASET COPY foolistdata.
DATASET ACTIVATE foolistdata.
AGGREGATE OUTFILE=* MODE=ADDVARIABLES
/BREAK
/NumberOfCases=N.
* Variable which contains the command as string in the first case.
STRING macrocommand (A18).
IF ($casenum=1) macroCommand = "!testmacro list = ".
EXECUTE.
* variable which contains a period (.) in the last case,
* for the ending of the command string.
STRING commandEnd (A1).
IF ($casenum=NumberOfCases) commandEnd = ".".
* Write the 'table' with the command and variable values into a textfile.
WRITE OUTFILE="macrocommand.txt" /macrocommand bar commandEnd.
EXECUTE.
* Macrocall.
INSERT FILE ="macrocommand.txt".