Julia - How macros are related (based on exemple from OpenCL.jl - macros

I've put a discord but no result. May be here somebody could help me
When you look in the OpenCL implementation code you have two macros that seems to refer one another in some (misterious) way, I cannot see how this happens. One macro is #check define in src/ macros.jl the other one #ocl_func define in api.jl.
#check is responsible for running the clfunc => $(esc(clfunc)), but as far as I understood it is the #ocl_func macro (in the api.jl file) that will ccall the clfunc.
My question how does the 'called' clfunc, for example this one:
#check api.clGetPlatformIDs(0, C_NULL, nplatforms) (in /platforms.jl)
is passed to the #ocl_func macro?
Is it implicit( because the ocl macro is define in api so every function called with api.will activate the macro, in the example it will be then:
#ocl_func(clGetPlatformIDs(0, C_NULL, nplatforms)) )
I cannot find the code which glue the two macros. Clarifying this will help me to understand how macros work.

An easy way to think about macros is that they are source-code generation tools.
as far as I understood it is the #ocl_func macro (in the api.jl file) that will ccall the clfunc.
If you look at the the implementation of #ocl_func, you'll see that the ccall is part of a quote block, that's being returned. So it's not that #ocl_func will ccall the clfunc, rather #ocl_func generates the source code expression which ccalls the clfunc. The quoted block that is returned by a macro is seen as source code by Julia and is compiled like any other code, a function definiton in this case.
So when Julia sees the line:
#ocl_func(clGetPlatformIDs, CL_int,
(CL_uint, Ptr{CL_platform_id}, Ptr{CL_uint}))
this gets turned into a function definition that uses ccall inside it.
Similarly, #check also generates code that does an error check and throws if it's not a success. #check doesn't need to know about #ocl_func, instead both just expand into the quoted code that they return, so any interaction is between those generated code blocks.
My question how does the 'called' clfunc, for example this one:
#check api.clGetPlatformIDs(0, C_NULL, nplatforms) (in /platforms.jl)
is passed to the #ocl_func macro?
The function definition for api.clGetPlatformIDs is generated by the #ocl_func macro, but once its generated it can be called like any normal function, and that's what's happening here. So there's no dependency between #check and #ocl_func necessary for this to work.

While Sundar's answer explains what is going on it is always a good idea to try #macroexpand too:
julia> #macroexpand OpenCL.cl.#check api.clGetPlatformIDs(0, C_NULL, nplatforms)
quote
#= C:\JuliaPkg\Julia-1.8.1\packages\OpenCL\yqbHv\src\macros.jl:3 =#
local var"#3#err"::OpenCL.cl.CL_int
#= C:\JuliaPkg\Julia-1.8.1\packages\OpenCL\yqbHv\src\macros.jl:4 =#
var"#3#err" = api.clGetPlatformIDs(0, C_NULL, nplatforms)
#= C:\JuliaPkg\Julia-1.8.1\packages\OpenCL\yqbHv\src\macros.jl:5 =#
if var"#3#err" != OpenCL.cl.CL_SUCCESS
#= C:\JuliaPkg\Julia-1.8.1\packages\OpenCL\yqbHv\src\macros.jl:6 =#
OpenCL.cl.throw(OpenCL.cl.CLError(var"#3#err"))
end
#= C:\JuliaPkg\Julia-1.8.1\packages\OpenCL\yqbHv\src\macros.jl:8 =#
var"#3#err"
end

Related

Julia equivalent of a lisp symbol macro?

I'm trying to do what a Lisp hacker would call a "symbol macro". To wit, here's what I'm using now:
global ghypers = Dict()
macro hyp(variable, value); ghypers[:($variable)] = :($value); end
macro hyp(variable); ghypers[:($variable)]; end
#hyp foo 5
println(ghypers)
println(#hyp foo)
println(#hyp(foo)+1)
So far so good, but the last thing is ugly, I want to do this:
#foo+1
Sort of like this:
macro foo(); ghypers[:foo]; end
println(#foo)
println(#foo()+1)
Close, and works, but not quite what I wanted, which, again, is:
println(#foo+1)
MethodError: no method matching #foo(::LineNumberNode, ::Module, ::Expr)
Except that that doesn't work.
Lisp has the concept of a symbol macro, where you can bind a macro expansion to a symbol, like foo, so that symbol would expand the to value.
Now, an obvious but equally bad (in the unhygienic sense) way to do this would simply be to bind the global var foo to the value, but I don't want global (or, rather, I want them localized in ghypers).
Is there any way to do the thing I'm seeking in Julia?
(<flame> Having been a happy lisp camper for 40+ years, Julia is the first programming language that I can say I actually like even a little. But not having grok'ed the importance of homoiconicity, their macro system is ... well, to my eyes, quite a mess, where Lisp's is clear as rain water. </flame> :-)
There are two direct ways. If the thing is hygienic and referentially transparent, just use a const value. That won't work in your example.
Otherwise you have to use the macro in call syntax: #foo(). There's no around that, it's how Julia syntax work. Although I won't recommend doing it either.
But a nicer alternative, IMHO, would be an "anaphoric context macro", something like:
#withhyper (stuff) begin
println(foo+1)
end
where foo is a name escaped inside some local scope. Expand the block to a variation of
let foo = setup_hyper(stuff, ...)
println(foo+1)
end

Language constructs vs Macros in elixir

I'm learning control structures from https://elixirschool.com/en/lessons/basics/control-structures/
and I noticed that it mentions
Chances are you’ve encountered if/2 before, and if you’ve used Ruby you’re familiar with unless/2. In Elixir they work much the same way but they are defined as macros, not language constructs. You can find their implementation in the Kernel module.
so what's the difference between a language construct and a macro in Elixir and when is it necessary to write a macro?
A macro is a way to program a programming language. Simply put, a macro is a way to generate program code, instead of writing it yourself all the time.
A language construct on the other hand (sometimes called "special form"), is something that is at the core of elixir itself. An oversimplification could be that the implementation of if is not done in Elixir, but in the language in which Elixir is implemented.
Suppose you want to use the mentioned unless.
Edit: unless is available in Elixir. But let's assume for the remainder that it is not.
In Elixir, there is no unless available in the language. José Valim did not implement it. But you can always write something that has the same semantics: a negated if.
We would like to have this, but we don't:
unless sun_shines() do
open_umbrella()
end
But we only have an if and a not, so we can write:
if not sun_shines() do
open_umbrella()
end
Secondly, a macro is a special kind of function, but its parameters are code, and the result of executing a macro is code as well. Assuming we have the unless macro, it takes in a condition (i.e., sun_shines()), and a body (i.e., open_umbrella()), and returns if not sun_shines(), do: open_umbrella(). So a macro is a function that works at the level of your "dead code" and generates "dead code".
You might think that this is just too stupid to write a macro for. That's true. But these types of problems happen more often than you think, and then a macro is a great solution to that problem. It's just a way to program your programming language.
An example implementation of the unless macro has been provided by Aleksei Matiushkin:
defmodule MyMacros do
defmacro unless(ast, do: block) do
quote do
if not unquote(ast) do
unquote(block)
end
end
end
end
Here you can clearly see that you give it an AST (Abstract Syntax Tree), and it will transform it to another AST (quote), and inject that in the place where you called the macro. Note that this all happens at compile time. Your program is not being executed at this point!
For example, suppose you have the above module available, and this is your program:
defmodule MyProgram do
def my_function(x) do
unless sun_shining() do
open_umbrella()
end
end
end
After compilation, and before execution, your program will look like this:
defmodule MyProgram do
def my_function(x) do
if not sun_shining() do
open_umbrella()
end
end
end
This phase is what we call macro expansion phase.
As an extra, here you can find two actual macros used in Elixir and ExUnit respectively.
https://github.com/elixir-lang/elixir/blob/d48b16cf549eca0629449a47cc5574a7170706c3/lib/ex_unit/lib/ex_unit/assertions.ex#L104
https://github.com/elixir-lang/elixir/blob/13ced80fcda1bea69037aacd4b052a0c44b4be61/lib/elixir/lib/stream/reducers.ex#L58
Note: I keep adding more and more information to this answer. The answer actually deserves a whole book and Metaprogramming Elixir by Chris McCord is the best one.

What is the function "defs" in Lisp?

In the "Dictio" file, located at the link "Text-only console version" of this site, I've noticed a Lisp command (?) called defs.
I assume that this is something similar to defun, but am unable to find any information on what defs does; is it used to define a function, or maybe a variable? I am looking to reproduce this code using modern techniques and it would help to know the purpose of defs.
The defs calls seem to also include more than a name before the arguments (I would expect it to read (defs name () body).
Looking at the first function in the list, there appears to be more included in this "function definition" [the word 'features' specifically] and in the third function, there is ['semantics'] included after what appears to be the name of the function (before the arguments).
DEFS is defined by the software in the file SYSCOM.
It is a FEXPR, which is a function which gets the arguments unevaluated. Common Lisp has no such feature. It uses macros instead.
Example use:
(DEFS \#COLOR
FEXPR (LAMBDA (A)
(EVAL (SUBST (CAR A)
'COLOR
'(OBJECT
(MARKERS\: (\#PHYSOB COLOR)
PROCEDURE\: ((\#COLOR *** COLOR)))))))
PRIORITY 192.
SYS (\#PROPERTY))
Here you have a symbol #COLOR. It gets a function (actually a FEXPR) defined under this name. Also it puts a PRIORITY and SYS onto the property list of the symbol. Thus DEFS is used to define symbols with functions and properties in one defining form.

Weird hygiene in macros

Suppose I have this macro definition in a module:
module Example
export #example_macro
macro example_macro(a)
quote
local r = RemoteRef()
put!(r, $(esc(a)))
remotecall_fetch(2, (r) -> fetch(r), r)
end
end
end
And here is its expansion:
julia> include("Example.jl")
julia> using Example
julia> macroexpand(quote #example_macro a end)
quote # none, line 1:
begin # /.../Example.jl, line 7:
local #121#r = Example.RemoteRef() # line 8:
Example.put!(#121#r,a) # line 9:
Example.remotecall_fetch(2,(r) -> Example.fetch(r),#121#r)
end
end
Every single one of globally available functions (like put! or fetch) are prefixed with the name of the module. I understand that this is needed for the macro to be hygienic - if, say, fetch was redefined in the module in which #example_macro is called, and fetch was inserted into the expansion as is, it wouldn't work correctly.
However, this also requires Example module to be available not only in the main process, but also on the second worker (since remotecall_fetch needs to execute Example.fetch on it). I don't want it - after all, fetch is a basic function available on all workers by default.
So, is there a way to disable prefixing all identifiers with the name of the current module? I think this would mean turning the macro non-hygienic as it is impossible to decide where some identifier (like fetch) is defined on macro expansion phase, and that's fine for me.
Since this is a pretty profound question, I think that you should give the Julia devs themselves a chance to answer it by asking on julia-users.
Currently, you can completely circumvent macro hygiene by wrapping the whole quote block in your macro in an esc(...) (don't forget to take away the esc around a), but I would in general advise against it - then you are on your own.

lisp defclass macro problem

Bit of background, I'm a total lisp noob, only started a few weeks ago, but I've been developing in other langs for years. Logic no problem, lisp, problem.
I'm trying to write a macro that will define two clsql classes for me to get around a problem with the library. I'd like the classes to be named x and `x-insert`` , so within the macro I'd like the macro to compute the symbol name of x-insert, but I'm having difficulity doing this. My attempt is below, but i'm stumped on two things.
How do I get it to create the class names. If i remove the space in ,class -insert, it wont eval, which I understand, so I presume I'm missing some straightforward way to tell it to ignore the space,and create the name as a single word, and the second problem is getting it to create two classes, not one, as its only expanding the last part of the macro from what I can see using macro expand.
Perhaps I'm going about this the wrong way altogether, so feel free to kick me in the right direction.
(defmacro gen-pair (class base-slots pkey-slot base-table)
`(clsql:def-view-class ,class -insert()
(
,base-slots
)
(:base-table ,base-table)
)
`(clsql:def-view-class ,class (,class -insert)
(
,pkey-slot
)
(:base-table ,base-table)
)
)
It is difficult to begin an explanation here, since you seem to have a
whole stack of misconceptions.
First question (how to compose symbol names): Lisp macros do not
operate on text but on code. In a backquote form, ,class
evaluates to the code passed into the class parameter of the macro,
most likely a class name in this case. Writing another symbol after
that does not magically merge the symbol names; why should it? If you
want to compose a new symbol name, you have to construct it:
,(intern (string-upcase (concatenate 'string
(symbol-name class)
"-insert")))
Second question (why it seems to expand only the second part): the
contents of a defmacro form are evaluated in an implicit progn
(that is why it does not complain about an invalid number of arguments
here). The return value of the last form is the return value of the
whole defmacro form. In this case, the return value is the code
produced by that backquote form. A macro defines a function that
expands a form into a new form; you cannot expand it into two
unrelated forms. You have to produce a progn form that contains the
two forms you want to have.
Third question (why your code looks so different from what Lispers
write): do not throw around parentheses like nail clippings. There
are several Lisp style guides flying around on the net. Read them.
Wer die Form beherrscht, kann mit ihr spielen (roughly: when you
know the proper way, you can play with it).
Fourth question (how to come around the perceived limitation of
clsql): you could ask that question directly, no? What limitation do
you mean?