Related
I am trying to build datalog queries programmatically, but keep running into the problem that I will illustrate with an example function:
(defn test-expr [attribute]
`[?entity ~attribute ?value]])
When I run (test-expr 3), I would expect the output:
[?entity 3 ?value]
But instead, I get
[mynamespace/?entity 3 mynamespace/?value]
Which obviously is not what I want. Is there a way to tell clojure "please just quote the list and expand variables I tell you to?"
Yes, there is.
(defn test-expr [attribute]
`[~'?entity ~attribute ~'?value])
Here you first unquote the syntax quotation and then immediately quote the symbol (~' construct) again. The result is namespace-less symbol.
It is equivalent to the following, which explains how it works:
(defn test-expr [attribute]
`[~(quote ?entity) ~attribute ~(quote ?value)])
What you're looking for is the backtick library by Brandon Bloom https://github.com/brandonbloom/backtick
It was built for the exact problem you describe. It supplies a command named 'template' that works like the backtick but without the namespacing stuff.
In Clojure, quasiquotation and namespace resolution are mixed together in a single feature. This has great benefits for writing macros in a language like Clojure, which is a "Lisp-1" (as opposed to Common Lisp, which is a "Lisp-2", with separate namespaces for functions and variables.)
I also agree that it might have been better not to conflate these features, but it would have made the writing of macros in Clojure less elegant, so I can see why it works the way it does.
I want to realize logging in my project.
I have macro, smth like
__LOG_TRACE(lg, expr,...) LOG_TRACE_STREAM(lg) << expr;
So I want to realize interface for this macro - another macro, but I want to support 2 types:
LOG_TRACE(msg);
LOG_TRACE(my_logger, msg);
I have some global logger, and first macro will write msg using global logger.
Second macro will take my_logger and write msg using it.
I can make it with LOG_TRACE(msg, my_logger); - but it's not good, it's harder to read in code. Order of arguments in __LOG_TRACE is not necessary.
Upd:
I don't mean overloading macros.
Look - for example I can do this
#define LOG_TRACE(...) __LOG_TRACE(__VA_ARGS__, current_active)
Now I can write
LOG_TRACE(msg);
LOG_TRACE(msg, logger);
But I want not msg,logger and logger,msg
Macro overloading is not allowed in C or C++. But there are workarounds. Here's an article that will help you "overload" your macro: http://cplusplus.co.il/2010/08/31/overloading-macros/
If you don't have a variable number of loggers, i recommend you to make a macro for each logger. ex (LOG_TRACE_XML, LOG_TRACE_OUT, LOG_TRACE_TXT). Because simpler is better.
But a better way to do this is to have LOG_TRACE_ERROR/ LOG_TRACE_WARNING/ LOG_TRACE_INFO and manage the way these macros behave using IPC or another macro (SET_MODE(XML/TXT/OUT))
You cannot overload pre-processor macros, your compiler will consider this a redeclaration, rather than an overload, and so only the 2nd will be valid.
You should attempt to name your macros differently, both for readability and because that's the only way you'll get the functionality you want.
Why not make it a function + do and stringify expression macro?
#define DO_AND_RETURN_STRING_EXPR(x) (x,#x)
ov(DO_AND_RETURN_STRING_EXPR(y))
ov(my_logger, DO_AND_RETURN_STRING_EXPR(y))
(note I haven't tested this code).
__VA_ARGS__ is an extension to the current C++ standard, but if you are willing to play with this P99 has a lot of utility macros to achieve what you want. In particular macros that implement conditionals according to the number of arguments they are called.
#define LOG_TRACE(...) \
P99_IF_EQ_1(P99_NARG(__VA_ARGS__)) \
(LOG_TRACE_(my_logger, __VA_ARGS__)) \
(LOG_TRACE_(__VA_ARGS__))
P99 is not really C++ compatible, so you'd have to adapt things a bit.
BTW, identifiers that start with _ and a capital letter or another underscore are reserved by C and C++. Double underscores in general are not allowed for C++ because they could interfere with name mangling. So you'd better chose a different name for your base macro.
I have seen one answer of How does Lisp let you redefine the language itself?
Stack Overflow question (answered by Noah Lavine):
Macros aren't quite a complete redefinition of the language, at least as far as I know (I'm actually a Schemer; I could be wrong), because there is a restriction. A macro can only take a single subtree of your code, and generate a single subtree to replace it. Therefore you can't write whole-program-transforming macros, as cool as that would be.
After reading this I am curious about whether there are "whole-program-transforming macros" in Lisp or Scheme (or some other language).
If not then why?
It is not useful and never required?
Same thing could be achieved by some other ways?
It is not possible to implement it even in Lisp?
It is possible, but not tried or implemented ever?
Update
One kind of use case
e.g.
As in stumpwm code
here are some functions all in different lisp source files
uses a dynamic/global defvar variable *screen-list* that is defined in primitives.lisp , but used in screen.lisp, user.lisp, window.lisp.
(Here each files have functions, class, vars related to one aspect or object)
Now I wanted to define these functions under the closure where
*screen-list* variable available by let form, it should not be
dynamic/global variable, But without moving these all functions into
one place (because I do not want these functions to lose place from their
related file)
So that this variable will be accessible to only these functions.
Above e.g. equally apply to label and flet, so that it will further possible
that we could make it like that only required variable, function will be available,
to those who require it.
Note one way might be
implement and use some macro defun_with_context for defun where first argument is
context where let, flet variables definend.
But apart from it could it be achieved by reader-macro as
Vatine and Gareth Rees answered.
You quoted Noah Lavine as saying:
A macro can only take a single subtree of your code, and generate a single subtree to replace it
This is the case for ordinary macros, but reader macros get access to the input stream and can do whatever they like with it.
See the Hyperspec section 2.2 and the set-macro-character function.
In Racket, you can implement whole-program-transforming macros. See the section in the documentation about defining new languages. There are many examples of this in Racket, for example the lazy language and Typed Racket.
Off the top of my head, a few approaches:
First, you can. Norvig points out that:
We can write a compiler as a set of macros.
so you can transform an entire program, if you want to. I've only seen it done rarely, because typically the intersection between "things you want to do to every part of your program" and "things that you need macro/AST-type transformations for" is a pretty small set. One example is Parenscript, which transforms your Lisp code ("an extended subset of CL") into Javascript. I've used it to compile entire files of Lisp code into Javascript which is served directly to web clients. It's not my favorite environment, but it does what it advertises.
Another related feature is "advice", which Yegge describes as:
Great systems also have advice. There's no universally accepted name for this feature. Sometimes it's called hooks, or filters, or aspect-oriented programming. As far as I know, Lisp had it first, and it's called advice in Lisp. Advice is a mini-framework that provides before, around, and after hooks by which you can programmatically modify the behavior of some action or function call in the system.
Another is special variables. Typically macros (and other constructs) apply to lexical scope. By declaring a variable to be special, you're telling it to apply to dynamic scope (I think of it as "temporal scope"). I can't think of any other language that lets you (the programmer) choose between these two. And, apart from the compiler case, these two really span the space that I'm interested in as a programmer.
A typical approach is to write your own module system. If you just want access to all the code, you can have some sort of pre-processor or reader extension wrap source files with your own module annotation. If you then write your own require or import form, you will ultimately be able to see all the code in scope.
To get started, you could write your own module form that lets you define several functions which you then compile in some clever way before emitting optimized code.
There's always the choice of using compiler macros (they can do whole-function transformation based on a lew of criteria, but shouldn't change the value returned, as that would be confusing).
There's reader macros, they transform the input "as it is read" (or "before it is read", if you prefer). I haven't done much large-scale reader-macro hacking, but I have written some code to allow elisp sourec to be (mostly) read in Common Lisp, with quite a few subtle differences in syntactic sugar between the two.
I believe those sorts of macros are called code-walking macros. I haven't implemented a code walker myself, so I am not familiar with the limits.
In Common LISP, at least, you may wrap top-level forms in PROGN and they still retain their status as top-level forms (see CLTL2, section 5.3). Therefore, the limitation of a macro generating a single subtree is not much of a limitation since it could wrap any number of resulting subtrees within PROGN. This makes whole-program macros quite possible.
E.g.
(my-whole-program-macro ...)
= expands to =>
(progn
(load-system ...)
(defvar ...)
(defconstant ...)
(defmacro ...)
(defclass ...)
(defstruct ...)
(defun ...)
(defun ...)
...
)
What is the minimum set of primitives required such that a language is Turing complete and a lisp variant?
Seems like car, cdr and some flow control and something for REPL is enough. It be nice if there is such list.
Assume there are only 3 types of data, integers, symbols and lists.(like in picolisp)
The lambda calculus is turing complete. It has one primitive - the lambda. Translating that to a lisp syntax is pretty trivial.
There's a good discussion of this in the Lisp FAQ. It depends on your choice of primitives. McCarthy's original "LISP 1.5 Programmer's Manual" did it with five functions: CAR, CDR, CONS, EQ, and ATOM.
I believe the minimum set is what John McCarthy published in the original paper.
The Roots of Lisp.
The code.
The best way to actually know this for sure is if you implement it. I used 3 summers to create Zozotez which is a McCarty-ish LISP running on Brainfuck.
I tried to find out what I needed and on a forum you'll find a thread that says You only need lambda. Thus, you can make a whole LISP in lambda calculus if you'd like. I found it interesting, but it's hardly the way to go if you want something that eventually has side effects and works in the real world.
For a Turing complete LISP I used Paul Grahams explanation of McCarthy's paper and all you really need is:
symbol-evaluation
special form quote
special form if (or cond)
special form lambda (similar to quote)
function eq
function atom
function cons
function car
function cdr
function-dispatch (basically apply but not actually exposed to the system so it handles a list where first element is a function)
Thats 10. In addition to this, to have a implementation that you can test and not just on a drawing board:
function read
function write
Thats 12. In my Zozotez I implemeted set and flambda (anonymous macroes, like lambda) as well. I could feed it a library implementing any dynamic bound lisp (Elisp, picoLisp) with the exception of file I/O (because the underlying BF does not support it other than stdin/stdout).
I recommend anyone to implement a LISP1-interpreter, in both LISP and (not LISP), to fully understand how a language is implemented. LISP has a very simple syntax so it's a good starting point. For all other programming languages how you implement an interpreter is very similar. Eg. in the SICP videos the wizards make an interpreter for a logical language, but the structure and how to implement it is very similar to a lisp interpreter even though this language is completely different than Lisp.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have read a lot that LISP can redefine syntax on the fly, presumably with macros. I am curious how far does this actually go? Can you redefine the language structure so much that it borderline becomes a compiler for another language? For example, could you change the functional nature of LISP into a more object oriented syntax and semantics, maybe say having syntax closer to something like Ruby?
Especially, is it possible to get rid of the parenthesis hell using macros? I have learned enough (Emacs-)LISP to customize Emacs with my own micro-features, but I am very curious how far macros can go in customizing the language.
That's a really good question.
I think it's nuanced but definitely answerable:
Macros are not stuck in s-expressions. See the LOOP macro for a very complex language written using keywords (symbols). So, while you may start and end the loop with parentheses, inside it has its own syntax.
Example:
(loop for x from 0 below 100
when (even x)
collect x)
That being said, most simple macros just use s-expressions. And you'd be "stuck" using them.
But s-expressions, like Sergio has answered, start to feel right. The syntax gets out of the way and you start coding in the syntax tree.
As for reader macros, yes, you could conceivably write something like this:
#R{
ruby.code.goes.here
}
But you'd need to write your own Ruby syntax parser.
You can also mimic some of the Ruby constructs, like blocks, with macros that compile to the existing Lisp constructs.
#B(some lisp (code goes here))
would translate to
(lambda () (some lisp (code goes here)))
See this page for how to do it.
Yes, you can redefine the syntax so that Lisp becomes a compiler. You do this using "Reader Macros," which are different from the normal "Compiler Macros" that you're probably thinking of.
Common Lisp has the built-in facility to define new syntax for the reader and reader macros to process that syntax. This processing is done at read-time (which comes before compile or eval time). To learn more about defining reader macros in Common Lisp, see the Common Lisp Hyperspec -- you'll want to read Ch. 2, "Syntax" and Ch. 23, "Reader". (I believe Scheme has the same facility, but I'm not as familiar with it -- see the Scheme sources for the Arc programming language).
As a simple example, let's suppose you want Lisp to use curly braces rather than parentheses. This requires something like the following reader definitions:
;; { and } become list delimiters, along with ( and ).
(set-syntax-from-char #\{ #\( )
(defun lcurly-brace-reader (stream inchar) ; this was way too easy to do.
(declare (ignore inchar))
(read-delimited-list #\} stream t))
(set-macro-character #\{ #'lcurly-brace-reader)
(set-macro-character #\} (get-macro-character #\) ))
(set-syntax-from-char #\} #\) )
;; un-lisp -- make parens meaningless
(set-syntax-from-char #\) #\] ) ; ( and ) become normal braces
(set-syntax-from-char #\( #\[ )
You're telling Lisp that the { is like a ( and that the } is like a ). Then you create a function (lcurly-brace-reader) that the reader will call whenever it sees a {, and you use set-macro-character to assign that function to the {. Then you tell Lisp that ( and ) are like [ and ] (that is, not meaningful syntax).
Other things you could do include, for example, creating a new string syntax or using [ and ] to enclose in-fix notation and process it into S-expressions.
You can also go far beyond this, redefining the entire syntax with your own macro characters that will trigger actions in the reader, so the sky really is the limit. This is just one of the reasons why Paul Graham and others keep saying that Lisp is a good language in which to write a compiler.
I'm not a Lisp expert, heck I'm not even a Lisp programmer, but after a bit of experimenting with the language I came to the conclusion that after a while the parenthesis start becoming 'invisible' and you start seeing the code as you want it to be. You start paying more attention to the syntactical constructs you create via s-exprs and macros, and less to the lexical form of the text of lists and parenthesis.
This is specially true if you take advantage of a good editor that helps with the indentation and syntax coloring (try setting the parenthesis to a color very similar to the background).
You might not be able to replace the language completely and get 'Ruby' syntax, but you don't need it. Thanks to the language flexibility you could end having a dialect that feels like you are following the 'Ruby style of programming' if you want, whatever that would mean to you.
I know this is just an empirical observation, but I think I had one of those Lisp enlightenment moments when I realized this.
Over and over again, newcomers to Lisp want to "get rid of all the parenthesis." It lasts for a few weeks. No project to build a serious general purpose programming syntax on top of the usual S-expression parser ever gets anywhere, because programmers invariably wind up preferring what you currently perceive as "parenthesis hell." It takes a little getting used to, but not much! Once you do get used to it, and you can really appreciate the plasticity of the default syntax, going back to languages where there's only one way to express any particular programming construct is really grating.
That being said, Lisp is an excellent substrate for building Domain Specific Languages. Just as good as, if not better than, XML.
Good luck!
The best explanation of Lisp macros I have ever seen is at
https://www.youtube.com/watch?v=4NO83wZVT0A
starting at about 55 minutes in. This is a video of a talk given by Peter Seibel, the author of "Practical Common Lisp", which is the best Lisp textbook there is.
The motivation for Lisp macros is usually hard to explain, because they really come into their own in situations that are too lengthy to present in a simple tutorial. Peter comes up with a great example; you can grasp it completely, and it makes good, proper use of Lisp macros.
You asked: "could you change the functional nature of LISP into a more object oriented syntax and semantics". The answer is yes. In fact, Lisp originally didn't have any object-oriented programming at all, not surprising since Lisp has been around since way before object-oriented programming! But when we first learned about OOP in 1978, we were able to add it to Lisp easily, using, among other things, macros. Eventually the Common Lisp Object System (CLOS) was developed, a very powerful object-oriented programming system that fits elegantly into Lisp. The whole thing can be loaded as an extension -- nothing is built-in! It's all done with macros.
Lisp has an entirely different feature, called "reader macros", that can be used to extend the surface syntax of the language. Using reader macros, you can make sublanguages that have C-like or Ruby-like syntax. They transform the text into Lisp, internally. These are not used widely by most real Lisp programmers, mainly because it is hard to extend the interactive development environment to understand the new syntax. For example, Emacs indentation commands would be confused by a new syntax. If you're energetic, though, Emacs is extensible too, and you could teach it about your new lexical syntax.
Regular macros operate on lists of objects. Most commonly, these objects are other lists (thus forming trees) and symbols, but they can be other objects such as strings, hashtables, user-defined objects, etc. These structures are called s-exps.
So, when you load a source file, your Lisp compiler will parse the text and produce s-exps. Macros operate on these. This works great and it's a marvellous way to extend the language within the spirit of s-exps.
Additionally, the aforementioned parsing process can be extended through "reader macros" that let you customize the way your compiler turns text into s-exps. I suggest, however, that you embrace Lisp's syntax instead of bending it into something else.
You sound a bit confused when you mention Lisp's "functional nature" and Ruby's "object-oriented syntax". I'm not sure what "object-oriented syntax" is supposed to be, but Lisp is a multi-paradigm language and it supports object-oriented programming extremelly well.
BTW, when I say Lisp, I mean Common Lisp.
I suggest you put your prejudices away and give Lisp an honest go.
Parenthesis hell? I see no more parenthesis in:
(function toto)
than in:
function(toto);
And in
(if tata (toto)
(titi)
(tutu))
no more than in:
if (tata)
toto();
else
{
titi();
tutu();
}
I see less brackets and ';' though.
What you are asking is somewhat like asking how to become an expert chocolatier so that you can remove all that hellish brown stuff from your favourite chocolate cake.
Yes, you can fundamentally change the syntax, and even escape "the parentheses hell". For that you will need to define a new reader syntax. Look into reader macros.
I do suspect however that to reach the level of Lisp expertise to program such macros you will need to immerse yourself in the language to such an extent that you will no longer consider parenthese "hell". I.e. by the time you know how to avoid them, you will have come to accept them as a good thing.
If you want lisp to look like Ruby use Ruby.
It's possible to use Ruby (and Python) in a very lisp like way which is one of the main reasons they have gained acceptance so quickly.
see this example of how reader macros can extend the lisp reader with complex tasks like XML templating:
http://common-lisp.net/project/cl-quasi-quote/present-class.html
this user library compiles the static parts of the XML into UTF-8 encoded literal byte arrays at compile time that are ready to be write-sequence'd into the network stream. and they are usable in normal lisp macros, they are orthogonal... the placement of the comma character influences which parts are constant and which should be evaluated at runtime.
more details available at: http://common-lisp.net/project/cl-quasi-quote/
another project that for Common Lisp syntax extensions: http://common-lisp.net/project/cl-syntax-sugar/
#sparkes
Sometimes LISP is the clear language choice, namely Emacs extensions. I'm sure I could use Ruby to extend Emacs if I wanted to, but Emacs was designed to be extended with LISP, so it seems to make sense to use it in that situation.
It's a tricky question. Since lisp is already structurally so close to a parse tree the difference between a large number of macros and implementing your own mini-language in a parser generator isn't very clear. But, except for the opening and closing paren, you could very easily end up with something that looks nothing like lisp.
One of the uses of macros that blew my mind was the compile-time verification of SQL requests against DB.
Once you realize you have the full language at hand at compile-time, it opens up interesting new perspectives. Which also means you can shoot yourself in the foot in interesting new ways (like rendering compilation not reproducible, which can very easily turn into a debugging nightmare).