Related
In the article Technical Issues of Separation in Function Cells and Value Cells, Kent Pitman and Richard Gabriel explains the decision of making Common Lisp a Lisp-2:
There are two ways to look at the arguments regarding macros and namespaces. The first is that a single namespace is of fundamental importance, and therefore macros are problematic. The second is that macros are fundamental, and therefore a single namespace is problematic.
According to that, when programming macros, a single namespace in macro programming is inherently problematic.
But Clojure's approach is a little bit different: the backquote does namespace resolution.
In chapter 9 of the book On Lisp, Paul Graham talks about avoiding variable capture by separating code in packages:
However, packages do not provide a very general solution to the problem of
capture. In the first place, macros are an integral part of some programs, and it
would be inconvenient to have to separate them in their own package. Second,
this approach offers no protection against capture by other code in the macros
package.
As far as I can see, Clojure's solution to variable capture looks like the packaged option showed by Paul Graham.
One of the major drawbacks pointed by Paul Graham is that it would be inconvenient to separate macros in different packages, but Clojure's backquote does it automatically, by prepending the namespace of the symbol, right?
So, is it a complete solution to variable capture? Or Kent Pitman's words still apply? If there is any problem that Common Lisp's separeted namespaces can handle that Clojure cannot, could you write down an example?
I have never encountered a limitation with Clojure's macro system. It's a fully general macro system, and to my knowledge it is exactly comparable with Common Lisp in terms of fundamental capabilities.
There are obviously many syntactical differences, but I think they are mostly superficial and don't affect the expressive power that you can achieve with macros.
My view is that Clojure gets a lot of design aspects right here:
Lisp-1 is simpler and conceptually clearer than Lisp-2, particularly in a functional language where you actually need to treat functions as first class values.
Symbol capture generally isn't a problem - the Clojure syntax quote and namespace system do a good job of making macros both usable and readable.
As a final comment, the summary of the linked article is quite illuminating:
The bulk of arguments that focus on clean semantics and notational
simplicity tend to favor uniting the function and value namespaces..... We feel that the time for
such radical changes to Common Lisp passed, and it would be the job of
future Lisp designers to take lessons from Common Lisp and Scheme to
produce an improved Lisp.
In my humble view - Clojure is a good example of an "improved Lisp".
Paul Graham writes:
For example, types seem to be an
inexhaustible source of research
papers, despite the fact that static
typing seems to preclude true macros--
without which, in my opinion, no
language is worth using.
What's the big deal with macros? I haven't spent a whole lot of time with them, but from the legacy C/C++ I've worked with they appear to be mostly used as a hack before templates/generics existed.
It's hard to imagine that
DECLARELIST(StrList, string);
StrList slist;
is somehow preferable to
List<String> slist;
Am I missing something?
Then there's the usage as a pseudo-function, like MAKEPOINTS:
POINTS MAKEPOINTS(
DWORD dwValue
);
Why not define it as a function instead? Is this some optimization, where you avoid code duplication without having the added overhead of another stack frame?
Then there's also tricky control flow things involving GOTO, which seem to be of dubious value.
What's so great about macros? They're less type safe (in C and C++) (right?). Why won't Paul Graham program without them?
LISP macros are an entirely different beast. C/C++ macros can merely replace a piece of text with abother piece of text using an extremely basic language. Whereas a LISP program is (after "reading") is a LISP data structure and can therefore be manipulated using the whole language.
With such macros, you could (given you're a really clever hacker) vastly extend the language and everybody could use it relatively easily, since you did it with macros. Take for example the the Common Lisp Object System. At its core, the language has nothing even remotely like objects. It is entirely implemented in the language itself, including a relatively simple syntax for use - using macros.
Of course macros are less necessary when the language has most things you'd every want built-in. OTOH, the LISP fans are of the opinion that a sufficiently simple language (LISP) with sufficiently powerful metaprogramming capabilities (macros) is better since new concepts can be incorporated into the language without changing the spec or working implementations. But the most compelling example for macro usage is the DSL area. Ruby on Rails and others show every day how useful DSLs can be. Yes, Ruby doesn't have macros, it just exploits how much Ruby syntax can be bent. In other languages, or when even Ruby's syntax isn't flexible enough, you need macros or a fully-blown parser/interpreter to implement a complex DSL.
Macros are really only good for two things in C/C++, and should generally be the tool of last resort (if you can accomplish something without using macros, do so).
Creating new syntactic structures or abstractions that do not exist in the language.
Eliminating duplication, especially between things that must be in sync with each other.
It's almost never to use a macro as a function.
You also have to realize that LISP macros are not C/C++ macros.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I've read The Nature of Lisp. The only thing I really got out of that was "code is data." But without defining what these terms mean and why they are usually thought to be separate, I gain no insight. My initial reaction to "code is data" is, so what?
The old fashioned view: 'it' is interactive computation with symbolic expressions.
Lisp enables easy representation of all kinds of expressions:
english sentence
(the man saw the moon)
math
(2 * x ** 3 + 4 * x ** 2 - 3 * x + 3)
rules
(<- (likes Kim ?x) (likes ?x Lee) (likes ?x Kim))
and also Lisp itself
(mapcar (function sqr) (quote (1 2 3 4 5)))
and many many many more.
Lisp now allows to write programs that compute with such expressions:
(translate (quote (the man saw the moon)) (quote german))
(solve (quote (2 * x ** 3 + 4 * x ** 2 - 3 * x + 3)) (quote (x . 3)))
(show-all (quote (<- (likes Kim ?x) (likes ?x Lee) (likes ?x Kim))))
(eval (quote (mapcar (function sqr) (quote (1 2 3 4 5)))))
Interactive means that programming is a dialog with Lisp. You enter an expression and Lisp computes the side effects (for example output) and the value.
So your programming session is like 'talking' with the Lisp system. You work with it until you get the right answers.
What are these expressions? They are sentences in some language. They are part descriptions of turbines. They are theorems describing a floating point engine of an AMD processor. They are computer algebra expressions in physics. They are descriptions of circuits. They are rules in a game. They are descriptions of behavior of actors in games. They are rules in a medical diagnosis system.
Lisp allows you to write down facts, rules, formulas as symbolic expressions. It allows you to write programs that work with these expressions. You can compute the value of a formula. But you can equally easy write programs that compute new formulas from formulas (symbolic math: integrate, derive, ...). That was Lisp designed for.
As a side effect Lisp programs are represented as such expressions too. Then there is also a Lisp program that evaluates or compiles other Lisp programs. So the very idea of Lisp, the computation with symbolic expressions, has been applied to Lisp itself. Lisp programs are symbolic expressions and the computation is a Lisp expression.
Alan Kay (of Smalltalk fame) calls the original definition of Lisp evaluation in Lisp the Maxwell's equations of programming.
Write Lisp code. The only way to really 'get' Lisp (or any language, for that matter) is to roll up your sleeves and implement some things in it. Like anything else, you can read all you want, but if you want to really get a firm grasp on what's going on, you've got to step outside the theoretical and start working with the practical.
The way you "get" any language is by trying to write some code in it.
About the "data is code" thing, in most languages there is a clear separation between the code that gets executed, and the data that is processed.
For example, the following simple C-like function:
void foo(int i){
int j;
if (i % 42 == 0){
bar(i-2);
}
for (j = 0; j < i; ++j){
baz();
}
}
the actual control flow is determined once, statically, while writing the code. The function bar isn't going to change, and the if statement at the beginning of the function isn't going to disappear. This code is not data, it can not be manipulated by the program.
All that can be manipulated is the initial value of i. And on the other hand, that value can not be executed the way code can. You can call the function foo, but you can't call the variable i. So i is data, but it is not code.
Lisp does not have this distinction. The program code is data that can be manipulated too. Your code can, at runtime, take the function foo, and perhaps add another if statement, perhaps change the condition in the for-loop, perhaps replace the call to baz with another function call. All your code is data that can be inspected and manipulated as simply as the above function can inspect and manipulate the integer i.
I would highly recommend Structure and Interpretation of Computer Programs, which actually uses scheme, but that is a dialect of lisp. It will help you "get" lisp by having you do many different exercises and goes on to show some of the ways that lisp is so usefull.
I think you have to have more empathy for compiler writers to understand how fundamental the code is data thing is. I'll admit, I've never taken a compilers course, but converting any sufficiently high-level language into machine code is a hard problem, and LISP, in many ways, resembles an intermediate step in this process. In the same way that C is "close to the metal", LISP is close to the compiler.
This worked for me:
Read "The Little Schemer". It's the shortest path to get you thinking in Lisp mode (minus the macros). As a bonus, it's relatively short/fun/inexpensive.
Find a good book/tutorial to get you started with macros. I found chapter 8 of "The Scheme
Programming Language" to be a good starting point for Scheme.
http://www.ccs.neu.edu/home/matthias/BTLS/
http://www.scheme.com/tspl3/syntax.html
By watching legendary Structure and Interpretation of Computer Programs?
In Common Lisp, "code is data" boils down to this. When you write, for example:
(add 1 2)
your Lisp system will parse that text and generate a list with three elements: the symbol ADD, and the numbers 1 and 2. So now they're data. You can do whatever you want with them, replace elements, insert other stuff, etc.
The fun part is that you can pass this data on to the compiler and, because you can manipulate these data structures using Lisp itself, this means you can write programs that write other programs. This is not as complicated as it sounds, and Lispers do it all the time using macros. So, just get a book about Lisp, and try it out.
Okay, I'm going to take a crack at this. I'm new to Lisp myself, just having arrived from the world of python. I haven't experienced that sudden moment of enlightenment that all the old Lispers talk about, but I'll tell you what I am seeing so far.
First, look at this random bit of python code:
def is_palindrome(st):
l = len(st)/2
return True if st[:l] == st[:-l-1:-1] else False
Now look at this:
"""
def is_palindrome(st):
l = len(st)/2
return True if st[:l] == st[:-l-1:-1] else False
"""
What do you, as a programmer, see? The code is identical, FYI.
If you are like me, you'll tend to think of the first as active code. It consists of a number of syntactic elements.
The second, despite its similarity, is a single syntactic item. It's a string. You interact with it as a single entity. To deal with it as code - to handle it comfortably along its syntactic boundaries - you will have to do some parsing. To execute it, you need to invoke an interpreter. It's not the same thing at all as the first.
So when we do code generation in most languages what are we dealing with? Strings. When I generate HTML or SQL with python I use python strings as the interface between the two languages. Even if I generate python with python, strings are the tool.*
Doesn't the thought of that just... make you want to dance with joy? There's always this grotesque mismatch between that which you are working with and that which you are working on. I sensed that the first time that I generated SQL with perl. Differences in escaping. Differences in formatting: think about trying to get a generated html document to look tidy. Stuff isn't easy to reuse. Etc.
To solve the problem we serially create templating libraries. Scads of them. Why so many? My guess is that they're never quite satisfactory. By the time they start getting powerful enough they've turned into monstrosities. Granted, some of them - such as SQLAlchemy and Genshi in the python world - are very beautiful and admirable monstrosities. Let's... um... avoid mention of PHP.
Because strings make an awkward interface between the worked-on language and the worked-with, we create a third language - templates - to avoid them. ** This also tends to be a little awkward.
Now let's look at a block of quoted Lisp code:
'(loop for i from 1 to 8 do (print i))
What do you see? As a new Lisp coder, I've caught myself looking at that as a string. It isn't. It is inactive Lisp code. You are looking at a bunch of lists and symbols. Try to evaluate it after turning one of the parentheses around. The language won't let you do it: syntax is enforced.
Using quasiquote, we can shoehorn our own values into this inactive Lisp code:
`(loop for i from 1 to ,whatever do (print i))
Note the nature of the shoehorning: one item has been replaced with another. We aren't formatting our value into a string. We're sliding it into a slot in the code. It's all neat and tidy.
In fact if you want to directly edit the text of the code, you are in for a hassle. For example if you are inserting a name <varname> into the code, and you also want to use <varname>-tmp in the same code you can't do it directly like you can with a template string: "%s-tmp = %s". You have to extract the name into a string, rewrite the string, then turn it into a symbol again and finally insert.
If you want to grasp the essence of Lisp, I think that you might gain more by ignoring defmacro and gensyms and all that window dressing for the moment. Spend some time exploring the potential of the quasiquote, including the ,# thing. It's pretty accessible. Defmacro itself only provides an easy way to execute the result of quasiquotes. ***
What you should notice is that the hermetic string/template barrier between the worked-on and the worked-with is all but eliminated in Lisp. As you use it, you'll find that your sense of two distinct layers - active and passive - tends to dissipate. Functions call macros which call macros or functions which have functions (or macros!) passed in with their arguments. It's kind of a big soup - a little shocking for the newcomer. That said, I don't find that the distinction between macros and functions is as seamless as some Lisp people say. Mostly it's ok, but every so often as I wander in the soup I find myself bumping up against the ghost of that old barrier - and it really creeps me out!
I'll get over it, I'm sure. No matter. The convenience pays for the scare.
Now that's Lisp working on Lisp. What about working on other languages? I'm not quite there yet, personally, but I think I see the light at the end of the tunnel. You know how Lisp people keep going on about S-expressions being the same thing as a parse tree? I think the idea is to parse the foreign language into S-expressions, work on them in the amazing comfort of the Lisp environment, then send them back to native code. In theory, every language out there could be turned into S-expressions, or even executable lisp code. You're not working in a first language combined with a third language to produce code in a second language. It is all - while you are working on it - Lisp, and you can generate it all with quasiquotes.
Have a look at this (borrowed from PCL):
(define-html-macro :mp3-browser-page ((&key title (header title)) &body body)
`(:html
(:head
(:title ,title)
(:link :rel "stylesheet" :type "text/css" :href "mp3-browser.css"))
(:body
(standard-header)
(when ,header (html (:h1 :class "title" ,header)))
,#body
(standard-footer))))
Looks like an S-expression version of HTML, doesn't it? I have a feeling that Lisp works just fine as its own templating library.
I've started to wonder about an S-expression version of python. Would it qualify as a Lisp? It certainly wouldn't be Common Lisp. Maybe it would be nicer - for python programmers at least. Hey, and what about P-expressions?
* Python now has something called AST, which I haven't explored. Also a person could use python lists to represent other languages. Relative to Lisp, I suspect that both are a bit of a hack.
** SQLAlchemy is kind of an exception. It's done a nice job of turning SQL directly into python. That said, it appears to have involved significant effort.
*** Take it from a newbie. I'm sure I'm glossing over something here. Also, I realize that quasiquote is not the only way to generate code for macros. It's certainly a nice one, though.
Data is code is an interesting paradigm that supports treating a data structure as a command. Treating data in this way allows you to process and manipulate the structure in various ways - e.g. traversal - by evaluating it. Moreover, the 'data is code' paradigm obviates the need in many cases to develop custom parsers for data structures; the language parser itself can be used to parse the structures.
The first step is forgetting everything you have learned with all the C and Pascal-like languages. Empty your mind. This is the hardest step.
Then, take a good introduction to programming that uses Lisp. Don't try to correlate what you see with anything that you know beforehand (when you catch yourself doing that, repeat step 1). I liked Structure and Interpretation of Computer Programs (uses Scheme), Practical Common Lisp, Paradigms of Artificial Intelligence Programming, Casting Spels in Lisp, among others. Be sure to write out the examples. Also try the exercises, but limit yourself to the constructs you have learned in that book. If you find yourself trying to find, for example, some function to set a variable or some statement that resembles a for loop, repeat step 1, then revisit the chapters before to find out how it is done in Lisp.
Read and understand the legendary page 13 of the Lisp 1.5 Programmer's Manual
According to Alan Kay, at least.
One of the reasons that some university computer science programs use Lisp for their intro courses is that it's generally true that a novice can learn functional, procedural, or object-oriented programming more or less equally well. However, it's much harder for someone who already thinks in procedural statements to begin thinking like a functional programmer than to do the inverse.
When I tried to pick up Lisp, I did it "with a C accent." set! amd begin were my friends and constant companions. It is surprisingly easy to write Lisp code without ever writing any functional code, which isn't the point.
You may notice that I'm not answering your question, which is true. I merely wanted to let you know that it's awfully hard to get your mind thinking in a functional style, and it'll be an exciting practice that will make you a stronger programmer in the long run.
Kampai!
P.S. Also, you'll finally understand that "my other car is a cdr" bumper sticker.
To truly grok lisp, you need to write it.
Learn to love car, cdr, and cons. Don't iterate when you can recurse. Start out writing some simple programs (factorial, list reversal, dictionary lookup), and work your way up to more complex ones (sorting sets of items, pattern matching).
On the code is data and data is code thing, I wouldn't worry about it at this point. You'll understand it eventually, and its not critical to learning lisp.
I would suggest checking out some of the newer variants of Lisp like Arc or Clojure. They clean up the syntax a little and are smaller and thus easier to understand than Common Lisp. Clojure would be my choice. It is written on the JVM and so you don't have the issues with various platform implementations and library support that exist with some Lisp implementations like SBCL.
Read On Lisp and Paradigms in Artificial Intelligence Programming. Both of these have excellent coverage of Lisp macros - which really make the code is data concept real.
Also, when writing Lisp, don't iterate when you can recurse or map (learn to love mapcar).
it's important to see that data is code AND code is data. This feeds the eval/apply loop. Recursion is also fun.
(This link is broken:
![Eval/Apply][1]
[1]: http://ely.ath.cx/~piranha/random_images/lolcode-eval_apply-2.jpg
)
I'd suggest that is a horrible introduction to the language. There are better places to start and better people/articles/books than the one you cited.
Are you a programmer? What language(s)?
To help you with your question more background might be helpful.
About the whole "code is data" thing:
Isn't that due to the "von Neumann architecture"? If code and data were located in physically separate memory locations, the bits in the data memory could not be executed whereas the bits in the program memory could not be interpreted as anything but instructions to the CPU.
Do I understand this correctly?
I think to learn anything you have to have a purpose for it, such as a simple project.
For Lisp, a good simple project is a symbolic differentiator, so for example
(diff 'x 'x) -> 1
(diff 'a 'x) -> 0
(diff `(+ ,xx ,yy) 'x) where xx and yy are subexpressions
-> `(+ ,(diff xx 'x),(diff yy 'x))
etc. etc.
and then you need a simplifier, such as
(simp `(+ ,x 0)) -> x
(simp `(* ,x 0)) -> 0
etc. etc.
so if you start with a math expression, you can eval it to get its value, and you can eval its derivative to get its derivative.
I hope this illustrates what can happen when program code manipulates program code.
As Marvin Minsky observed, computer math is always worried about accuracy and roundoff error, right? Well, this is either exactly right or completely wrong!
You can get LISP in many ways, the most common is by using Emacs or working next to somebody who has developed LISP already.
Sadly, once you get LISP, it's hard to get rid of it, antibiotics won't work.
BTW: I also recommend The Adventures of a Pythonista in Schemeland.
This may be helpful: http://www.defmacro.org/ramblings/fp.html (isn't about LISP but about functional programming as a paradigm)
The way I think about it is that the best part of "code is data" is the face that function are, well, functionally no different than another variable. The fact that you can write code that writes code is one of the single most powerful (and often overlooked) features of Lisp. Functions can accept other functions as parameters, and even return functions as a result.
This lets one code at a much higher level of abstraction than, say, Java. It makes many tasks elegant and concise, and therefore, makes the code easier to modify, maintain, and read, or at least in theory.
I would say that the only way to truly "get" Lisp is to spend a lot of time with it -- once you get the hang of it, you'll wish you had some of the features of Lisp in your other programming languages.
I was working with a Lisp dialect but also learning some Haskell as well. They share some similarities but the main difference in Common Lisp seems to be that you don't have to define a type for each function, argument, etc. whereas in Haskell you do. Also, Haskell is mostly a compiled language. Run the compiler to generate the executable.
My question is this, are there different applications or uses where a language like Haskell may make more sense than a more dynamic language like Common Lisp. For example, it seems that Lisp could be used for more bottom programming, like in building websites or GUIs, where Haskell could be used where compile time checks are more needed like in building TCP/IP servers or code parsers.
Popular Lisp applications:
Emacs
Popular Haskell applications:
PUGS
Darcs
Do you agree, and are there any studies on this?
Programming languages are tools for thinking with. You can express any program in any language, if you're willing to work hard enough. The chief value provided by one programming language over another is how much support it gives you for thinking about problems in different ways.
For example, Haskell is a language that emphasizes thinking about your problem in terms of types. If there's a convenient way to express your problem in terms of Haskell's data types, you'll probably find that it's a convenient language to write your program in.
Common Lisp's strengths (which are numerous) lie in its dynamic nature and its homoiconicity (that is, Lisp programs are very easy to represent and manipulate as Lisp data) -- Lisp is a "programmable programming language". If your program is most easily expressed in a new domain-specific language, for example, Lisp makes it very easy to do that. Lisp (and other dynamic languages) are a good fit if your problem description deals with data whose type is poorly specified or might change as development progresses.
Language choice is often as much an aesthetic decision as anything. If your project requirements don't limit you to specific languages for compatibility, dependency, or performance reasons, you might as well pick the one you feel the best about.
You're opening multiple cans of very wriggly worms. First off, the whole strongly vs weakly typed languages can. Second, the functional vs imperative language can.
(Actually, I'm curious: by "lisp dialect" do you mean Clojure by any chance? Because it's largely functional and closer in some ways to Haskell.)
Okay, so. First off, you can write pretty much any program in pretty much any normal language, with more or less effort. The purported advantage to strong typing is that a large class of errors can be detected at compile time. On the other hand, less typeful languages can be easier to code in. Common Lisp is interesting because it's a dynamic language with the option of declaring and using stronger types, which gives the CL compiler hints on how to optimize. (Oh, and real Common Lisp is usually implemented with a compiler, giving you the option of compiling or sticking with interpreted code.)
There are a number of studies about comparing untyped, weakly typed, and strongly typed languages. These studies invariably either say one of them is better, or say there's no observable difference. There is, however, little agreement among the studies.
The biggest area in which there may be some clear advantage is in dealing with complicated specifications for mathematical problems. In those cases (cryptographic algorithms are one example) a functional language like Haskell has advantages because it is easier to verify the correspondence between the Haskell code and the underlying algorithm.
I come mostly from a Common Lisp perspective, and as far as I can see, Common Lisp is suited for any application.
Yes, the default is dynamic typing (i.e. type detection at runtime), but you can declare types anyway for optimization (as a side note for other readers: CL is strongly typed; don't confuse weak/strong with static/dynamic!).
I could imagine that Haskell could be a bit better suited as a replacement for Ada in the avionics sector, since it forces at least all type checks at compile time.
I do not see how CL should not be as useful as Haskell for TCP/IP servers or code parsers -- rather the opposite, but my contacts with Haskell have been brief so far.
Haskell is a pure functional language. While it does allow imperative constructs (using monads), it generally forces the programmer to think the problem in a rather different way, using a more mathematical-oriented approach. You can't reassign another value to a variable, for example.
It is claimed that this reduces the probability of making some types of mistakes. Moreover, programs written in Haskell tend to be shorter and more concise than those written in typical programming languages. Haskell also makes heavy use of non-strict, lazy evaluation, which could theoretically allow the compiler to make optimizations not otherwise possible (along with the no-side-effects paradigm).
Since you asked about it, I believe Haskell's typing system is quite nice and useful. Not only it catches common errors, but it can also make code more concise (!) and can effectively replace object-oriented constructs from common OO languages.
Some Haskell development kits, like GHC, also feature interactive environments.
The best use for dynamic typing that I've found is when you depend on things that you have no control over so it could as well be used dynamically. For example getting information from XML document we could do something like this:
var volume = parseXML("mydoc.xml").speaker.volume()
Not using duck typing would lead to something like this:
var volume = parseXML("mydoc.xml").getAttrib["speaker"].getAttrib["volume"].ToString()
The benefit of Haskell on the other hand is in safety. You can for example make sure, using types, that degrees in Fahrenheit and Celsius are never mixed unintentionally. Besides that I find that statically typed languages have better IDEs.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have read a lot that LISP can redefine syntax on the fly, presumably with macros. I am curious how far does this actually go? Can you redefine the language structure so much that it borderline becomes a compiler for another language? For example, could you change the functional nature of LISP into a more object oriented syntax and semantics, maybe say having syntax closer to something like Ruby?
Especially, is it possible to get rid of the parenthesis hell using macros? I have learned enough (Emacs-)LISP to customize Emacs with my own micro-features, but I am very curious how far macros can go in customizing the language.
That's a really good question.
I think it's nuanced but definitely answerable:
Macros are not stuck in s-expressions. See the LOOP macro for a very complex language written using keywords (symbols). So, while you may start and end the loop with parentheses, inside it has its own syntax.
Example:
(loop for x from 0 below 100
when (even x)
collect x)
That being said, most simple macros just use s-expressions. And you'd be "stuck" using them.
But s-expressions, like Sergio has answered, start to feel right. The syntax gets out of the way and you start coding in the syntax tree.
As for reader macros, yes, you could conceivably write something like this:
#R{
ruby.code.goes.here
}
But you'd need to write your own Ruby syntax parser.
You can also mimic some of the Ruby constructs, like blocks, with macros that compile to the existing Lisp constructs.
#B(some lisp (code goes here))
would translate to
(lambda () (some lisp (code goes here)))
See this page for how to do it.
Yes, you can redefine the syntax so that Lisp becomes a compiler. You do this using "Reader Macros," which are different from the normal "Compiler Macros" that you're probably thinking of.
Common Lisp has the built-in facility to define new syntax for the reader and reader macros to process that syntax. This processing is done at read-time (which comes before compile or eval time). To learn more about defining reader macros in Common Lisp, see the Common Lisp Hyperspec -- you'll want to read Ch. 2, "Syntax" and Ch. 23, "Reader". (I believe Scheme has the same facility, but I'm not as familiar with it -- see the Scheme sources for the Arc programming language).
As a simple example, let's suppose you want Lisp to use curly braces rather than parentheses. This requires something like the following reader definitions:
;; { and } become list delimiters, along with ( and ).
(set-syntax-from-char #\{ #\( )
(defun lcurly-brace-reader (stream inchar) ; this was way too easy to do.
(declare (ignore inchar))
(read-delimited-list #\} stream t))
(set-macro-character #\{ #'lcurly-brace-reader)
(set-macro-character #\} (get-macro-character #\) ))
(set-syntax-from-char #\} #\) )
;; un-lisp -- make parens meaningless
(set-syntax-from-char #\) #\] ) ; ( and ) become normal braces
(set-syntax-from-char #\( #\[ )
You're telling Lisp that the { is like a ( and that the } is like a ). Then you create a function (lcurly-brace-reader) that the reader will call whenever it sees a {, and you use set-macro-character to assign that function to the {. Then you tell Lisp that ( and ) are like [ and ] (that is, not meaningful syntax).
Other things you could do include, for example, creating a new string syntax or using [ and ] to enclose in-fix notation and process it into S-expressions.
You can also go far beyond this, redefining the entire syntax with your own macro characters that will trigger actions in the reader, so the sky really is the limit. This is just one of the reasons why Paul Graham and others keep saying that Lisp is a good language in which to write a compiler.
I'm not a Lisp expert, heck I'm not even a Lisp programmer, but after a bit of experimenting with the language I came to the conclusion that after a while the parenthesis start becoming 'invisible' and you start seeing the code as you want it to be. You start paying more attention to the syntactical constructs you create via s-exprs and macros, and less to the lexical form of the text of lists and parenthesis.
This is specially true if you take advantage of a good editor that helps with the indentation and syntax coloring (try setting the parenthesis to a color very similar to the background).
You might not be able to replace the language completely and get 'Ruby' syntax, but you don't need it. Thanks to the language flexibility you could end having a dialect that feels like you are following the 'Ruby style of programming' if you want, whatever that would mean to you.
I know this is just an empirical observation, but I think I had one of those Lisp enlightenment moments when I realized this.
Over and over again, newcomers to Lisp want to "get rid of all the parenthesis." It lasts for a few weeks. No project to build a serious general purpose programming syntax on top of the usual S-expression parser ever gets anywhere, because programmers invariably wind up preferring what you currently perceive as "parenthesis hell." It takes a little getting used to, but not much! Once you do get used to it, and you can really appreciate the plasticity of the default syntax, going back to languages where there's only one way to express any particular programming construct is really grating.
That being said, Lisp is an excellent substrate for building Domain Specific Languages. Just as good as, if not better than, XML.
Good luck!
The best explanation of Lisp macros I have ever seen is at
https://www.youtube.com/watch?v=4NO83wZVT0A
starting at about 55 minutes in. This is a video of a talk given by Peter Seibel, the author of "Practical Common Lisp", which is the best Lisp textbook there is.
The motivation for Lisp macros is usually hard to explain, because they really come into their own in situations that are too lengthy to present in a simple tutorial. Peter comes up with a great example; you can grasp it completely, and it makes good, proper use of Lisp macros.
You asked: "could you change the functional nature of LISP into a more object oriented syntax and semantics". The answer is yes. In fact, Lisp originally didn't have any object-oriented programming at all, not surprising since Lisp has been around since way before object-oriented programming! But when we first learned about OOP in 1978, we were able to add it to Lisp easily, using, among other things, macros. Eventually the Common Lisp Object System (CLOS) was developed, a very powerful object-oriented programming system that fits elegantly into Lisp. The whole thing can be loaded as an extension -- nothing is built-in! It's all done with macros.
Lisp has an entirely different feature, called "reader macros", that can be used to extend the surface syntax of the language. Using reader macros, you can make sublanguages that have C-like or Ruby-like syntax. They transform the text into Lisp, internally. These are not used widely by most real Lisp programmers, mainly because it is hard to extend the interactive development environment to understand the new syntax. For example, Emacs indentation commands would be confused by a new syntax. If you're energetic, though, Emacs is extensible too, and you could teach it about your new lexical syntax.
Regular macros operate on lists of objects. Most commonly, these objects are other lists (thus forming trees) and symbols, but they can be other objects such as strings, hashtables, user-defined objects, etc. These structures are called s-exps.
So, when you load a source file, your Lisp compiler will parse the text and produce s-exps. Macros operate on these. This works great and it's a marvellous way to extend the language within the spirit of s-exps.
Additionally, the aforementioned parsing process can be extended through "reader macros" that let you customize the way your compiler turns text into s-exps. I suggest, however, that you embrace Lisp's syntax instead of bending it into something else.
You sound a bit confused when you mention Lisp's "functional nature" and Ruby's "object-oriented syntax". I'm not sure what "object-oriented syntax" is supposed to be, but Lisp is a multi-paradigm language and it supports object-oriented programming extremelly well.
BTW, when I say Lisp, I mean Common Lisp.
I suggest you put your prejudices away and give Lisp an honest go.
Parenthesis hell? I see no more parenthesis in:
(function toto)
than in:
function(toto);
And in
(if tata (toto)
(titi)
(tutu))
no more than in:
if (tata)
toto();
else
{
titi();
tutu();
}
I see less brackets and ';' though.
What you are asking is somewhat like asking how to become an expert chocolatier so that you can remove all that hellish brown stuff from your favourite chocolate cake.
Yes, you can fundamentally change the syntax, and even escape "the parentheses hell". For that you will need to define a new reader syntax. Look into reader macros.
I do suspect however that to reach the level of Lisp expertise to program such macros you will need to immerse yourself in the language to such an extent that you will no longer consider parenthese "hell". I.e. by the time you know how to avoid them, you will have come to accept them as a good thing.
If you want lisp to look like Ruby use Ruby.
It's possible to use Ruby (and Python) in a very lisp like way which is one of the main reasons they have gained acceptance so quickly.
see this example of how reader macros can extend the lisp reader with complex tasks like XML templating:
http://common-lisp.net/project/cl-quasi-quote/present-class.html
this user library compiles the static parts of the XML into UTF-8 encoded literal byte arrays at compile time that are ready to be write-sequence'd into the network stream. and they are usable in normal lisp macros, they are orthogonal... the placement of the comma character influences which parts are constant and which should be evaluated at runtime.
more details available at: http://common-lisp.net/project/cl-quasi-quote/
another project that for Common Lisp syntax extensions: http://common-lisp.net/project/cl-syntax-sugar/
#sparkes
Sometimes LISP is the clear language choice, namely Emacs extensions. I'm sure I could use Ruby to extend Emacs if I wanted to, but Emacs was designed to be extended with LISP, so it seems to make sense to use it in that situation.
It's a tricky question. Since lisp is already structurally so close to a parse tree the difference between a large number of macros and implementing your own mini-language in a parser generator isn't very clear. But, except for the opening and closing paren, you could very easily end up with something that looks nothing like lisp.
One of the uses of macros that blew my mind was the compile-time verification of SQL requests against DB.
Once you realize you have the full language at hand at compile-time, it opens up interesting new perspectives. Which also means you can shoot yourself in the foot in interesting new ways (like rendering compilation not reproducible, which can very easily turn into a debugging nightmare).