Related
In Lisp, any program's code is actually a valid data structure. For example, this adds one and two together, but it's also a list of three items.
(+ 1 2)
What benefit does that provide? What does that enable you to do that's impossible and/or less elegant in other languages?
To make things a little clearer with respect to code representation, consider that in every language code is data: all you need is strings. (And perhaps a few file operations.) Contemplating how that helps you to mimic the Lisp benefits of having a macro system is a good way for enlightenment. Even better if you try to implement such a macro system. You'll run into the advantages of having a structured representation vs the flatness of strings, the need to run the transformations first and define "syntactic hooks" to tell you where to apply them, etc etc.
But the main thing that you'll see in all of this is that macros are essentially a convenient facility for compiler hooks -- ones that are hooked on newly created keywords. As such, the only thing that is really needed is some way to have user code interact with compiler code. Flat strings are one way to do it, but they provide so little information that the macro writer is left with the task of implementing a parser from scratch. On the other hand, you could expose some internal compiler structure like the pre-parsed AST trees, but those tend to expose too much information to be convenient and it means that the compiler needs to somehow be able to parse the new syntactic extension that you intend to implement. S-expressions are a nice solution to the latter: the compiler can parse anything since the syntax is uniform. They're also a solution to the former, since they're simple structures with rich support by the language for taking them apart and re-combining them in new ways.
But of course that's not the end of the story. For example, it's interesting to compare plain symbolic macros as in CL and hygienic macros as in Scheme implementations: those are usually implemented by adding more information to the represented data, which means that you can do more with those macro system. (You can also do more with CL macros since the extra information is also available, but instead of making it part of the syntax representation, it's passed as an extra environment argument to macros.)
My favorite example... In college some friends of mine were writing a compiler in Lisp. So all their data structures including the parse tree was lisp s-expressions. When it was time to implement their code generation phase, they simply executed their parse tree.
Lisp has been developed for the manipulation of symbolic data of all kinds. It turns out that one can also see any Lisp program as symbolic data. So you can apply the manipulation capabilities of Lisp to itself.
If you want to compute with programs (to create new programs, to compile programs to machine code, to translate programs from Lisp to other languages) one has basically three choices:
use strings. This gets tedious parsing and unparsing strings all the time.
use parse trees. Useful but gets complex.
use symbolic expressions like in Lisp. The programs are preparsed into familiar datastructures (lists, symbols, strings, numbers, ...) and the manipulation routines are written in the usual language provided functionality.
So what you get with Lisp? A relatively simple way to write programs that manipulate other programs. From Macros to Compilers there are many examples of this. You also get that you can embed languages into Lisp easily.
It allows you to write macros that simply transform one tree of lists to another. Simple (Scheme) example:
(define-syntax and
(syntax-rules ()
((and) #t)
((and thing) thing)
((and thing rest ...) (if thing (and rest ...) #f))))
Notice how the macro invocations are simply matched to the (and), (and thing), and (and thing rest ...) clauses (depending on the arity of the invocation), and handled appropriately.
With other languages, macros would have to deal with some kind of internal AST of the code being transformed---there wouldn't otherwise be an easy way to "see" your code in a programmatic format---and that would increase the friction of writing macros.
In Lisp programs, macros are generally used pretty frequently, precisely because of the low friction of writing them.
I need to find out, from within a Common Lisp program, how much memory is currently being used.
I'm given to understand there is no portable method (the standard function room prints the information to standard output in text form instead of returning it as a value), but sb-kernel:dynamic-usage works in SBCL.
What are the equivalents in other Common Lisp implementations? Or is there another way to solve this problem I should be looking at?
It may not help you much, but anyway:
You can capture the output of (room) and parse it.
(with-output-to-string (*standard-output*)
(room))
Above returns a string with the output of ROOM.
Additionally it may help to request the memory size of the process via an external call to a standard unix command (if you are on Unix).
For things which virtually every implementation supports, but not in the same way (because it's not in CL), one common approach is to make a library called trivial-whatever.
If you started a package like trivial-memory, and supplied the first implementation, I'm sure we could get everybody to contribute the function for their own favorite Lisp compiler in short order. :-)
What is the minimum set of primitives required such that a language is Turing complete and a lisp variant?
Seems like car, cdr and some flow control and something for REPL is enough. It be nice if there is such list.
Assume there are only 3 types of data, integers, symbols and lists.(like in picolisp)
The lambda calculus is turing complete. It has one primitive - the lambda. Translating that to a lisp syntax is pretty trivial.
There's a good discussion of this in the Lisp FAQ. It depends on your choice of primitives. McCarthy's original "LISP 1.5 Programmer's Manual" did it with five functions: CAR, CDR, CONS, EQ, and ATOM.
I believe the minimum set is what John McCarthy published in the original paper.
The Roots of Lisp.
The code.
The best way to actually know this for sure is if you implement it. I used 3 summers to create Zozotez which is a McCarty-ish LISP running on Brainfuck.
I tried to find out what I needed and on a forum you'll find a thread that says You only need lambda. Thus, you can make a whole LISP in lambda calculus if you'd like. I found it interesting, but it's hardly the way to go if you want something that eventually has side effects and works in the real world.
For a Turing complete LISP I used Paul Grahams explanation of McCarthy's paper and all you really need is:
symbol-evaluation
special form quote
special form if (or cond)
special form lambda (similar to quote)
function eq
function atom
function cons
function car
function cdr
function-dispatch (basically apply but not actually exposed to the system so it handles a list where first element is a function)
Thats 10. In addition to this, to have a implementation that you can test and not just on a drawing board:
function read
function write
Thats 12. In my Zozotez I implemeted set and flambda (anonymous macroes, like lambda) as well. I could feed it a library implementing any dynamic bound lisp (Elisp, picoLisp) with the exception of file I/O (because the underlying BF does not support it other than stdin/stdout).
I recommend anyone to implement a LISP1-interpreter, in both LISP and (not LISP), to fully understand how a language is implemented. LISP has a very simple syntax so it's a good starting point. For all other programming languages how you implement an interpreter is very similar. Eg. in the SICP videos the wizards make an interpreter for a logical language, but the structure and how to implement it is very similar to a lisp interpreter even though this language is completely different than Lisp.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have read that most languages are becoming more and more like lisp, adopting features that lisp has had for a long time. I was wondering, what are the features, old or new, that lisp does not have? By lisp I mean the most common dialects like Common Lisp and Scheme.
This question has been asked a million times, but here goes. Common Lisp was created at a time when humans were considered cheap, and machines were considered expensive. Common Lisp made things easier for humans at the expense of making it harder for computers. Lisp machines were expensive; PCs with DOS were cheap. This was not good for its popularity; better to get a few more humans making mistakes with less expressive languages than it was to buy a better computer.
Fast forward 30 years, and it turns out that this isn't true. Humans are very, very expensive (and in very short supply; try hiring a programmer), and computers are very, very cheap. Cheaper than dirt, even. What today's world needs is exactly what Common Lisp offered; if Lisp were invented now, it would become very popular. Since it's 30-year-old (plus!) technology, though, nobody thought to look at it, and instead created their own languages with similar concepts. Those are the ones you're using today. (Java + garbage collection is one of the big innovations. For years, GC was looked down upon for being "too slow", but of course, a little research and now it's faster than managing your own memory. And easier for humans, too. How times change...)
Pass-by-reference (C++/C#)
String interpolation (Perl/Ruby) (though a feature of CL21)
Nice infix syntax (though it's not clear that it's worth it) (Python)
Monadic 'iteration' construct which can be overloaded for other uses (Haskell/C#/F#/Scala)
Static typing (though it's not clear that it's worth it) (many languages)
Type inference (not in the standard at least) (Caml and many others) (though CL does some type inference, unlike Python)
Abstract Data Types (Haskell/F#/Caml)
Pattern matching (Haskell/F#/Caml/Scala/others) (in CL, there are libs like optima)
Backtracking (though it's not clear that it's worth it) (Prolog)
ad-hoc polymorphism (see Andrew Myers' answer)
immutable data structures (many languages) (available through libraries, like Fsets
lazy evaluation (Haskell) (available through libraries, like clazy or a cl21 module)
(Please add to this list, I have marked it community wiki.)
This just refers to the Common Lisp and Scheme standards, because particular implementations have added a lot of these features independently. In fact, the question is kind of mistaken. It's so easy to add features to Lisp that it's better to have a core language without many features. That way, people can customize their language to perfectly fit their needs.
Of course, some implementations package the core Lisp with a bunch of these features as libraries. At least for Scheme, PLT Scheme provides all of the above features*, mostly as libraries. I don't know of an equivalent for Common Lisp, but there may be one.
*Maybe not infix syntax? I'm not sure, I never looked for it.
For Common Lisp, I think the following features would be worth adding to a future standard, in the ridiculously unlikely hypothetical situation that another standard is produced. All of these are things that are provided by pretty much every actively maintained CL implementation in subtly incompatible ways, or exist in widely used and portable libraries, so having a standard would provide significant benefits to users while not making life unduly difficult for implementors.
Some features for working with an underlying OS, like invoking other programs or handling command line arguments. Every implementation of CL I've used has something like this, and all of them are pretty similar.
Underlying macros or special forms for BACKQUOTE, UNQUOTE and UNQUOTE-SPLICING.
The meta-object protocol for CLOS.
A protocol for user-defined LOOP clauses. There are some other ways LOOP could be enhanced that probably wouldn't be too painful, either, like clauses to bind multiple values, or iterate over a generic sequence (instead of requiring different clauses for LISTs and VECTORs).
A system-definition facility that integrates with PROVIDE and REQUIRE, while undeprecating PROVIDE and REQUIRE.
Better and more extensible stream facilities, allowing users to define their own stream classes. This might be a bit more painful because there are two competing proposals out there, Gray streams and "simple streams", both of which are implemented by some CL implementations.
Better support for "environments", as described in CLTL2.
A declaration for merging tail calls and a description of the situations where calls that look like tail calls aren't (because of UNWIND-PROTECT forms, DYNAMIC-EXTENT declarations, special variable bindings, et c.).
Undeprecate REMOVE-IF-NOT and friends. Eliminate the :TEST-NOT keyword argument and SET.
Weak references and weak hash tables.
User-provided hash-table tests.
PARSE-FLOAT. Currently if you want to turn a string into a floating point number, you either have to use READ (which may do all sorts of things you don't want) or roll your own parsing function. This is silly.
Here are some more ambitious features that I still think would be worthwhile.
A protocol for defining sequence classes that will work with the standard generic sequence functions (like MAP, REMOVE and friends). Adding immutable strings and conses alongside their mutable kin might be nice, too.
Provide a richer set of associative array/"map" data types. Right now we have ad-hoc stuff built out of conses (alists and plists) and hash-tables, but no balanced binary trees. Provide generic sequence functions to work with these.
Fix DEFCONSTANT so it does something less useless.
Better control of the reader. It's a very powerful tool, but it has to be used very carefully to avoid doing things like interning new symbols. Also, it would be nice if there were better ways to manage readtables and custom reader syntaxes.
A read syntax for "raw strings", similar to what Python offers.
Some more options for CLOS classes and slots, allowing for more optimizations and better performance. Some examples are "primary" classes (where you can only have one "primary class" in a class's list of superclasses), "sealed" generic functions (so you can't add more methods to them, allowing the compiler to make a lot more assumptions about them) and slots that are guaranteed to be bound.
Thread support. Most implementations either support SMP now or will support it in the near future.
Nail down more of the pathname behavior. There are a lot of gratuitously annoying incompatibilities between implementations, like CLISP's insistance on signaling an error when you use PROBE-FILE on a directory, or indeed the fact that there's no standard function that tells you whether a pathname is the name of a directory or not.
Support for network sockets.
A common foreign function interface. It would be unavoidably lowest-common-denominator, but I think having something you could portably rely upon would be a real advantage even if using some of the cooler things some implementations provide would still be relegated to the realm of extensions.
This is in response to the discussion in comments under Nathan Sanders reply. This is a bit much for a comment so I'm adding it here. I hope this isn't violating Stackoverflow etiquette.
ad-hoc polymorphism is defined as different implementations based on specified types. In Common Lisp using generic methods you can define something like the following which gives you exactly that.
;This is unnecessary and created implicitly if not defined.
;It can be explicitly provided to define an interface.
(defgeneric what-am-i? (thing))
;Provide implementation that works for any type.
(defmethod what-am-i? (thing)
(format t "My value is ~a~%" thing))
;Specialize on thing being an integer.
(defmethod what-am-i? ((thing integer))
(format t "I am an integer!~%")
(call-next-method))
;Specialize on thing being a string.
(defmethod what-am-i? ((thing string))
(format t "I am a string!~%")
(call-next-method))
CL-USER> (what-am-i? 25)
I am an integer!
My value is 25
NIL
CL-USER> (what-am-i? "Andrew")
I am a string!
My value is Andrew
NIL
It can be harder than in more popular languages to find good libraries.
It is not purely functional like haskell
Whole-program transformations. (It would be just like macros, but for everything. You could use it to implement declarative language features.) Equivalently, the ability to write add-ons to the compiler. (At least, Scheme is missing this. CL may not be.)
Built-in theorem assistant / proof checker for proving assertions about your program.
Of course, I don't know of any other language that has these, so I don't think there's much competition in terms of features.
You are asking the ronge question. The language with the most features isnt the best. A language needs a goal.
We could add all of this and more
* Pass-by-reference (C++/C#)
* String interpolation (Perl/Ruby)
* Nice infix syntax (though it's not clear that it's worth it) (Python)
* Monadic 'iteration' construct which can be overloaded for other uses (Haskell/C#/F#/Scala)
* Static typing (though it's not clear that it's worth it) (many languages)
* Type inference (not in the standard at least) (Caml and many others)
* Abstract Data Types (Haskell/F#/Caml)
* Pattern matching (Haskell/F#/Caml/Scala/others)
* Backtracking (though it's not clear that it's worth it) (Prolog)
* ad-hoc polymorphism (see Andrew Myers' answer)
* immutable data structures (many languages)
* lazy evaluation (Haskell)
but that would make a good language. A language is not functional if you use call by ref.
If you look at the new list Clojure. Some of them are implemented but other that CL has are not and that makes for a good language.
Clojure for example added some:
ad-hoc polymorphism
lazy evaluation
immutable data structures
Type inference (most dynamic languages have compilers that do that)
My Answer is:
Scheme schooled stay as it is.
CL could add some ideos to the standard if they would make a new one.
Its LISP most can be added with libs.
Decent syntax. (Someone had to say it.) It may be simple/uniform/homoiconic/macro-able/etc, but as a human, I just loathe looking at it :)
It's missing a great IDE
I've heard that Lisp lets you redefine the language itself, and I have tried to research it, but there is no clear explanation anywhere. Does anyone have a simple example?
Lisp users refer to Lisp as the programmable programming language. It is used for symbolic computing - computing with symbols.
Macros are only one way to exploit the symbolic computing paradigm. The broader vision is that Lisp provides easy ways to describe symbolic expressions: mathematical terms, logic expressions, iteration statements, rules, constraint descriptions and more. Macros (transformations of Lisp source forms) are just one application of symbolic computing.
There are certain aspects to that: If you ask about 'redefining' the language, then redefine strictly would mean redefine some existing language mechanism (syntax, semantics, pragmatics). But there is also extension, embedding, removing of language features.
In the Lisp tradition there have been many attempts to provide these features. A Lisp dialect and a certain implementation may offer only a subset of them.
A few ways to redefine/change/extend functionality as provided by major Common Lisp implementations:
s-expression syntax. The syntax of s-expressions is not fixed. The reader (the function READ) uses so-called read tables to specify functions that will be executed when a character is read. One can modify and create read tables. This allows you for example to change the syntax of lists, symbols or other data objects. One can also introduce new syntax for new or existing data types (like hash-tables). It is also possible to replace the s-expression syntax completely and use a different parsing mechanism. If the new parser returns Lisp forms, there is no change needed for the Interpreter or Compiler. A typical example is a read macro that can read infix expressions. Within such a read macro, infix expressions and precedence rules for operators are being used. Read macros are different from ordinary macros: read macros work on the character level of the Lisp data syntax.
replacing functions. The top-level functions are bound to symbols. The user can change the this binding. Most implementations have a mechanism to allow this even for many built-in functions. If you want to provide an alternative to the built-in function ROOM, you could replace its definition. Some implementations will raise an error and then offer the option to continue with the change. Sometimes it is needed to unlock a package. This means that functions in general can be replaced with new definitions. There are limitations to that. One is that the compiler may inline functions in code. To see an effect then one needs to recompile the code that uses the changed code.
advising functions. Often one wants to add some behavior to functions. This is called 'advising' in the Lisp world. Many Common Lisp implementations will provide such a facility.
custom packages. Packages group the symbols in name spaces. The COMMON-LISP package is the home of all symbols that are part of the ANSI Common Lisp standard. The programmer can create new packages and import existing symbols. So you could use in your programs an EXTENDED-COMMON-LISP package that provides more or different facilities. Just by adding (IN-PACKAGE "EXTENDED-COMMON-LISP") you can start to develop using your own extended version of Common Lisp. Depending on the used namespace, the Lisp dialect you use may look slighty or even radically different. In Genera on the Lisp Machine there are several Lisp dialects side by side this way: ZetaLisp, CLtL1, ANSI Common Lisp and Symbolics Common Lisp.
CLOS and dynamic objects. The Common Lisp Object System comes with change built-in. The Meta-Object Protocol extends these capabilities. CLOS itself can be extended/redefined in CLOS. You want different inheritance. Write a method. You want different ways to store instances. Write a method. Slots should have more information. Provide a class for that. CLOS itself is designed such that it is able to implement a whole 'region' of different object-oriented programming languages. Typical examples are adding things like prototypes, integration with foreign object systems (like Objective C), adding persistance, ...
Lisp forms. The interpretation of Lisp forms can be redefined with macros. A macro can parse the source code it encloses and change it. There are various ways to control the transformation process. Complex macros use a code walker, which understands the syntax of Lisp forms and can apply transformations. Macros can be trivial, but can also get very complex like the LOOP or ITERATE macros. Other typical examples are macros for embedded SQL and embedded HTML generation. Macros can also used to move computation to compile time. Since the compiler is itself a Lisp program, arbitrary computation can be done during compilation. For example a Lisp macro could compute an optimized version of a formula if certain parameters are known during compilation.
Symbols. Common Lisp provides symbol macros. Symbol macros allow to change the meaning of symbols in source code. A typical example is this: (with-slots (foo) bar (+ foo 17)) Here the symbol FOO in the source enclosed with WITH-SLOTS will be replaced with a call (slot-value bar 'foo).
optimizations, with so-called compiler macros one can provide more efficient versions of some functionality. The compiler will use those compiler macros. This is an effective way for the user to program optimizations.
Condition Handling - handle conditions that result from using the programming language in a certain way. Common Lisp provides an advanced way to handle errors. The condition system can also be used to redefine language features. For example one could handle undefined function errors with a self-written autoload mechanism. Instead of landing in the debugger when an undefined function is seen by Lisp, the error handler could try to autoload the function and retry the operation after loading the necessary code.
Special variables - inject variable bindings into existing code. Many Lisp dialects, like Common Lisp, provide special/dynamic variables. Their value is looked up at runtime on the stack. This allows enclosing code to add variable bindings that influence existing code without changing it. A typical example is a variable like *standard-output*. One can rebind the variable and all output using this variable during the dynamic scope of the new binding will go to a new direction. Richard Stallman argued that this was very important for him that it was made default in Emacs Lisp (even though Stallman knew about lexical binding in Scheme and Common Lisp).
Lisp has these and more facilities, because it has been used to implement a lot of different languages and programming paradigms. A typical example is an embedded implementation of a logic language, say, Prolog. Lisp allows to describe Prolog terms with s-expressions and with a special compiler, the Prolog terms can be compiled to Lisp code. Sometimes the usual Prolog syntax is needed, then a parser will parse the typical Prolog terms into Lisp forms, which then will be compiled. Other examples for embedded languages are rule-based languages, mathematical expressions, SQL terms, inline Lisp assembler, HTML, XML and many more.
I'm going to pipe in that Scheme is different from Common Lisp when it comes to defining new syntax. It allows you to define templates using define-syntax which get applied to your source code wherever they are used. They look just like functions, only they run at compile time and transform the AST.
Here's an example of how let can be defined in terms of lambda. The line with let is the pattern to be matched, and the line with lambda is the resulting code template.
(define-syntax let
(syntax-rules ()
[(let ([var expr] ...) body1 body2 ...)
((lambda (var ...) body1 body2 ...) expr ...)]))
Note that this is NOTHING like textual substitution. You can actually redefine lambda and the above definition for let will still work, because it is using the definition of lambda in the environment where let was defined. Basically, it's powerful like macros but clean like functions.
Macros are the usual reason for saying this. The idea is that because code is just a data structure (a tree, more or less), you can write programs to generate this data structure. Everything you know about writing programs that generate and manipulate data structures, therefore, adds to your ability to code expressively.
Macros aren't quite a complete redefinition of the language, at least as far as I know (I'm actually a Schemer; I could be wrong), because there is a restriction. A macro can only take a single subtree of your code, and generate a single subtree to replace it. Therefore you can't write whole-program-transforming macros, as cool as that would be.
However, macros as they stand can still do a whole lot of stuff - definitely more than any other language will let you do. And if you're using static compilation, it wouldn't be hard at all to do a whole-program transformation, so the restriction is less of a big deal then.
A reference to 'structure and interpretation of computer programs' chapter 4-5 is what I was missing from the answers (link).
These chapters guide you in building a Lisp evaluator in Lisp. I like the read because not only does it show how to redefine Lisp in a new evaluator, but also let you learn about the specifications of Lisp programming language.
This answer is specifically concerning Common Lisp (CL hereafter), although parts of the answer may be applicable to other languages in the lisp family.
Since CL uses S-expressions and (mostly) looks like a sequence of function applications, there's no obvious difference between built-ins and user code. The main difference is that "things the language provides" is available in a specific package within the coding environment.
With a bit of care, it is not hard to code replacements and use those instead.
Now, the "normal" reader (the part that reads source code and turns it into internal notation) expects the source code to be in a rather specific format (parenthesised S-expressions) but as the reader is driven by something called "read-tables" and these can be created and modified by the developer, it is also possible to change how the source code is supposed to look.
These two things should at least provide some rationale as to why Common Lisp can be considered a re-programmable programming language. I don't have a simple example at hand, but I do have a partial implementation of a translation of Common Lisp to Swedish (created for April 1st, a few years back).
From the outside, looking in...
I always thought it was because Lisp provided, at its core, such basic, atomic logical operators that any logical process can be built (and has been built and provided as toolsets and add-ins) from the basic components.
It is not so much that it can redefine itself as that its basic definition is so malleable that it can take any form and that no form is assumed/presumed into the structure.
As a metaphor, if you only have organic compounds you do organic chemistry, if you only have metal oxides you do metallurgy but if you have only elements you can do everything but you have extra initial steps to complete....most of which others have already done for you....
I think.....
Cool example at http://www.cs.colorado.edu/~ralex/papers/PDF/X-expressions.pdf
reader macros define X-expressions to coexist with S-expressions, e.g.,
? (cx <circle cx="62" cy="135" r="20"/>)
62
plain vanilla Common Lisp at http://www.AgentSheets.com/lisp/XMLisp/XMLisp.lisp
...
(eval-when (:compile-toplevel :load-toplevel :execute)
(when (and (not (boundp '*Non-XMLISP-Readtable*)) (get-macro-character #\<))
(warn "~%XMLisp: The current *readtable* already contains a #/< reader function: ~A" (get-macro-character #\<))))
... of course the XML parser is not so simple but hooking it into the lisp reader is.