Note: I am not asking which to learn, which is better, or anything like that.
I picked up the free version of SICP because I felt it would be nice to read (I've heard good stuff about it, and I'm interested in that sort of side of programming).
I know Scheme is a dialect of Lisp and I wondered: what is the actual difference is between Scheme and, say, Common Lisp?
There seems to be a lot about 'CL has a larger stdlib...Scheme is not good for real-world programming..' but no actual thing saying 'this is because CL is this/has this'.
This is a bit of a tricky question, since the differences are both technical and (more importantly, in my opinion) cultural. An answer can only ever provide an imprecise, subjective view. This is what I'm going to provide here. For some raw technical details, see the Scheme Wiki.
Scheme is a language built on the principle of providing an elegant, consistent, well thought-through base language substrate which both practical and academic application languages can be built upon.
Rarely will you find someone writing an application in pure R5RS (or R6RS) Scheme, and because of the minimalistic standard, most code is not portable across Scheme implementations. This means that you will have to choose your Scheme implementation carefully, should you want to write some kind of end-user application, because the choice will largely determine what libraries are available to you. On the other hand, the relative freedom in designing the actual application language means that Scheme implementations often provide features unheard of elsewhere; PLT Racket, for example, enables you to make use of static typing and provides a very language-aware IDE.
Interoperability beyond the base language is provided through the community-driven SRFI process, but availability of any given SRFI varies by implementation.
Most Scheme dialects and libraries focus on functional programming idioms like recursion instead of iteration. There are various object systems you can load as libraries when you want to do OOP, but integration with existing code heavily depends on the Scheme dialect and its surrounding culture (Chicken Scheme seems to be more object-oriented than Racket, for instance).
Interactive programming is another point that Scheme subcommunities differ in. MIT Scheme is known for strong interactivitiy support, while PLT Racket feels much more static. In any case, interactive programming does not seem to be a central concern to most Scheme subcommunities, and I have yet to see a programming environment similarly interactive as most Common Lisps'.
Common Lisp is a battle-worn language designed for practical programming. It is full of ugly warts and compatibility hacks -- quite the opposite of Scheme's elegant minimalism. But it is also much more featureful when taken for itself.
Common Lisp has bred a relatively large ecosystem of portable libraries. You can usually switch implementations at any time, even after application deployment, without too much trouble. Overall, Common Lisp is much more uniform than Scheme, and more radical language experiments, if done at all, are usually embedded as a portable library rather than defining a whole new language dialect. Because of this, language extensions tend to be more conservative, but also more combinable (and often optional).
Universally useful language extensions like foreign-function interfaces are not developed through formal means but rely on quasi-standard libraries available on all major Common Lisp implementations.
The language idioms are a wild mixture of functional, imperative, and object-oriented approaches, and in general, Common Lisp feels more like an imperative language than a functional one. It is also extremely dynamic, arguably more so than any of the popular dynamic scripting languages (class redefinition applies to existing instances, for example, and the condition handling system has interactivity built right in), and interactive, exploratory programming is an important part of "the Common Lisp way." This is also reflected in the programming environments available for Common Lisp, practically all of which offer some sort of direct interaction with the running Lisp compiler.
Common Lisp features a built-in object system (CLOS), a condition handling system significantly more powerful than mere exception handling, run-time patchability, and various kinds of built-in data structures and utilites (including the notorious LOOP macro, an iteration sublanguage much too ugly for Scheme but much too useful not to mention, as well as a printf-like formatting mechanism with GOTO support in format strings).
Both because of the image-based, interactive development, and because of the larger language, Lisp implementations are usually less portable across operating systems than Scheme implementations are. Getting a Common Lisp to run on an embedded device is not for the faint of heart, for example. Similarly to the Java Virtual Machine, you also tend to encounter problems on machines where virtual memory is restricted (like OpenVZ-based virtual servers). Scheme implementations, on the other hand, tend to be more compact and portable. The increasing quality of the ECL implementation has mitigated this point somewhat, though its essence is still true.
If you care for commercial support, there are a couple of companies that provide their own Common Lisp implementations including graphical GUI builders, specialized database systems, et cetera.
Summing up, Scheme is a more elegantly designed language. It is primarily a functional language with some dynamic features. Its implementations represent various incompatible dialects with distinctive features. Common Lisp is a fully-fledged, highly dynamic, multi-paradigm language with various ugly but pragmatic features, whose implementations are largely compatible with one another. Scheme dialects tend to be more static and less interactive than Common Lisp; Common Lisp implementations tend to be heavier and trickier to install.
Whichever language you choose, I wish you a lot of fun! :)
Some basic practical differences:
Common Lisp has separate scopes for variables and functions; whereas in Scheme there is just one scope -- functions are values and defining a function with a certain name is just defining a variable set to the lambda. As a result, in Scheme you can use a function name as a variable and store or pass it to other functions, and then perform a call with that variable as if it were a function. But in Common Lisp, you need to explicitly convert a function into a value using (function ...), and explicitly call a function stored in a value using (funcall ...)
In Common Lisp, nil (the empty list) is considered false (e.g. in if), and is the only false value. In Scheme, the empty list is considered true, and (the distinct) #f is the only false value
That's a hard question to answer impartially, especially because many of the LISP folks would classify Scheme as a LISP.
Josh Bloch (and this analogy may not be his invention) describes choosing a language as being akin to choosing a local pub. In that light, then:
The "Scheme" pub has a lot of programming-languages researchers in it. These people spend a lot of attention on the meaning of the language, on keeping it well-defined and simple, and on discussing innovative new features. Everyone's got their own version of the language, designed to allow them to explore their own particular corner of programming languages. The Scheme people really like the parenthesized syntax that they took from LISP; it's flexible and lightweight and uniform and removes many barriers to language extension.
The "LISP" pub? Well... I shouldn't comment; I haven't spent enough time there :).
scheme:
orginally very few specifications (new R7RS seems to be heavier)
due to the easy syntax, scheme can be learned quickly
implementations provide additional functions, but names can differ in different implementations
common lisp:
many functions are defined by the bigger specification
different namespace for functions and variables (lisp-2)
that are some points, sure there are many more, which i don't remember right now.
Related
My understanding of the history of Scheme macros is as follows:
R4RS introduced macros in the appendix. This was an expansion to the language rather than a part of the standard. It is essentially a library that appears in the R4RS document.
R5RS used a similar system to R4RS, but it was entirely a part of the standard rather than an extension.
R6RS and R7RS (small) have made minor upgrades/changes, but nothing substantial.
Any Scheme before R4RS did not feature any macros, except for fringe external libraries.
Is this correct? It seems generally right when I search through R4RS and anything after, but I'm struggling to find an easily-searchable copy of R3RS.
Scheme implementations had macros from the beginning, but the first real standard was R2RS, which did not include a macro system in the specification.
The original SCHEME paper, "SCHEME: An Interpreter for Extended Lambda Calculus" (AIM-349, 1975), also known as R0RS, presented a SCHEME interpreter written in MacLISP. Even this early description of a SCHEME interpreter discusses macros:
...SCHEME has a class of primitives known as AMACROS. These are similar to MacLISP MACROs, in that they are expanded into equivalent code before being executed.
The section goes on to describe some AMACROS which are provided with the SCHEME interpreter, including COND, AND, OR, BLOCK, and DO.
At the end of the paper a simple implementation of a SCHEME interpreter is presented, but the authors point out that "the 'production version' of SCHEME is coded somewhat more intricately." After presenting this implementation, they write:
AMACROs are fairly complicated beasties, and have very little to do with the basic issues of the implementation of SCHEME per se, so the code for them will not be given here. AMACROs behave almost exactly like MacLISP macros.
In R1RS, "The Revised Report on SCHEME: A Dialect of Lisp" (AIM-452, 1978), one finds:
C. Syntactic Extensions
SCHEME has a syntactic extension mechanism which provides a way to define an identifier to be a magic
word, and to associate a function with that word. The function accepts
the magic form as an argument, and produces a new form; this new form
is then evaluated in place of the original (magic) form. This is
precisely the same as the MacLISP macro facility.
These early macro systems were more traditional Lisp macro systems, akin to Common Lisp macros. Hygienic macros were introduced as a language extension in R4RS (1991), but not made a part of the Standard until R5RS (1998).
The early reports R0RS and R1RS were not really language standards so much as reports on the ongoing development of Scheme at MIT. In the introduction to R2RS (AIM-848, 1985) the authors say that 15 representatives of the major Scheme implementations convened to create a new standard because the implementations were starting to become too divergent. There was no macro system included in this new standard specification, but R2RS does say something about macros, including that all of the major implementations of the time did have macro systems:
Scheme does not have any standard facility for defining new special forms.
Rationale: The ability to define new special forms creates numerous problems. All current implementations of Scheme have macro facilities that solve those problems to one degree or another, but the solutions are quite different and it isn't clear at this time which solution is best....
R3RS (1986) had some similar language:
Macros
Scheme does not have any standard facility for defining new kinds of expressions. The ability to alter the syntax of the language creates numerous problems. All current implementations of Scheme have macro facilities that solve those problems to one degree or another, but the solutions are quite different and it isn’t clear at this time which solution is best, or indeed whether any of the solutions are truly adequate. Rather than standardize, we are encouraging implementations to continue to experiment with different solutions.
So I would say it is unfair to characterize macro systems prior to R4RS as "fringe libraries," and rather more fair to say the Scheme macro systems of the time were implementation-specific. Once macro hygiene was identified as an important issue, there was a concerted effort find the best solutions for hygienic macros within the Scheme community, and a fair bit of disagreement. This is the situation described in the above quotes from R2RS and R3RS with "...it isn’t clear at this time which solution is best...."
AIM-349 and AIM-452 (R0RS and R1RS) were really reports on the development of Scheme at MIT, and R2RS was the first real attempt at bringing disparate implementations together under a standard, i.e., R2RS was the first real standard for Scheme. Macros were present in the early implementations described by AIM-349 and AIM-452, but macros were left out of the standards to which implementations were required to adhere beginning with R2RS, until something approaching consensus was achieved on the topic; each implementation had its own approach to macros.
Resources
R3RS. This is a searchable pdf version, better quality than most of the scans I have seen online.
Hygienic Macro Technology. This is a pretty long paper by William Clinger and Mitchell Wand that describes the history of the development of hygienic macros.
While reading Peter Seibel's "Practical Common Lisp", I learned that aside from the core parts of the language like list processing and evaluating, there are macros like loop, do, etc that were written using those core constructs.
My question is two-fold. First is what exactly is the "core" of Lisp? What is the bare minimum from which other things can be re-created, if needed? The second part is where can one look at the code of the macros which come as part of Common Lisp, but were actually written in Lisp? As a side question, when one writes a Lisp implementation, in what language does he do it?
what exactly is the "core" of Lisp? What is the bare minimum from which other things can be re-created, if needed?
The minimum set of syntactic operators were called "special forms" in CLtL. The term was renamed to "special operators" in ANSI CL. There are 24 of them. This is well explained in CLtL section "Special Forms". In ANSI CL they are 25.
where can one look at the code of the macros which come as part of Common Lisp, but were actually written in Lisp?
Many Common Lisp implementations are free software (list); you can look at their source code. For example, here for SBCL, here for GNU clisp, etc.
when one writes a Lisp implementation, in what language does he do it?
Usually a Lisp implementation consists of
a lower-level part, written in a system programming language. This part includes the implementation of the function call mechanism and of the runtime part of the 24 special forms. And
a higher-level part, for which Lisp itself is used because it would be too tedious to write everything in the system programming language. This usually includes the macros and the compiler.
The choice of the system programming language depends. For implementations that are built on top of the Java VM, for example, the natural choice is Java. For implementations that include their own memory management, it is often C, or some Lisp extensions with similar semantics than C (i.e. where you have fixed-width integer types, explicit pointers etc.).
First is what exactly is the "core" of Lisp? What is the bare minimum from which other things can be re-created, if needed?
Most Lisps have a core of primitive constructs, which is usually written in C (or maybe assembly). The usual reason for choosing those languages is performance. The bare minimum from which other things can be re-created depends on how bare-minimum you want to go. That is to say, you don't need much to be Turing-complete. You really only need lambdas for your language to have a bare minimum, from which other things can be created. Though, typically, people also include defmacro, cond, defun, etc. Those things aren't strictly necessary, but are probably what you mean by "bare minimum" and what people usually include as primitive language constructs.
The second part is where can one look at the code of the macros which come as part of Common Lisp, but were actually written in Lisp?
Typically, you look in in the Lisp sources of the language. Sometimes, though, your macro is not a genuine macro and is a primitive language construct. For such things, you may also need to look in the C sources to see how these really primitive things are implemented.
Of course, if your Lisp implementation is not open-source, you need to disassemble its binary files and look at them piece-by-piece in order to understand how primitives are implemented.
As a side question, when one writes a Lisp implementation, in what language does he do it?
As I said above, C is a common choice, and assembly used to be more common. Though, there are Lisps written in high-level languages like Ruby, Python, Haskell, and even Lisp itself. The trade-off here is performance vs. readability and comprehensibility.
If you want a more-or-less canonical example of a Lisp to look at which is totally open-source, check out Emacs' source code. Of course, this isn't Common Lisp, although there is a cl package in the Emacs core which implements a fairly large subset of Common Lisp.
I'm working towards my Ph.D regarding better software reuse by integrating different types of computer languages. Due to performance and safety issues I don't consider to integrate them with foreign function calls and/or the use of web services.
Lisp is my favorite vehicle, because of interactive development, macros, doing modifications at runtime, code as data (the usual things one would imagine hearing the word Lisp), and others.
There are some approaches to port different types of Lisp to virtual machines like the JVM (clojure, kawa, SISC, ABCL, etc.) or .NET (clojure .NET, DotLisp, IronLisp). This is quite interesting, but one is restricted to the "universe" of the respective virtual machine.
Does anybody know of approaches the other way round, i.e. running Java or C# on a Lisp system? I have found the rest of cloak. It seem to be more or less a dead project. To me it would be much more sensible to have Lisp as a common abstraction, hosting other languages like Java and C#.
Which obstacles do you see to overcome this lack of a generic and extendable "language environment" integrating languages like Java or C# (without foreign function calls or (web) services))? Is it due to the fact that no Lisp system is running on a kind of a virtual machine, like the LLVM for instance, or what else?
Best regards, Ingmar
Lisp is a good platform for this kind of language hosting because of its macro capabilities. However, you want many more language features to do it well: modules, reader macros, high-level macro specification, and so on. Racket is one Lisp variant that's going forward in this direction. You can already use Algol 60, a variant of Prolog, a typed sister language, and so on. Guile is also moving in this direction with an ECMAScript implementation.
As far as implementing Java or C# on Lisp, it is possible in theory but it would require a massive amount of work to support these languages at a practical level (Racket used to implement a small portion of Java). It's also not clear that you would really gain anything considering that the CLR and JVM are both multi-language platforms now. What is more interesting is harnessing Lisp macros to define better custom languages (DSLs), defining useful dialects of your Lisp, or implementing another language specifically to bootstrap a useful tool (e.g., Guile implementing Emacs Lisp).
well "it depends", as always, right?
How much of Lisp do you want expose to Java, if any? For example, if you port the JVM to Lisp, do you somehow mate the JVMs need for a garbage collector to the actual underlying GC of the Lisp implementation, or do you simply write your own that GCs the JVM objects within the JVM heap.
It may very well be impractical to mate the two, for several reasons. The Lisp GC is pretty much hidden, much like Javas GC, from the actual implementation. That may be too hidden to work with a JVM implementation.
There's no reason you can't build a JVM in Lisp, it's just a bunch of byte codes. Lisp handles bytes just fine.
There have been implementations of the JVM in JavaScript, it's not much different than a Lisp at its core.
But beyond having a lispy command line to interact with the JVM, the JVM itself wouldn't be very "lispy". How could it be? It's a JAVA VM. The IMPLEMENTATION can be "lispy", but, ideally, none of that lisp-ness would bubble out to the JVM itself.
Beyond any advantages Lisp has in developing ANY program, I don't think Lisp lends itself specifically to being "better" to developing a virtual machine.
Lisp is great at developing other languages, particularly other S-Exp based languages. But a VM is a VM. Monster case statement or some other dispatch base on numeric values mechanism.
Lisp is a perfect host language for such a meta-platform, but it is not necessarily an ideal target language for compiling something low level and imperative. Fortunately, nothing stops you from generating, say, an assembly code within your Lisp environment.
Paul Graham writes:
For example, types seem to be an
inexhaustible source of research
papers, despite the fact that static
typing seems to preclude true macros--
without which, in my opinion, no
language is worth using.
What's the big deal with macros? I haven't spent a whole lot of time with them, but from the legacy C/C++ I've worked with they appear to be mostly used as a hack before templates/generics existed.
It's hard to imagine that
DECLARELIST(StrList, string);
StrList slist;
is somehow preferable to
List<String> slist;
Am I missing something?
Then there's the usage as a pseudo-function, like MAKEPOINTS:
POINTS MAKEPOINTS(
DWORD dwValue
);
Why not define it as a function instead? Is this some optimization, where you avoid code duplication without having the added overhead of another stack frame?
Then there's also tricky control flow things involving GOTO, which seem to be of dubious value.
What's so great about macros? They're less type safe (in C and C++) (right?). Why won't Paul Graham program without them?
LISP macros are an entirely different beast. C/C++ macros can merely replace a piece of text with abother piece of text using an extremely basic language. Whereas a LISP program is (after "reading") is a LISP data structure and can therefore be manipulated using the whole language.
With such macros, you could (given you're a really clever hacker) vastly extend the language and everybody could use it relatively easily, since you did it with macros. Take for example the the Common Lisp Object System. At its core, the language has nothing even remotely like objects. It is entirely implemented in the language itself, including a relatively simple syntax for use - using macros.
Of course macros are less necessary when the language has most things you'd every want built-in. OTOH, the LISP fans are of the opinion that a sufficiently simple language (LISP) with sufficiently powerful metaprogramming capabilities (macros) is better since new concepts can be incorporated into the language without changing the spec or working implementations. But the most compelling example for macro usage is the DSL area. Ruby on Rails and others show every day how useful DSLs can be. Yes, Ruby doesn't have macros, it just exploits how much Ruby syntax can be bent. In other languages, or when even Ruby's syntax isn't flexible enough, you need macros or a fully-blown parser/interpreter to implement a complex DSL.
Macros are really only good for two things in C/C++, and should generally be the tool of last resort (if you can accomplish something without using macros, do so).
Creating new syntactic structures or abstractions that do not exist in the language.
Eliminating duplication, especially between things that must be in sync with each other.
It's almost never to use a macro as a function.
You also have to realize that LISP macros are not C/C++ macros.
I was working with a Lisp dialect but also learning some Haskell as well. They share some similarities but the main difference in Common Lisp seems to be that you don't have to define a type for each function, argument, etc. whereas in Haskell you do. Also, Haskell is mostly a compiled language. Run the compiler to generate the executable.
My question is this, are there different applications or uses where a language like Haskell may make more sense than a more dynamic language like Common Lisp. For example, it seems that Lisp could be used for more bottom programming, like in building websites or GUIs, where Haskell could be used where compile time checks are more needed like in building TCP/IP servers or code parsers.
Popular Lisp applications:
Emacs
Popular Haskell applications:
PUGS
Darcs
Do you agree, and are there any studies on this?
Programming languages are tools for thinking with. You can express any program in any language, if you're willing to work hard enough. The chief value provided by one programming language over another is how much support it gives you for thinking about problems in different ways.
For example, Haskell is a language that emphasizes thinking about your problem in terms of types. If there's a convenient way to express your problem in terms of Haskell's data types, you'll probably find that it's a convenient language to write your program in.
Common Lisp's strengths (which are numerous) lie in its dynamic nature and its homoiconicity (that is, Lisp programs are very easy to represent and manipulate as Lisp data) -- Lisp is a "programmable programming language". If your program is most easily expressed in a new domain-specific language, for example, Lisp makes it very easy to do that. Lisp (and other dynamic languages) are a good fit if your problem description deals with data whose type is poorly specified or might change as development progresses.
Language choice is often as much an aesthetic decision as anything. If your project requirements don't limit you to specific languages for compatibility, dependency, or performance reasons, you might as well pick the one you feel the best about.
You're opening multiple cans of very wriggly worms. First off, the whole strongly vs weakly typed languages can. Second, the functional vs imperative language can.
(Actually, I'm curious: by "lisp dialect" do you mean Clojure by any chance? Because it's largely functional and closer in some ways to Haskell.)
Okay, so. First off, you can write pretty much any program in pretty much any normal language, with more or less effort. The purported advantage to strong typing is that a large class of errors can be detected at compile time. On the other hand, less typeful languages can be easier to code in. Common Lisp is interesting because it's a dynamic language with the option of declaring and using stronger types, which gives the CL compiler hints on how to optimize. (Oh, and real Common Lisp is usually implemented with a compiler, giving you the option of compiling or sticking with interpreted code.)
There are a number of studies about comparing untyped, weakly typed, and strongly typed languages. These studies invariably either say one of them is better, or say there's no observable difference. There is, however, little agreement among the studies.
The biggest area in which there may be some clear advantage is in dealing with complicated specifications for mathematical problems. In those cases (cryptographic algorithms are one example) a functional language like Haskell has advantages because it is easier to verify the correspondence between the Haskell code and the underlying algorithm.
I come mostly from a Common Lisp perspective, and as far as I can see, Common Lisp is suited for any application.
Yes, the default is dynamic typing (i.e. type detection at runtime), but you can declare types anyway for optimization (as a side note for other readers: CL is strongly typed; don't confuse weak/strong with static/dynamic!).
I could imagine that Haskell could be a bit better suited as a replacement for Ada in the avionics sector, since it forces at least all type checks at compile time.
I do not see how CL should not be as useful as Haskell for TCP/IP servers or code parsers -- rather the opposite, but my contacts with Haskell have been brief so far.
Haskell is a pure functional language. While it does allow imperative constructs (using monads), it generally forces the programmer to think the problem in a rather different way, using a more mathematical-oriented approach. You can't reassign another value to a variable, for example.
It is claimed that this reduces the probability of making some types of mistakes. Moreover, programs written in Haskell tend to be shorter and more concise than those written in typical programming languages. Haskell also makes heavy use of non-strict, lazy evaluation, which could theoretically allow the compiler to make optimizations not otherwise possible (along with the no-side-effects paradigm).
Since you asked about it, I believe Haskell's typing system is quite nice and useful. Not only it catches common errors, but it can also make code more concise (!) and can effectively replace object-oriented constructs from common OO languages.
Some Haskell development kits, like GHC, also feature interactive environments.
The best use for dynamic typing that I've found is when you depend on things that you have no control over so it could as well be used dynamically. For example getting information from XML document we could do something like this:
var volume = parseXML("mydoc.xml").speaker.volume()
Not using duck typing would lead to something like this:
var volume = parseXML("mydoc.xml").getAttrib["speaker"].getAttrib["volume"].ToString()
The benefit of Haskell on the other hand is in safety. You can for example make sure, using types, that degrees in Fahrenheit and Celsius are never mixed unintentionally. Besides that I find that statically typed languages have better IDEs.