The function find-data-from-command works fine when I run it without first doing the 'quickload' of the package. If I load the package it gives the error that split-sequence is undefined.
I have tried to reload split-sequence after loading the custom package. Doesn't work
(ql:quickload :<a-custom-package>)
(defun find-data-from-command (x desired-command)
(setq desired-data ())
(loop for line = (read-line x nil)
while (and line (not desired-data)) do
(progn
(setq commands (first (split-sequence ":" line)))
(setq data (split-sequence "," (first (rest (split-sequence ":" line)))))
(print (cons "command:" commands))
(cond
((equal commands desired-command) (return-from find-data-from-command data))))))
FIND-DATA-FROM-COMMAND
SIGMA 24 >
(setq obj-type (find-data-from-command (open "log.txt") "types"))
Error: Undefined operator SPLIT-SEQUENCE in form (SPLIT-SEQUENCE ":" LINE).
The problem is nothing to do with Quicklisp, it's to do with a package you've defined somewhere called SIGMA. In particular somewhere in your code is a form which looks like:
(defpackage "SIGMA" ;or :sigma or :SIGMA or #:sigma or ...
...
(:use ...)
...)
And then later
(in-package "SIGMA")
And the problem with this is that your package definition has an explicit (:use ...) clause.
defpackage, and the underlying function make-package has slightly interesting behaviour for the :use clause (or keyword argument in the case of make-package):
if none is given then there is an implementation-defined default;
if one is given then it overrides the default.
The idea, I think, is that implementations may want to provide a bunch of additional functionality which is available by default, and this functionality can't be in the CL package since the contents of that package is defined in the standard. So if you just say
(defpackage "FOO")
Then the implementation is allowed (and, perhaps, encouraged), to make the use-list of FOO have some useful packages in it. These packages might be the same ones that are in the default use-list of CL-USER, but I'm not sure that's required: the whole thing is somewhat under-specified.
The end result of this is that if you want to define packages which both make use of implementation-defined functionality and have explicit use-lists you have to resort to some slight trickery. How you do this is slightly up to you, but since you are by definition writing implementation-dependent code where you are defining packages like this, you probably want to make it clear that what you are doing is implementation-dependent, by some form like
(defpackage :foo
(:use ...)
#+LispWorks
(:use :lispworks :harlequin-common-lisp :cl)
...)
Or, if you just want some particular set of symbols
(defpackage :foo
(:use ...)
#+LispWorks
(:import-from :lispworks #:split-sequence))
Note that this is not quite the same thing as using a package containing the symbol.
In all these cases if your code has pretensions to be portable then there should be appropriate clauses for other implementations and a way of knowing when you're trying to run on an implemention you haven't yet seen: how to do this is outwith the scope of this answer I think.
Solved it.
The quickloading of the custom package was taking me into the package. I did not realize that. So I had to specify split-sequence is from outside. So, I replaced all occurrences of split-sequence in the function definition with
LISPWORKS:split-sequence
Then it worked.
If any one has a better solution, please do let me know. Thanks
Related
In Lisp, a function's arguments are evaluated first before entering the function body. Macro arguments stay not evaluated.
But sometimes, one wants to inject code pieces stored in variables into a macro. This means evaluating the argument for the macro first, and then apply the macro-of-choice on this evaluated result.
One has to resort to
(eval `(macro ,arg))
To achieve this - but eval does not behave correctly in different environments.
The best thing would be, if one could do:
(apply macro (list arg))
or
(funcall macro arg)
But since the macro is not a function this doesn't work.
Is it possible to achieve something like this? - To circumvent that problem oder to make the macro available in the functions namespace?
Or am I missing some other ways to solve such problems?
I came to this question while trying to answer How to produce HTML from a list. but also in Generate TYPECASE with macro in common lisp, Evaluate arguments passed to a macro that generates functions in lisp, and How to convert a list to code/lambda in scheme?. But I always thought while answering them it would be good to have an apply or funcall-like function which can take macros.
It is not clear what you are trying to do, although it is almost certain that you are confused about something. In particular if you are calling eval inside macroexpansions then in almost all cases you are doing something both seriously wrong and seriously dangerous. I can't ever think of a case where I've wanted macros which expand to things including eval and I have written Lisp for a very very long time.
That being said, here is how you call the function associated with a macro, and why it is very seldom what you want to do.
Macros are simply functions whose domain and range is source code: they are compilers from a language to another language. It is perfectly possible to call the function associated with a macro, but what that function will return is source code, and what you will then need to do with that source code is evaluate it. If you want a function which deals with run-time data which is not source code, then you need that function, and you can't turn a macro into that function by some magic trick which seems to be what you want to do: that magic trick does not, and can not, exist.
So for instance if I have a macro
(defmacro with-x (&body forms)
`(let ((x 1))
,#forms))
Then I can call its macro function on a bit of source code:
> (funcall (macro-function 'with-x)
'(with-x (print "foo")) nil)
(let ((x 1)) (print "foo"))
But the result of this is another bit of source code: I need to compile or evaluate it, and nothing I can do will get around this.
Indeed in (almost?) all cases this is just the same as macroexpand-1):
> (macroexpand-1 '(with-x (print "foo")))
(let ((x 1)) (print "foo"))
t
And you can probably write macroexpand-1 in terms of macro-function:
(defun macroexpand-1/equivalent (form &optional (env nil))
(if (and (consp form)
(symbolp (first form))
(macro-function (first form)))
(values (funcall (macro-function (first form)) form env)
t)
(values form nil)))
So, if the result of calling a macro is source code, what do you do with that source code to get a result which is not source code? Well, you must evaluate it. And then, well, since the evaluator expands macros for you anyway, you might as well just write something like
(defun evaluate-with-x (code)
(funcall (compile nil `(lambda ()
(with-x ,#code)))))
So you didn't need to call the macro's function in any case. And this is not the magic trick which turns macros into functions dealing with data which is not source code: it is a terrible horror which is entirely made of exploding parts.
A concrete example: CL-WHO
It looks like this question might have its origins in this one and the underlying problem there is that that's not what CL-WHO does. In particular it is a confusion to think that something like CL-WHO is a tool for taking some kind of list and turning it into HTML. It's not: it's a tool for taking the source code of a language which is built on CL but includes a way of expressing HTML output mingled with CL code, and compiles it into CL code which will do the same thing. It happens to be the case that CL source code is expressed as lists & symbols, but CL-WHO isn't really about that: it's a compiler from, if you like, 'the CL-WHO language' to CL.
So, let's try the trick we tried above and see why it's a disaster:
(defun form->html/insane (form)
(funcall
(compile nil `(lambda ()
(with-html-output-to-string (,(make-symbol "O"))
,#form)))))
And you might, if you did not look at this too closely, think that this function does in fact do the magic trick:
> (form->html/insane '(:p ((:a :href "foo") "the foo")))
"<p></p><a href='foo'>the foo</a>"
But it doesn't. What happens if we call form->html/insane on this perfectly innocuous list:
(:p (uiop/run-program:run-program "rm -rf $HOME" :output t))
Hint: don't call form->html/insane on this list if you don't have very good backups.
CL-WHO is an implementation of a programming language which is a strict superset of CL: if you try to turn it into a function to turn lists into HTML you end up with something involving the same nuclear weapon you tinker with every time you call eval, except that nuclear weapon is hidden inside a locked cupboard where you can't see it. But it doesn't care about that: if you set it off it will still reduce everything within a few miles to radioactive ash and rubble.
So if you want a tool which will turn lists – lists which aren't source code – into HTML then write that tool. CL-WHO might have the guts of such a tool in its implemenentation, but you can't use it as it is.
And this is the same problem you face whenever you are trying to abuse macros this way: the result of calling a macro's function is Lisp source code, and to evaluate that source code you need eval or an equivalent of eval. And eval is not only not a terrible solution to almost any problem: it's also a nuclear weapon. There are, perhaps problems for which nuclear weapons are good solutions, but they are few and far between.
Reading this question got me thinking about what constitutes a valid car of an expression. Obviously, symbols and lambdas can be "called" using the usual syntax. According to the hyperspec,
function name n. 1. (in an environment) A symbol or a list (setf symbol) that is the name of a function in that environment. 2. A symbol or a list (setf symbol).
So, theoretically, (setf some-name) is a function name. I decided to give it a try.
(defun (setf try-this) ()
(format t "Don't name your functions like this, kids :)"))
((setf try-this))
(funcall '(setf try-this))
(setf (try-this))
GNU CLISP, SBCL, and ABCL will all let me define this function. However, SBCL and ABCL won't let me call it using any of the syntaxes shown in the snippet. CLISP, on the other hand, will run the first two but still errs on the third.
I'm curious about which compiler is behaving correctly. Since SBCL and ABCL agree, I would hazard a guess that a correct implementation should reject that code. As a second question, how would I call my incredibly-contrived not-useful function from the code snippet, since the things I tried above aren't working portably. Or, perhaps more usefully,
A SETF function has to take at least one argument, which is the new value to be stored in the place. It can take additional arguments as well, these will be filled in from arguments in the place expression in the call to SETF.
When you use SETF, it has to have an even number of arguments: every place you're assigning to needs a value to be assigned.
So it should be:
(defun (setf try-this) (new-value)
(format t "You tried to store ~S~%" new-value))
(setf (try-this) 3)
(funcall #'(setf try-this) 'foo)
You can't use
((setf try-this) 'bar)
because the car of a form does not contain a function name. It can only be a symbol or a lambda expression (although implementations may allow other formats as extensions).
I've written an ad hoc parser generator that creates code to convert an old and little known 7-bit character set into unicode. The call to the parser generator expands into a bunch of defuns enclosed in a progn, which then get compiled. I only want to expose one of the generated defuns--the top-level one--to the rest of the system; all the others are internal to the parser and only get called from within the dynamic scope of the top-level one. Therefore, the other defuns generated have uninterned names (created with gensym). This strategy works fine with SBCL, but I recently tested it for the first time with CLISP, and I get errors like:
*** - FUNCALL: undefined function #:G16985
It seems that CLISP can't handle functions with uninterned names. (Interestingly enough, the system compiled without a problem.) EDIT: It seems that it can handle functions with uninterned names in most cases. See the answer by Rörd below.
My questions is: Is this a problem with CLISP, or is it a limitation of Common Lisp that certain implementations (e.g. SBCL) happen to overcome?
EDIT:
For example, the macro expansion of the top-level generated function (called parse) has an expression like this:
(PRINC (#:G75735 #:G75731 #:G75733 #:G75734) #:G75732)
Evaluating this expression (by calling parse) causes an error like the one above, even though the function is definitely defined within the very same macro expansion:
(DEFUN #:G75735 (#:G75742 #:G75743 #:G75744) (DECLARE (OPTIMIZE (DEBUG 2)))
(DECLARE (LEXER #:G75742) (CONS #:G75743 #:G75744))
(MULTIPLE-VALUE-BIND (#:G75745 #:G75746) (POP-TOKEN #:G75742)
...
The two instances of #:G75735 are definitely the same symbol--not two different symbols with the same name. As I said, this works with SBCL, but not with CLISP.
EDIT:
SO user Joshua Taylor has pointed out that this is due to a long standing CLISP bug.
You don't show one of the lines that give you the error, so I can only guess, but the only thing that could cause this problem as far as I can see is that you are referring to the name of the symbol instead of the symbol itself when trying to call it.
If you were referring to the symbol itself, all your lisp implementation would have to do is lookup that symbol's symbol-function. Whether it's interned or not couldn't possibly matter.
May I ask why you haven't considered another way to hide the functions, i.e. a labels statement or defining the functions within a new package that exports only the one external function?
EDIT: The following example is copied literally from an interaction with the CLISP prompt.
As you can see, calling the function named by a gensym is working as expected.
[1]> (defmacro test ()
(let ((name (gensym)))
`(progn
(defun ,name () (format t "Hello!"))
(,name))))
TEST
[2]> (test)
Hello!
NIL
Maybe your code that's trying to call the function gets evaluated before the defun? If there's any code in the macro expansion besides the various defuns, it may be implementation-dependent what gets evaluated first, and so the behaviour of SBCL and CLISP may differ without any of them violating the standard.
EDIT 2: Some further investigation shows that CLISP's behaviour varies depending upon whether the code is interpreted directly or whether it's first compiled and then interpreted. You can see the difference by either directly loading a Lisp file in CLISP or by first calling compile-file on it and then loading the FASL.
You can see what's going on by looking at the first restart that CLISP offers. It says something like "Input a value to be used instead of (FDEFINITION '#:G3219)." So for compiled code, CLISP quotes the symbol and refers to it by name.
It seems though that this behaviour is standard-conforming. The following definition can be found in the HyperSpec:
function designator n. a designator for a function; that is, an object that denotes a function and that is one of: a symbol (denoting the function named by that symbol in the global environment), or a function (denoting itself). The consequences are undefined if a symbol is used as a function designator but it does not have a global definition as a function, or it has a global definition as a macro or a special form. See also extended function designator.
I think an uninterned symbol matches the "a symbol is used as a function designator but it does not have a global definition as a function" case for unspecified consequences.
EDIT 3: (I can agree that I'm not sure whether CLISP's behaviour is a bug or not. Someone more experienced with details of the standard's terminology should judge this. It comes down to whether the function cell of an uninterned symbol - i.e. a symbol that cannot be referred to by name, only by having a direct hold on the symbol object - would be considered a "global definition" or not)
Anyway, here's an example solution that solves the problem in CLISP by interning the symbols in a throwaway package, avoiding the matter of uninterned symbols:
(defmacro test ()
(let* ((pkg (make-package (gensym)))
(name (intern (symbol-name (gensym)) pkg)))
`(progn
(defun ,name () (format t "Hello!"))
(,name))))
(test)
EDIT 4: As Joshua Taylor notes in a comment to the question, this seems to be a case of the (10 year old) CLISP bug #180.
I've tested both workarounds suggested in that bug report and found that replacing the progn with locally actually doesn't help, but replacing it with let () does.
You can most certainly define functions whose names are uninterned symbols. For instance:
CL-USER> (defun #:foo (x)
(list x))
#:FOO
CL-USER> (defparameter *name-of-function* *)
*NAME-OF-FUNCTION*
CL-USER> *name-of-function*
#:FOO
CL-USER> (funcall *name-of-function* 3)
(3)
However, the sharpsign colon syntax introduces a new symbol each time such a form is read read:
#: introduces an uninterned symbol whose name is symbol-name. Every time this syntax is encountered, a distinct uninterned symbol is created. The symbol-name must have the syntax of a symbol with no package prefix.
This means that even though something like
CL-USER> (list '#:foo '#:foo)
;=> (#:FOO #:FOO)
shows the same printed representation, you actually have two different symbols, as the following demonstrates:
CL-USER> (eq '#:foo '#:foo)
NIL
This means that if you try to call such a function by typing #: and then the name of the symbol naming the function, you're going to have trouble:
CL-USER> (#:foo 3)
; undefined function #:foo error
So, while you can call the function using something like the first example I gave, you can't do this last one. This can be kind of confusing, because the printed representation makes it look like this is what's happening. For instance, you could write such a factorial function like this:
(defun #1=#:fact (n &optional (acc 1))
(if (zerop n) acc
(#1# (1- n) (* acc n))))
using the special reader notation #1=#:fact and #1# to later refer to the same symbol. However, look what happens when you print that same form:
CL-USER> (pprint '(defun #1=#:fact (n &optional (acc 1))
(if (zerop n) acc
(#1# (1- n) (* acc n)))))
(DEFUN #:FACT (N &OPTIONAL (ACC 1))
(IF (ZEROP N)
ACC
(#:FACT (1- N) (* ACC N))))
If you take that printed output, and try to copy and paste it as a definition, the reader creates two symbols named "FACT" when it comes to the two occurrences of #:FACT, and the function won't work (and you might even get undefined function warnings):
CL-USER> (DEFUN #:FACT (N &OPTIONAL (ACC 1))
(IF (ZEROP N)
ACC
(#:FACT (1- N) (* ACC N))))
; in: DEFUN #:FACT
; (#:FACT (1- N) (* ACC N))
;
; caught STYLE-WARNING:
; undefined function: #:FACT
;
; compilation unit finished
; Undefined function:
; #:FACT
; caught 1 STYLE-WARNING condition
I hope I get the issue right. For me it works in CLISP.
I tried it like this: using a macro for creating a function with a GENSYM-ed name.
(defmacro test ()
(let ((name (gensym)))
`(progn
(defun ,name (x) (* x x))
',name)))
Now I can get the name (setf x (test)) and call it (funcall x 2).
Yes, it is perfectly fine defining functions that have names that are unintenred symbols. The problem is that you cannot then call them "by name", since you can't fetch the uninterned symbol by name (that is what "uninterned" means, essentially).
You would need to store the uninterned symbol in some sort of data structure, to then be able to fetch the symbol. Alternatively, store the defined function in some sort of data structure.
Surprisingly, CLISP bug 180 isn't actually an ANSI CL conformance bug. Not only that, but evidently, ANSI Common Lisp is itself so broken in this regard that even the progn based workaround is a courtesy of the implementation.
Common Lisp is a language intended for compilation, and compilation produces issues regarding the identity of objects which are placed into compiled files and later loaded ("externalized" objects). ANSI Common Lisp requires that literal objects reproduced from compiled files are only similar to the original objects. (CLHS 3.2.4 Literal Objects in Compiled Files).
Firstly, according to the definition similarity (3.2.4.2.2 Definition of Similarity), the rules for uninterned symbols is that similarity is name based. If we compile code with a literal that contains an uninterned symbol, then when we load the compiled file, we get a symbol which is similar and not (necessarily) the same object: a symbol which has the same name.
What if the same uninterned symbol is inserted into two different top-level forms which are then compiled as a file? When the file is loaded, are those two similar to each other at least? No, there is no such requirement.
But it gets worse: there is also no requirement that two occurrences of the same uninterned symbol in the same form will be externalized in such a way that their relative identity is preserved: that the re-loaded version of that object will have the same symbol object in all the places where the original was. In fact, the definition of similarity contains no provision for preserving the circular structure and substructure sharing. If we have a literal like '#1=(a b . #1#), as a literal in a compiled file, there appears to be no requirement that this be reproduced as a circular object with the same graph structure as the original (a graph isomorphism). The similarity rule for conses is given as naive recursion: two conses are similar if their respective cars and cdrs are similar. (The rule can't even be evaluated for circular objects; it doesn't terminate).
That the above works is because of implementations going beyond what is required in the spec; they are providing an extension consistent with (3.2.4.3 Extensions to Similarity Rules).
Thus, purely according to ANSI CL, we cannot expect to use macros with gensyms in compiled files, at least in some ways. The expectation expressed in code like the following runs afoul of the spec:
(defmacro foo (arg)
(let ((g (gensym))
(literal '(blah ,g ,g ,arg)))
...))
(defun bar ()
(foo 42))
The bar function contains a literal with two insertions of a gensym, which according to the similarity rules for conses and symbols need not reproduce as a list containing two occurrences of the same object in the second and third positions.
If the above works as expected, it's due to "extensions to the similarity rules".
So the answer to the "Why can't CLISP ..." question is that although CLISP does provide an extension for similarity which preserves the graph structure of literal forms, it doesn't do it across the entire compiled file, only within individual top level items within that file. (It uses *print-circle* to emit the individual items.) The bug is that CLISP doesn't conform to the best possible behavior users can imagine, or at least to a better behavior exhibited by other implementations.
Is there a way to temporarily import a few functions from a package into the current package, using standard common-lisp functions/macros?
I couldn't find one and had to roll my own. I'd rather not have to code anything up, or introduce another language construct, if the standard already provides such functionality.
(defmacro with-functions (functions the-package &body body)
"Allows functions in the-package to be visible only for body.
Does this by creating local lexical function bindings that redirect calls
to functions defined in the-package"
`(labels
,(mapcar (lambda (x) `(,x (&rest args)
(apply (find-symbol ,(format nil "~:#(~a~)" x)
,the-package)
args)))
functions)
,#body))
Example usage:
(defclass-default test-class ()
((a 5 "doc" )
(b 4 "doc")))
#<STANDARD-CLASS TEST-CLASS>
CL-USER>
(with-functions (class-direct-slots slot-definition-name) 'sb-mop
(with-functions (slot-definition-initform) 'sb-mop
(slot-definition-initform
(car (class-direct-slots (find-class 'test-class))))))
5
CL-USER>
EDIT: Incorporated some of Rainer's suggestions to the macro.
I decided to keep the run-time lookup capability, at the time cost of the run-time lookup to find the function in the package.
I tried to write a with-import macro that used shadowing-import and unintern, but I couldn't get it to work. I had issues with the reader saying that the imported functions didn't exist yet (at read time) before the code that imported the functions was evaluated.
I think getting it to work with shadowing-import and unintern is a better way to go, as this would be much cleaner, faster (no run-time lookup capability though) and work with functions and symbols in packages.
I would be very interested to see if someone can code up a with-import macro using unintern and shadowing-import.
It makes runtime function calls much more costly: it conses an arg list, looks up a symbol in a package, calls a function through the symbol's function cell.
It only works through symbols, not lexical functions. That makes it less useful in cases, where code is generated via macros.
Its naming is confusing. 'import' is a package operation and packages deal only with symbols, not functions. You can't import a function in a package, only a symbol.
(labels ((foo () 'bar))
(foo))
The lexical function name FOO is only in the source code a symbol. There is no way to access the function through its source symbol later (for example by using (symbol-function 'foo)). If a compiler will compile above code, it does not need to keep the symbol - it is not needed other than for debugging purposes. Your call to APPLY will fail to find any function created by LABELS or FLET.
Your macro does not import a symbol, it creates a local lexical function binding.
For slightly similar macros see CL:WITH-SLOTS and CL:WITH-ACCESSORS. Those don't support runtime lookup, but allow efficient compilation.
Your macro does not nest like this (here using "CLOS" as a package, just like your "SB-MOP"):
(defpackage "P1" (:use "CL"))
(defpackage "P2" (:use "CL"))
(with-import (p1::class-direct-slots) 'CLOS
(with-import (p2::class-direct-slots) 'P1
(p2::class-direct-slots (find-class 'test-class))))
The generated code is:
(LABELS ((P1::CLASS-DIRECT-SLOTS (&REST ARGS)
(APPLY (FIND-SYMBOL "CLASS-DIRECT-SLOTS" 'CLOS) ARGS)))
(LABELS ((P2::CLASS-DIRECT-SLOTS (&REST ARGS)
(APPLY (FIND-SYMBOL "CLASS-DIRECT-SLOTS" 'P1) ARGS)))
(P2::CLASS-DIRECT-SLOTS (FIND-CLASS 'TEST-CLASS))))
You can use import with a list of qualified symbols (i.e. package:symbol or package::symbol) you want to import and then unintern them.
Still struggling to understand what best practices are with respect to macros. I'm attempting to write a macro which defines packages on the fly.
(defmacro def-dynamic-package (name)
`(defpackage ,(intern (string-upcase name) "KEYWORD")
(:use :common-lisp)))
This works fine only for expressions such as:
(def-dynamic-package "helloworld")
But fails miserably for something like this:
(defun make-package-from-path (path)
(def-dynamic-package (pathname-name path)))
or
(defun make-package-from-path (path)
(let ((filename (pathname-path)))
(def-dynamic-package filename)))
I understand how most basic macros work but how to implement this one escapes me.
defpackage is a macro. As such, it's expanded at compile-time, not run-time. What you want is something that is called at run-time in order to make a new package. Therefore, defpackage can't do anything for you.
Fortunately, there's also make-package, which provides defpackage's features as a function. Use it instead of defpackage.
Failure is to be expected here, because a macro is used when its argument should not be evaluated.
In your first make-package-from-path, the def-dynamic-package will receive as argument a list that is EQUAL to the value of the following expression:
(list 'pathname-name 'path)
In your case, you only want a function:
(defun def-dynamic-package (name)
(defpackage (string-upcase name)
(:use :common-lisp)))
BTW, if you check the CLHS, you'll see that the first argument of defpackage needn't be a symbol, but any string designator.