(solved, see the comments)
Recently I've been working on an API that has to interface with an already existing service. Everything seems to be working quite well, and my project is just starting to get large enough that I would see some benefit from throwing things into a package. As this is my first "real" project in CL, I think I'm not fully understanding the packaging/loading mechanisms going on here.
My basic problem is that I have a bunch of code that uses macros to generate functions/classes, interns them into my package, and then exports certain functions/accessors that people will eventually use to interact with the API. If I load the files individually like so:
(load "~/src/lisp/cl-bitcoin/bitcoin.lisp")
(load "~/src/lisp/cl-bitcoin/classes.lisp")
(load "~/src/lisp/cl-bitcoin/functions.lisp")
Everything works well. The functions declared by my macros are interned and exported correctly and I'm able to call them to interact with the API. However, if I attempt to do the following:
(ql:quickload :btc)
Quicklisp tells me that everything was loaded properly - and it seems that most of the loading process happened as I expected since all of my dependencies are loaded and available for use. The problem is that everything related to my package is not available. This includes functions that are exported directly from my package.lisp file. For reference, here are my .asd and package files:
package.lisp
(defpackage #:btc
(:use #:cl)
(:export #:set-connection-parameters
#:reset-rpc-id
#:with-connection-parameters
#:btc-base-err
#:btc-base-id))
btc.asd
(asdf:defsystem #:btc
:serial t
:depends-on (#:drakma
#:flexi-streams
#:cl-json)
:components ((:file "package")
(:file "bitcoin")
(:file "classes")
(:file "functions")))
I feel like I'm missing something fairly obvious here - I've been looking into eval-when and other related loading functions but haven't been able to figure this out. Can someone explain to me what's going on here?
Thanks for your help.
Edit: Here's what my REPL looks like:
; SLIME 2011-02-04
CL-USER> (ql:quickload :btc)
To load "btc":
Load 1 ASDF system:
btc
; Loading "btc"
..
(:BTC)
CL-USER> (btc:help "getinfo")
Invoking restart: Return to SLIME's top level.
; Evaluation aborted on #<SIMPLE-ERROR #x30200175DF0D>.
CL-USER> ; Reader error: No external symbol named "HELP" in package #<Package "BTC"> .
; No value
CL-USER> (load "/Users/jordan/src/lisp/cl-bitcoin/bitcoin.lisp")
#P"/Users/jordan/src/lisp/cl-bitcoin/bitcoin.lisp"
CL-USER> (load "/Users/jordan/src/lisp/cl-bitcoin/classes.lisp")
#P"/Users/jordan/src/lisp/cl-bitcoin/classes.lisp"
CL-USER> (load "/Users/jordan/src/lisp/cl-bitcoin/functions.lisp")
#P"/Users/jordan/src/lisp/cl-bitcoin/functions.lisp"
CL-USER> (btc:help "getinfo")
#<BTC::BTC-SINGLE #x3020017DBDDD>
CL-USER>
And the code to generate functions:
;;;; package information
(in-package #:btc)
;;; externally visible functions - this class contains the public api for
;;; cl-bitcoin as well as the function building framework that we need to
;;; easily handle the multiple return types that are possible from the
;;; bitcoind server methods
;; function building framework - resolving function return types into
;; specific btc objects (as defined in classes.lisp)
(defun create-btc-obj (fn result err id)
(case fn
((:getbalance :help) (make-btc-single result err id))
(otherwise (error "Unable to parse function ~S to a btc object" fn))))
(defmacro defbtcfun (name &rest args)
(let ((g (gensym)) (result (gensym)) (err (gensym)) (id (gensym)))
`(progn
(defun ,name ,args
(let ((,g (intern (string ',name) :keyword)))
(multiple-value-bind (,result ,err ,id) (get-bitcoind-result ,g ,#args)
(create-btc-obj ,g ,result ,err ,id))))
(export ',name 'btc))))
;; function definitions (each of these functions should have a corresponding case in
;; create-btc-obj above, otherwise a condition will be signaled
(defbtcfun help method)
(defbtcfun getbalance account minconf)
Jordan Kaye posted this as a comment, but never turned it into an answer:
It turns out that this was just an incorrect configuration issue in
asdf by me.. I accidentally set up the project in an incorrect
directory. When loading the project, quickload was actually loading an
entirely separate .asd file than the one that I was expecting. So,
when I loaded the (correct) files manually, the code worked as
expected - quickload was loading the wrong files. Thanks a lot for
your help, it led me to figure out the cause!
Related
How can I determine where the load point is for an emacs library? For example, I'm trying to track down and remove any runtime requires of subr-x during initialization, so I'd like to know which library loaded it.
The load-history lists loaded files along with the requires they made when they were loaded, but doesn't seem to provide information about any requires that weren't evaluated initially, but may have been later.
As a simple example, if I M-xload-file "/path/to/the/following/test.el"
(defun my-f ()
(require 'misc))
(provide 'my-test)
I see the first entry in load-history is
("/path/to/test.el"
(defun . my-f)
(provide . my-test))
Then, evaluating (my-f), adds an entry for "misc.el", but there is no indication where it was loaded from (neither is the above entry updated).
How can I find that out?
How can I determine where the load point is for an emacs library?
You can't. There are many reasons an Emacs library will be loaded, for example,
autoload
C-x C-e some lisp code
M-: some lisp code
M-x load-library
For example, I'm trying to track down and remove any runtime requires of subr-x during initialization, so I'd like to know which library loaded it.
Use C-h v load-history, the order is meaningful, for example, your init file loads foo.el, and foo.el requires bar.el, then bar.el requires subr-x.el, load-history should looks like
(foo.el bar.el subr-x.el)
It's not an elegant solution, but worked for me.
As a starting point, that seems works fine for my purposes, I ended up "watching" for an initial call by load or require to a specific library. It's easy to get the name of the file where the require/load took place when an actual load is in progress and load-file-name is defined.
I was less interested in other cases, eg. interactive evaluation, but the following still works -- at least after very minimal testing, it just dumps a backtrace instead of the filename. From the backtrace, it's not hard to find the calling function, and if desired, the calling function's file could presumably be found with symbol-file.
Running the following locates loads/requires of subr-x, reporting in the message buffer the filenames of packages where it was loaded and dumping backtraces around deferred loading locations.
emacs -q -l /path/to/this.el -f find-initial-load
(require 'cl-lib)
(defvar path-to-init-file "~/.emacs.d/init.elc")
(defun find-load-point (lib &optional continue)
"During the first `require' or `load', print `load-file-name' when defined.
Otherwise, dump a backtrace around the loading call.
If CONTINUE is non-nil, don't stop after first load."
(let* ((lib-sym (intern lib))
(lib-path (or (locate-library lib) lib))
(load-syms (mapcar
(lambda (s)
(cons s (intern (format "%s#watch-%s" s lib-sym))))
'(require load)))
(cleanup (unless continue
(cl-loop for (ls . n) in load-syms
collect `(advice-remove ',ls ',n)))))
(pcase-dolist (`(,load-sym . ,name) load-syms)
(advice-add
load-sym :around
(defalias `,name
`(lambda (f sym &rest args)
(when (or (equal sym ',lib-sym)
(and (stringp sym)
(or (string= sym ,lib)
(file-equal-p sym ',lib-path))))
,#cleanup
(prin1 (or (and load-in-progress
(format "%s => %s" ',lib-sym load-file-name))
(backtrace))))
(apply f sym args)))))))
(defun find-initial-load ()
"Call with 'emacs -q -l /this/file.el -f find-initial-load'."
(find-load-point "subr-x" 'continue)
(load path-to-init-file))
;; test that deferred requires still get reported
(defun my-f () (require 'subr-x))
(add-hook 'emacs-startup-hook #'my-f)
The function find-data-from-command works fine when I run it without first doing the 'quickload' of the package. If I load the package it gives the error that split-sequence is undefined.
I have tried to reload split-sequence after loading the custom package. Doesn't work
(ql:quickload :<a-custom-package>)
(defun find-data-from-command (x desired-command)
(setq desired-data ())
(loop for line = (read-line x nil)
while (and line (not desired-data)) do
(progn
(setq commands (first (split-sequence ":" line)))
(setq data (split-sequence "," (first (rest (split-sequence ":" line)))))
(print (cons "command:" commands))
(cond
((equal commands desired-command) (return-from find-data-from-command data))))))
FIND-DATA-FROM-COMMAND
SIGMA 24 >
(setq obj-type (find-data-from-command (open "log.txt") "types"))
Error: Undefined operator SPLIT-SEQUENCE in form (SPLIT-SEQUENCE ":" LINE).
The problem is nothing to do with Quicklisp, it's to do with a package you've defined somewhere called SIGMA. In particular somewhere in your code is a form which looks like:
(defpackage "SIGMA" ;or :sigma or :SIGMA or #:sigma or ...
...
(:use ...)
...)
And then later
(in-package "SIGMA")
And the problem with this is that your package definition has an explicit (:use ...) clause.
defpackage, and the underlying function make-package has slightly interesting behaviour for the :use clause (or keyword argument in the case of make-package):
if none is given then there is an implementation-defined default;
if one is given then it overrides the default.
The idea, I think, is that implementations may want to provide a bunch of additional functionality which is available by default, and this functionality can't be in the CL package since the contents of that package is defined in the standard. So if you just say
(defpackage "FOO")
Then the implementation is allowed (and, perhaps, encouraged), to make the use-list of FOO have some useful packages in it. These packages might be the same ones that are in the default use-list of CL-USER, but I'm not sure that's required: the whole thing is somewhat under-specified.
The end result of this is that if you want to define packages which both make use of implementation-defined functionality and have explicit use-lists you have to resort to some slight trickery. How you do this is slightly up to you, but since you are by definition writing implementation-dependent code where you are defining packages like this, you probably want to make it clear that what you are doing is implementation-dependent, by some form like
(defpackage :foo
(:use ...)
#+LispWorks
(:use :lispworks :harlequin-common-lisp :cl)
...)
Or, if you just want some particular set of symbols
(defpackage :foo
(:use ...)
#+LispWorks
(:import-from :lispworks #:split-sequence))
Note that this is not quite the same thing as using a package containing the symbol.
In all these cases if your code has pretensions to be portable then there should be appropriate clauses for other implementations and a way of knowing when you're trying to run on an implemention you haven't yet seen: how to do this is outwith the scope of this answer I think.
Solved it.
The quickloading of the custom package was taking me into the package. I did not realize that. So I had to specify split-sequence is from outside. So, I replaced all occurrences of split-sequence in the function definition with
LISPWORKS:split-sequence
Then it worked.
If any one has a better solution, please do let me know. Thanks
Is there a way to temporarily import a few functions from a package into the current package, using standard common-lisp functions/macros?
I couldn't find one and had to roll my own. I'd rather not have to code anything up, or introduce another language construct, if the standard already provides such functionality.
(defmacro with-functions (functions the-package &body body)
"Allows functions in the-package to be visible only for body.
Does this by creating local lexical function bindings that redirect calls
to functions defined in the-package"
`(labels
,(mapcar (lambda (x) `(,x (&rest args)
(apply (find-symbol ,(format nil "~:#(~a~)" x)
,the-package)
args)))
functions)
,#body))
Example usage:
(defclass-default test-class ()
((a 5 "doc" )
(b 4 "doc")))
#<STANDARD-CLASS TEST-CLASS>
CL-USER>
(with-functions (class-direct-slots slot-definition-name) 'sb-mop
(with-functions (slot-definition-initform) 'sb-mop
(slot-definition-initform
(car (class-direct-slots (find-class 'test-class))))))
5
CL-USER>
EDIT: Incorporated some of Rainer's suggestions to the macro.
I decided to keep the run-time lookup capability, at the time cost of the run-time lookup to find the function in the package.
I tried to write a with-import macro that used shadowing-import and unintern, but I couldn't get it to work. I had issues with the reader saying that the imported functions didn't exist yet (at read time) before the code that imported the functions was evaluated.
I think getting it to work with shadowing-import and unintern is a better way to go, as this would be much cleaner, faster (no run-time lookup capability though) and work with functions and symbols in packages.
I would be very interested to see if someone can code up a with-import macro using unintern and shadowing-import.
It makes runtime function calls much more costly: it conses an arg list, looks up a symbol in a package, calls a function through the symbol's function cell.
It only works through symbols, not lexical functions. That makes it less useful in cases, where code is generated via macros.
Its naming is confusing. 'import' is a package operation and packages deal only with symbols, not functions. You can't import a function in a package, only a symbol.
(labels ((foo () 'bar))
(foo))
The lexical function name FOO is only in the source code a symbol. There is no way to access the function through its source symbol later (for example by using (symbol-function 'foo)). If a compiler will compile above code, it does not need to keep the symbol - it is not needed other than for debugging purposes. Your call to APPLY will fail to find any function created by LABELS or FLET.
Your macro does not import a symbol, it creates a local lexical function binding.
For slightly similar macros see CL:WITH-SLOTS and CL:WITH-ACCESSORS. Those don't support runtime lookup, but allow efficient compilation.
Your macro does not nest like this (here using "CLOS" as a package, just like your "SB-MOP"):
(defpackage "P1" (:use "CL"))
(defpackage "P2" (:use "CL"))
(with-import (p1::class-direct-slots) 'CLOS
(with-import (p2::class-direct-slots) 'P1
(p2::class-direct-slots (find-class 'test-class))))
The generated code is:
(LABELS ((P1::CLASS-DIRECT-SLOTS (&REST ARGS)
(APPLY (FIND-SYMBOL "CLASS-DIRECT-SLOTS" 'CLOS) ARGS)))
(LABELS ((P2::CLASS-DIRECT-SLOTS (&REST ARGS)
(APPLY (FIND-SYMBOL "CLASS-DIRECT-SLOTS" 'P1) ARGS)))
(P2::CLASS-DIRECT-SLOTS (FIND-CLASS 'TEST-CLASS))))
You can use import with a list of qualified symbols (i.e. package:symbol or package::symbol) you want to import and then unintern them.
I want to customize environment while the specific package is installed properly. How to check whether some package is installed in elisp?
Something like this?:
(if (require 'ecb)
(progn (setq ....))
(message "ECB not installed!"))
tripleee's answer is a handy example of error handling, but unnecessary in this instance.
(when (require 'some-library nil 'noerror)
do-things)
That 'noerror can be any non-nil value, but of course it's more descriptive this way. I often see :noerror used as well, but I've no idea if there's any particular advantage to using a keyword argument over a symbol (comments, anyone? I'm quite interested to know).
require is a built-in function in C source code.
(require FEATURE &optional FILENAME NOERROR)
If feature FEATURE is not loaded, load it from FILENAME.
If FEATURE is not a member of the list features, then the feature
is not loaded; so load the file FILENAME.
If FILENAME is omitted, the printname of FEATURE is used as the file name,
and load will try to load this name appended with the suffix .elc or
.el, in that order. The name without appended suffix will not be used.
See get-load-suffixes for the complete list of suffixes.
If the optional third argument NOERROR is non-nil,
then return nil if the file is not found instead of signaling an error.
Normally the return value is FEATURE.
The normal messages at start and end of loading FILENAME are suppressed.
The (require) will throw an error if it fails. That should really be all you need.
If you want to configure ECB's behavior only when it is available, look primarily into adding stuff to ecb-hook -- this is the normal way to configure an Emacs package conditionally.
If no hook is available, or you want to roll it by hand for some reason, try something like
(eval-after-load 'ecb '(setq ecb-be-more-like-better-yes-p t))
If you really really want to roll this by hand, you can trap the error from a failed require like this:
(condition-case nil
(progn
(require 'ecb)
(setq ecb-be-more-like-better-yes-p t) )
(file-error (message "ECB not available; not configuring") ))
Note that the condition-case will catch any file-error from inside the progn so you want to make sure you don't do any other file operations inside. Ultimately you may want to put just the require inside a condition-case and use that as the condition for your original if form, but this is already getting out of hand ...
(if (condition-case nil (require 'ecb) (error nil))
(setq ecb-be-more-like-better-yes-p t)
(message "ECB not available; not configuring") )
Why not using "featurep"?
(featurep FEATURE &optional SUBFEATURE)
Return t if FEATURE is present in this Emacs.
(if (featurep 'ecb)
(message "ECB is there!"))
For people wondering how to check if a package.el package is installed, use package-installed-p.
Four years late on this question, but... here's a simple macro that will do this for you.
(defmacro when-feature-loaded (feature &rest body)
"Executes BODY if and only if FEATURE is loaded."
(declare (indent defun))
`(when (featurep ,module)
#,body))
For example:
(when-feature-loaded 'foo
(message "foo is loaded!"))
Here's another version with an "else" case, in case you need to handle that as well.
(defmacro if-feature-loaded (module then-form else-form)
"Executes THEN-FORM if and only if MODULE is already loaded, otherwise executes ELSE-FORM."
(declare (indent 2))
`(if (featurep ,module)
,then-form
,else-form))
The simplest answer is to use require and eval-after-load as stated in other answers.
However that is not always convenient, such as in a function called by a mode hook that wants to activate another minor mode but only if the package for it is installed. In this case featurep is relevant.
Emacs packages increasingly use autoload to improve startup time. If you test for the presence of a package by using require then you are wearing the cost of loading the file(s). If using ELPA/MELPA/Marmalade packages (available by default since version 24) then many packages may be available in an as-yet unloaded state, but a package foo with autoloads will provide a feature foo-autoloads. I wrote a function that is useful for testing whether a package is available in terms of already being loaded or set to autoload.
(defun autofeaturep (feature)
"For a feature symbol 'foo, return a result equivalent to:
(or (featurep 'foo-autoloads) (featurep 'foo))
Does not support subfeatures."
(catch 'result
(let ((feature-name (symbol-name feature)))
(unless (string-match "-autoloads$" feature-name)
(let ((feature-autoloads (intern-soft (concat feature-name "-autoloads"))))
(when (and feature-autoloads (featurep feature-autoloads))
(throw 'result t))))
(featurep feature))))
N.B.: Using (require ..) will load the package, which defeats the benefit of autoloads (deferred loading).
I use this in my code to configure functions which are autoloaded:
(when (fboundp 'some-function)
...)
Example
I have this in my init.el:
(when (fboundp 'ace-select-window)
(keymap-global-set "M-s-u" 'ace-select-window))
If you have ace-window installed, you can test this yourself by copying it into your own init file, restarting Emacs, and running:
(autoloadp (symbol-function 'ace-select-window))
It should return T. Then run the command, and execute that snippet again; it should return nil.
(newbie tip: Hit altshift:, then enter that, and press enter)
This avoids loading your package just to configure it (which is the point of autoloading).
(newbie tip nr 2: using fbound is for functions; use boundp for values)
I recently started using django-html-mumamo-mode which is part of nXhtml in emacs and everything seems to work except that when I start writing javascript code in an html page, I get the warning/error
Can't find library /usr/share/emacs/23.2/lisp/progmodes/js.el
I checked in that folder and all of the files have the .elc extension including js.elc, which is probably why emacs can't find it. Can I change something to make emacs just load the .elc file?
Edit: This continues to occur if I run M-x load-library js or M-x load-library js.elc
Edit2: I have confirmed that load-suffixes is set to ("el" "elc"), and that js.elc is in the progmodes folder, which is in load-path and that all users have read permissions for that file. I am using emacs version 23.2.1, and when I set debug-on-error to t I got a traceback, and it looks like the following part contains the error:
error("Can't find library %s" "/usr/share/emacs/23.2/lisp/progmodes/js.el")
find-library-name("/usr/share/emacs/23.2/lisp/progmodes/js.el")
find-function-search-for-symbol(js-indent-line nil "/usr/share/emacs/23.2/lisp/progmodes/js.elc")
(let* ((lib ...) (where ...) (buf ...) (pos ...)) (with-current-buffer buf (let ... ... ... ...)) (put fun (quote mumamo-evaled) t))
(if (get fun (quote mumamo-evaled)) nil (let* (... ... ... ...) (with-current-buffer buf ...) (put fun ... t)))
(unless (get fun (quote mumamo-evaled)) (let* (... ... ... ...) (with-current-buffer buf ...) (put fun ... t)))
(progn (unless (get fun ...) (let* ... ... ...)))
(if mumamo-stop-widen (progn (unless ... ...)))
(when mumamo-stop-widen (unless (get fun ...) (let* ... ... ...)))
Notably, the third line contains a reference to the correct file, but it ends up trying to load the wrong one. Has anyone seen this kind of thing before or have any idea how to fix it?
If you read the section in the Emacs manual on "How Programs Do Loading, the js.elc file should be loaded if normal library -loading commands (e.g. - "require", "autoload", "load-file", etc) are being used. Some things to do to debug this:
Does your userid have system security permissions to access the js.el file in that location?
If you type M-x emacs-version, what version of Emacs are you running?
The "load-library" command searches for lisp files in the "load-path". When you examine the contents of your load-path, is the specified directory in it?
Set the variable "debug-on-error" to "t" and re-attempt to write javascript code in an html page - when the error occurs, check the source line where the error occurs and, if it's not apparent from that what is causing the problem, post an update to your question with a few lines of the source where the error occurred as well as the stack trace that was produced by Emacs.
EDIT: Ok, now that you've added the stack trace, it's possible to see why the error is occurring. Here are the key lines from the "find-function-search-for-symbol" function (which is the function where the error is occurring in):
(when (string-match "\\.el\\(c\\)\\'" library)
(setq library (substring library 0 (match-beginning 1))))
;; Strip extension from .emacs.el to make sure symbol is searched in
;; .emacs too.
(when (string-match "\\.emacs\\(.el\\)" library)
(setq library (substring library 0 (match-beginning 1))))
(let* ((filename (find-library-name library))
In line#2, the function is setting the library name equal to the "*.elc" library name minus the "c" (e.g. it's converting it from "/usr/share/emacs/23.2/lisp/progmodes/js.elc" to "/usr/share/emacs/23.2/lisp/progmodes/js.el". Then, in line#7 of the above code, it's trying to find that source member (and failing as it doesn't exist). Looking further at the stack trace, the key line is:
(if (get fun (quote mumamo-evaled)) nil (let* (... ... ... ...) (with-current-buffer buf ...) (put fun ... t)))
which is called in the nXhtml "mumamo-funcall-evaled" function. The author of nXhtml obviously has not considered that the ".elc" file may exist but that the ".el" is not in the same directory. It appears that he used to distribute js.el with nXhtml but stopped doing so since it is now shipped with most recent Emacs distributions. So, in his environment, he probably has the ".el" files in the same directory as the ".elc" files and hasn't encountered this problem.So, you should probably do 2 things:
Notify the author of the nXhtml library so that he can fix the bug in his code.
Copy the necessary ".el" source files to "/usr/share/emacs/23.2/lisp/progmodes/" so that you don't get the error. Alternatively, you may choose to re-install js.el (and possibly some other modules) in another directory and put that directory ahead of "/usr/share/emacs/23.2/lisp/progmodes/" in your "load-path".
Doing #1 will get the problem fixed in the long-term while doing #2 should let you use nXhtml in the short-term.
Check your value of load-suffixes
C-h v load-suffixes. You probably want this to be something like (".elc" ".el"). If it is make sure that your mode hasn't set it to something weird, or bound it dynamically.