As a package size grows, the readability of the code decreased. The best practice is to separate the package into different files according to their functions. Sometimes, the files need variables and functions of each other.
Example:
foo.el
(defvar foo-data-path "~/foo/data/")
(foo-log-write 'somedata)
foo-log.el
(defvar foo-logfile (concat foo-data-path "foo.log"))
(defun foo-log-write (data)
;; Write to log file
)
As foo.el use function from foo-log-write, I put (require 'foo-log) in foo.el.
In foo-log.el, it uses foo-data-path from foo.el, should I also put (require 'foo) in it?
The syntax checker always complains reference to free variable if I don't add (require 'foo).
There are two techniques I've seen to handle the situation you describe. It should be noted that you can get into a circular dependency problem is your not careful. While you can have require in a file multiple times, if they are mutually dependent and if your 'provide lines are at the end, you can easily end up in a loop.
The two solutions I've used successfully are
(Preferred). Factor out the mutually dependent code into a 3rd file and have the other two require that file.
When all else fails and you cannot factor things out and cannot avoid the mutual dependency, make sure that at least one of the files has the provide statement at the beginning. This is not normally a good idea because if your file errors out before it finishes loading, you will appear to have satisfied the require, but not everything you expect to have been loaded will be.
The second technique helps break the infinite loop because you have satisfied one of the provide requirements. The problem with having the provide at the end of the file (where it normally should be) is that you do not satisfy the require request until the file has been fully loaded. However, if the file has a require for another file, it will first load that file before continuing to load the first file. If that second file also has a require and that require is to the first file, emacs will start loading the first file again until it hits the require for the second file and around the loop we go.
by putting the provide at the beginning of the file, the next time it hits a require for that file it will not try to load it as it will believe it has already been loaded. the loop is broken.
Related
Over the weekend, I had a name clash that was very hard to track down, but I managed to boil it down to a short example - thing is, I thought the package system was supposed to protect me from this, so I'm wondering how it can in future.
If I do this:
(ql:quickload "cl-irc")
(defpackage #:clash-demo
(:use #:cl
#:cl-irc))
(in-package #:clash-demo)
;; This is the name that clashes - I get a warning about this if I compile
;; this interactively (i.e. from slime) but not if I quickload the whole project.
(defun server-name (server)
(format nil "server-name ~a" server))
;; This needs an IRC server to work - if you have docker
;; then this will do the trick:
;;
;; docker run -d --rm --name ircd -p 6667:6667 inspircd/inspircd-docker
(defparameter *connection*
(cl-irc:connect :nickname "clash-demo"
:server "localhost"
:port 6667
:connection-security :none
:username "username"))
After the above, I get the following warning when defining server-name:
WARNING: redefining CL-IRC:SERVER-NAME in DEFUN
And the following error if I try and print *connection* (in my more full-fledged project I got a missing slot in a class that I'd defined - I think the root cause of both the problem I found and the minimal example above is the same though):
Control stack guard page temporarily disabled: proceed with caution
While I get the warning if I define things interactively, in practice I moved a bunch of code into a quickproject:make-project'd and ql:quickload-ed it, which I think silenced the warning as it always loaded cleanly, hence why it took me so long to track down the name clash.
My questions are:
Isn't the package system supposed to protect me from this? I think I can sort of see why the above happens - the reader's already seen the symbol server-name so it thinks I'm referring to the already defined cl-irc:server-name, and re-uses that - but surely the package system should somehow allow me to work around this?
I'm assuming the warning when I quickload-ed the project was silence because quickload assumes I don't want to see warnings from projects, is there a way I can make this more forceful when I load projects I'm making so that it raises an error, or at least warns me of these name clashes? For all I know there are a bunch more that just haven't caused me a problem yet.
I was expecting either (i) the names not to clash (i.e. my file would define the symbol clash-demo:server-name, not re-use cl-irc:server-name and cause it to be redefined) or (ii) this to be an error, or at least a warning when I quickload the project.
Thanks very much in advance for any advice!
Suppose you really like packages A and B and want to :use both of them in your package MINE. They have a hundred external symbols each and you don't want to have to type any package prefixes. However, they both export a symbol with the same name, a:frob and b:frob. If you simply :use them, you will get a symbol conflict.
To resolve the conflict, there are three options to decide what to do when you're in package MINE and you refer to the unqualified symbol frob:
Prefer A so it refers to the the symbol a:frob: (defpackage mine (:use a b) (:shadowing-import-from a frob))
Prefer B so it refers to the symbol b:frob: (defpackage mine (:use a b) (:shadowing-import-from b frob))
Prefer neither so it refers to mine::frob: (defpackage mine (:use a b) (:shadow frob)) - then, to use one from A or B you must write a:frob or b:frob explicitly
Any of these three cases may be preferable depending on your situation. Common Lisp will not automatically choose one for you. I think this is a reasonable design choice.
Briefly, no: the package system is not meant to protect you against this. If you use a package CL-IRC then you're saying that you want, for instance, to get the symbol CL-IRC:SERVER-NAME when you type server-name. What you are not saying is whether you should be allowed to modify the values associated with that symbol in any way, which is an orthogonal question. The package system is just about names, not values.
In the case of functions, then it's very often (but not always! consider loading patches) a mistake to define a function with a given name in multiple contexts. In the case of variables that's slightly less clear: it's probably a mistake if there are multiple def* forms for a given variable in different contexts, but simply assigning to the variable might well be fine.
So what is needed is a way for defining macros (defun etc) to be able to detect this and complain about it.
CL does not provide such a mechanism. Many implementations do however, either by detecting 'different contexts' (which I have been vague about) or by providing a way of saying that certain packages are sacred and redefinitions should not be allowed, or both.
In this case, the implementation has warned you about the redefinition, but Quicklisp may have suppressed that. I am not sure how to desupress warnings like this in Quicklisp.
In summary the answer is that the problem of controlling and limiting redefinition is orthogonal to what the package system does, and unfortunately CL does not provide a standard solution to this second problem.
If you are interested I have a little shim which uses the condition system to make very sure warnings are treated as errors in contexts like this. I could append it to this answer, but not until after Christmas probably.
I want to keep a list of required modules of a particular module (let's say the current-module).
I feel like there are some other options (such as parsing the module?) that could be tried, but I started playing with the idea of shadowing (require) and adding the required items to a hash-table with the module-name. The problem is I cannot figure how to write a syntax definition for it.
Although not working, a function definition equivalent would be like below:
(define require-list (make-hash))
(define require
(lambda vals
; add vals to hash-table with key (current-namespace)
(let ([cn (current-namespace)])
(hash-set! require-list cn
(append vals (hash-ref require-list cn))))
(require vals)))
.. it seems the last line call should be modified to call the original (require) as well?
A correct version or a pointer to how to do it, or any other way of achieving the original goal highly appreciated.
If you just want to get a list of imports for a particular module, there is a convenient built-in called module->imports that will do what you want. It will return a mapping between phase levels and module imports—phase levels higher than 0 indicate imports used at compile-time for use in macro expansion.
> (require racket/async-channel)
> (module->imports 'racket/async-channel)
'((0
#<module-path-index:(racket/base)>
#<module-path-index:(racket/contract/base)>
#<module-path-index:(racket/contract/combinator)>
#<module-path-index:(racket/generic)>))
Note that the module in question must be included into the current namespace in order for module->imports to work, which require or dynamic-require will both do.
This will inspect the information known by the compiler, so it will find all static imports for a particular module. However, the caveats about dynamic requires mentioned by John Clements still apply: those can be dynamically performed at runtime and therefore will not be detected by module->imports.
Short short version:
Have you tried turning on the module browser?
Short version:
You're going to need a macro for this, and
It's not going to be a complete solution
The existing require is not a function; it's a language form, implemented as a macro. This is because the compiler needs to collect the same information you do, and therefore the required modules must be known at compile time.
The right way to do this--as you suggest--is definitely to leverage the existing parsing. If you expand the module and then walk the resulting tree, you should be able to find
everything you need. The tree will be extremely large, but will contain (many instances of) a relatively small number of primitives, so writing this traversal shouldn't be too hard. There will however be a lot of fiddling involved in setting up namespace anchors etc. in order to get the expansion to happen in the first place.
Regarding your original idea: you can definitely create a macro that shadows require. You're going to want to define it in another file and rename it on the way out so that your macro can refer to the original require. Also, the require form has a bunch of interesting subforms, and coming up with a macro that tries to handle all of these subforms will be tricky. If you're looking at writing a macro, though, you're already thinking about an 80% solution, so maybe this won't bother you.
Finally: there are forms that perform dynamic module evaluation, and so you can't ever know for sure all of the modules that might be required, though you could potentially annotate these forms (or maybe shadow the dynamic module-loading function) to see these as they happen.
Also, it's worth mentioning that you may get more precise answers on the Racket mailing list.
No matter how many times I reload my .emacs file,
M-x load-file RET ~/.emacs RET
I want the results to be the same as the first time. I want to make my .emacs file be idempotent.
Motivation
I know I can surgically evaluate a region (C-c C-r), a defun (C-M-x), or the last sexp (C-x C-e). I often take such a more refined approach when making small changes. However, when re-working a .emacs file, I sometimes want to check results of the change conclusively by reloading the entire .emacs file. Restarting emacs each time gets old quick, especially when doing major .emacs housekeeping.
Specific Steps
What specific steps must I take to update my .emacs file to replace non-idempotent operations with idempotent ones?
For example,
Search for "-hook" and replace direct additions to hooks with calls to add-hook, which
will not re-add a function to the hook if already there.
Replace toggling of any flags with direct setting or clearing. Beware of ?? in particular.
...
A comprehensive check-and-correct list would be ideal, but any key individual checks that occur to you would be helpful as well.
I don't know as it's possible to ensure this completely, as most .emacs files depend on libraries which may not have idempotent initialization routines. However, there are a few useful tricks to reduce problems:
Use named functions for add-hook, and keybindings instead of anonymous functions. For add-hook in particular, this allows it to swap out the existing reference.
Careful use of defvar, by default the body is only evaluated if the variable is undefined. C-M-x on a defvar will re-eval the body, but eval-buffer will not if the variable is already initialized.
Careful use of (quote function) instead of function to reference a named function instead of the value of the function. See Anonymous Functions for more advanced discussion about this.
Remember that require will only load the corresponding library the first time it is executed, it will not re-eval on repeated calls as load does. Autoload also uses require to load the corresponding library.
Prefer add-to-list instead of cons to only add an element if it doesn't exist.
For older mode activation, make sure to use (mode-name-mode t) to activate instead of the toggle function. Likewise for turn-on- minor mode functions instead of the mode toggle.
Guard blocks that do have side effects if executed repeatedly. In particular for server mode, (unless (server-running-p) (server-start)), and similar guards for installing packages.
Being careful about side effects in eval-after-load or custom mode hooks. Remember the default mode hooks run when a mode is first enabled, and on each subsequent buffer, but will not rerun in existing buffers when the hook function is modified. eval-after-load is less likely to trip things up, it's still important to remember when it's evaluated.
Related to #2, makunbound may be useful if a chain of vars that depend on each other need to be re-evaluated, as it will force the defvar to always execute on evaluation of the file.
Running eval-buffer on an init file should be as idempotent as possible, but it's important to remember that emacs lisp is fond of side effects and state. While it's possible to ameliorate this to some extent, re-evaling init will never return emacs to the state it was when it first started.
Limit yourself to things you know are idempotent:
defun.
setq to a constant.
add-to-list with a constant.
add-hook, but preferably adding a symbol rather than a lambda expression.
enabling/disabling a minor mode.
wrapping some of the above in conditions.
Of course idempotent doesn't actually mean that the result is the same as re-starting (e.g. removing a setq and then re-evaluating your .emacs won't remove the effect of the previous setq), but the above is pretty much the principles I try to follow in my own ~/.emacs.
In addition to what others have said, load (e.g. require) libraries, including 3rd-party libraries, whose loading is idempotent.
Yes, to find that out for a given library you might need to either read the code or experiment. But nowadays libraries are supposed to be idempotent (or close to it), and many are.
In ELisp, you can skip the evaluation of a definition with the autoload cookie. The definition is evaluated only once it's used.
;; File foo.el
;;;###autoload
(defun foo ()
"Doc"
42)
(defun bar ()
"Doc"
43)
So, if I understand correctly the autoload functionnality is a hack to load file faster. But when I load foo.el, in order to skip the definition of foo the interpreter still has to read the whole form. I don't understand why it's faster.
The simplest way to load the content of the file foo.el is to do a load.
However, this is costly because you have to read the whole file.
With autoload you are allowed to tell emacs: "function foo is defined in file foo.el".
This way emacs does not read the file and you do not pay the loading price. When you use the function foo for the first time, emacs will find the definition for you by reading foo.el.
The ;;;###autoload comment in your file is not doing anything for autoload by itself.
You need to use a program that will grab all of these definitions and put them in a file foo-autoloads.el (or any other name).
For each function it will put a line telling emacs which file contains it.
Then in your .emacs you will load foo-autoloads.el instead of foo.el.
foo.el will be read by emacs the first time you use function foo.
Note: require can also be used instead of load in the explanation above.
Where did you get that "understanding" from? Certainly not the manual.
The autoload cookies are used to pull out skeletal function/variable definitions in a separate autoload file. This is usually done at compile time.
Without going through that compilation step the autoload cookies have no effect.
The autoload forms have to be loaded separatly. They can be generated from the autoload cookies, but you can also write them manually. For example, in your .emacs you can put autoload forms for functions which you only use occasionally, so they are only loaded on demand.
For example:
(autoload 'ahk-mode "ahk-mode")
The example below is given in Paul Grahams ANSI Common Lisp as an example of doing encapsulation:
(defpackage "CTR"
(:use "COMMON-LISP")
(:export "COUNTER" "INCREMENT" "CLEAR"))
(in-package ctr)
;function definitions here
However in Peter Seibels Practical Common Lisp, link here, he says:
Because packages are used by the reader, a package must be defined
before you can LOAD or COMPILE-FILE a file that contains an IN-PACKAGE
expression switching to that package. Packages also must be defined
before other DEFPACKAGE forms can refer to them...
The best first step toward making sure packages exist when they need
to is to put all your DEFPACKAGEs in files separate from the code that
needs to be read in those packages
So he recommends creating two files for every package, one for the defpackage and one for the code. The files containing defpackages should start with (in-package "COMMON-LISP-USER").
To me it seems like putting the defpackage in the same file, before the in-package and code, is a good way to ensure that the package is defined before used. So the first method, collecting everything into one file seems easier. Are there any problems with using this method for package creation?
I think that using a separate file for defpackage is a good habit
because:
You don't « pollute » your files with defpackage.
It makes it easier to find the exported/shadowed/... symbols, you
know you just have to look at package.lisp.
You don't have to worry about the order when you use ASDF.
(defsystem :your-system
:components ((:file "package")
... the rest ...))`
Peter Seibel says so ;)
EDIT:
I forgot to mention quickproject which facilitates the creation of
new CL projects.
REPL> (quickproject:make-project "~/src/lisp/my-wonderful-project/"
:depends-on '(drakma cl-ppcre local-time))`
This command will create a directory "~/src/lisp/my-wonderful-project/"
and the following files:
package.lisp
my-wonderful-project.asd (filled)
my-wonderful-project.lisp
README.txt
And thus, I think it's good to use the same convention.
I tend to use multiple source code files, a single "packages.lisp" file and a singe "project.asd" system definition file for most of my projects. If the project requires multiple packages, they're all defined in "packages.lisp", with the relevant exports in place exported.
There is this reason for putting DEFPACKAGE in its own file: if you have a large package, then you might have several groups of related functions, and you might want to have separate source files per function group. Then all the source files would have their own IN-PACKAGE at the top, but they would all "share" the external DEFPACKAGE. Then as long as you get the DEFPACKAGE loaded first, it doesn't matter the order you load the other source files.
An example I'm currently working on has multiple classes in the package, and the source files are broken up to be per class, each having a class definition and the related generic function and method definitions.