Consider the following code, which creates a simple table (alist) of age-related capabilities, attaching a finalizer to the data structure:
#lang racket
(require ffi/unsafe/alloc)
(define (finalize-capabilities-table ignored)
(printf "finalizing capabilities table\n"))
(define make-capabilities-table
((allocator finalize-capabilities-table)
(lambda ()
(printf "creating capabilities table\n")
'((16 . drive)
(18 . vote)
(21 . drink)))))
(make-capabilities-table)
I expect this code, when run, to produce
creating capabilities table
'((16 . drive) (18 . vote) (21 . drink))
finalizing capabilities table
However, I observe that the last message – finalizing capabilities table – does not print. Why not? Did I fail to attach the finalizer correctly?
First, the value returned by the function is from a quote expression, so the list is allocated once when the function's code is compiled/loaded, and a reference to that constant is held by the function. The following allocates a fresh (outer) list instead:
(define make-capabilities-table
((allocator finalize-capabilities-table)
(lambda ()
(printf "creating capabilities table\n")
(list '(16 . drive)
'(18 . vote)
'(21 . drink)))))
Second, the finalizer seems to be run in a thread with the current output and error ports parameterized to suppresses output. The docs for register-finalizer say:
The given finalizer procedure should generally not rely on the
environment of the triggering thread, such as its output ports or
custodians, except that relying on a default logger is reasonable.
You can get around that (for the purpose of testing, at least) with the following:
(define err (current-error-port))
(define (finalize-capabilities-table ignored)
(fprintf err "finalizing capabilities table\n"))
Finally, finalizers aren't run as soon as a value becomes unreachable; they're only run when the GC notices, and the GC doesn't run after every REPL interaction. You can prompt the GC to run with (collect-garbage).
Related
When serving large files from Clack/Hunchentoot with Slime connected, I sometimes see error messages like SB-IMPL::SIMPLE-STREAM-PERROR "Couldn't write to ~s"... Those are caused by the browser prematurely dropping connections (which is totally OK). The problem is that each time it happens, SLDB pops up. Which is annoying.
Is there a way I can inhibit certain errors in SLDB such as the above? I still would like to see them in an error log, but definitely not in SLDB.
You can subclass PROCESS-CONNECTION for your acceptor and do your own error handling for this error.
Let's start by defining a custom acceptor:
(defclass no-error-acceptor (hunchentoot:acceptor)
())
Then we can create a wrapper around PROCESS-CONNECTION that inhibits printing of a message for this specific error:
(defmethod hunchentoot:process-connection ((acceptor no-error-acceptor) (socket t))
(handler-case
(call-next-method)
(sb-impl::simple-stream-perror (condition)
;; Perhaps log the error here?
nil)))
Make sure you actually start the server using this acceptor in order for it to be used.
UPDATED
Since your system uses Hunchentoot, you could set the global variable HUNCHENTOOT:*CATCH-ERRORS-P* to T. This should guarantee that the all the conditions arising in code managed by Hunchentoot are catched by Hanchentoot itself and not passed to the debugger.
To disable the debugger in any Common Lisp implementation (both inside a shell REPL as well as the Slime REPL inside Emacs) you could use the predefined global variable *debugger-hook*, by assigning to it a two argument function. The function will receive the condition and the current value of *debugger-hook* when it is called, and can handle the condition or return normally, and in this case the debugger is invoked. For instance, you could simply print the condition:
* (defun my-debug(condition hook)
(declare (ignore hook))
(print condition)
(abort))
DEBUG-IGNORE
* (setf *debugger-hook* #'my-debug)
#<FUNCTION MY-DEBUG>
This second method however cannot work when using Hunchentoot together with Slime, due to the way the two packages interact with respect to the debugging strategies.
In this case one could adopt the solution found by Mike Ivanov, that redefines the swank-debugger-hook function before starting Swank:
(in-package swank)
(setq swank-debugger-hook-orig #'swank-debugger-hook)
(defun swank-debugger-hook (condition hook)
(etypecase condition
(sb-int:simple-stream-error
(progn
(princ "*** Stream error" *error-output*)
(abort)))
(t (funcall swank-debugger-hook-orig condition hook))))
(in-package cl-user)
(swank:create-server :port 4008 :dont-close t)
I have a system written in Lisp that runs state machines. I'd like to dynamically load the definition of the state machine and any required assets (images, etc) from a directory, given just the name of the directory. There will be multiple different state machines. This is similar, but not identical, to Apache loading and running a WAR file.
My concern is that simply compiling and loading a file could run literally anything. Ideally I'd like to get just the state machine definition, configure it with the path to the assets, and have it available to execute. Right now I'm playing around with loading a class that implements a particular base class, but that's not straightforward. Is there a standard technique for this?
Thanks.
Are you saying you're worried about the possibilities of arbitrary code execution from reading in a file? I so you should look into redefining the read-table to exclude unwanted symbols.
For an example checkout this, look for 'SAFE-READ-FROM-STRING'.
It's not complete but then you can use #'read to get the datastructure, do some sanity check and compile if you need to.
If this isn't what you were looking for then my apologies, would you be able to explain further what you are looking for?
Given that you want to read definition of a state machine without executing arbitrary code, you may consider the following macro:
(defmacro def-state-machine (name (&rest assets) &rest states)
`(defparameter ,name
(list
:assets ',(remove-if-not #'legal-asset? assets)
:states ',(remove-if-not #'legal-state? states))))
which will create list of valid assets and states (since I don't know, how your machine looks like, I'm putting some abstract predicates here - they may check for legal syntax, or if argument is of the certain type, or e throw an error if asset or state are illegal).
Let's assume, you also need to define some function to run machine:
(defmacro def-transition (name args &body body)
`(defun ,name (,#args)
,#body))
Separate macro for defining function allows additional sanity checks. Finally, you may define reader function:
(defun load-toy-state-machine (directory)
(let ((path (cl-fad:merge-pathnames-as-file directory #P"machine.lisp"))
;(*readtable* (copy-readtable nil))
)
; (make-dispatch-macro-character #\#)
(with-open-file (stream path :direction :input)
(do ((form (read stream nil 'done)
(read stream nil 'done)))
((eql form 'done) T)
(if (member (car form) '(def-state-machine def-transition))
(eval form)
(error "malformed state machine definition file"))))))
Which will eval only allowed macros (def-state-machine and def-transition), which have fixed syntax, and may contain additional sanity checks. Neither of these executes code.
I am experimenting processes in emacs/ielm for a sort of parallel computing, i.e. to start many processes for different computations and then to wait for all the processes have terminated to compose the result. In order to do this I set up the following simple function:
(defun testp ()
(while (> (length (process-list)) 1)))
I call testp after all the child-processes have been started and, when I get again the control, I compose the result:
If there are no child-processes testp exits immediately: ok;
If there is at least one child-process testp loops: ok;
When all child-processes have finished testp continues looping, and this is
not good.
May I ask you to help me understand where I am wrong.
After a process finishes, it is not necessarily deleted immediately. So (process-list) may still list it. The user variable delete-exited-processes controls this, so check that you have set it to t. The function list-processes will also explicitly delete finished processes, so that may be helpful.
Check out the chapter on processes in the manual for further details.
Emacs is single-threaded and doesn't handle external input (e.g. about the change in a process's status) until it reaches a "safe point". So the above loop (as #juanleon suggests) is "too tight". You want to add a sleep into it.
But better is to use set-process-sentinel so that Emacs gets told when the processes end rather than having to busy-wait for it.
E.g.:
...start a new process stored in proc...
(push proc my-list-of-running-processes)
(set-process-sentinel proc #'my-run-when-its-over)
...
(defun my-run-when-its-over (proc msg)
(setq my-list-of-running-processes (delq proc my-list-of-running-processes))
(unless my-list-of-running-processes
(message "Haha! all my processes are done!")))
Note also that you probably don't want to use process-list since that can contain unrelated processes (e.g. if you use M-x shell or various other things), which is why I used my-list-of-running-processes above.
Maybe the tight loop is not letting emacs do a proper process cleanup. My recommendation would be to apply a filter to the result of (process-list) (since if you have a runnning compilations, server, shell, etc. the process list won't be empty), to filter out processes different of those you are interested in, and also to add a small delay to the while ((sleep 0 500), for instance).
I'm reading the following section of SICP
http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-26.html#%_sec_4.1.7
According to the text, the following transformation of eval will improve offers a performance improvement, since an expression that gets evaluated many times will only be analyzed once?
(define (eval exp env)
((analyze exp) env))
Here is an analyze function given in the book:
(define (analyze-if exp)
(let ((pproc (analyze (if-predicate exp)))
(cproc (analyze (if-consequent exp)))
(aproc (analyze (if-alternative exp))))
(lambda (env)
(if (true? (pproc env))
(cproc env)
(aproc env)))))
I don't understand why the book says that analyze will only run once. Doesn't the the body of eval, which is ((analyze exp) env)) basically say that each time eval is called, analyze will be called with exp as its parameter? This would mean that analyze would be called every time eval is called.
What's wrong with my understanding? I would appreciate any feedback, thanks!
Indeed, every time you call eval with program code as a parameter, the syntactic evaluator will be invoked. However, when a function within that code calls another function within that code (or, in the simplest case, it calls itself by recursion), the inner apply will get the analyzed expression (which is in the end a lambda function) as an argument, rather than a blob of code which would need to be syntactically analyzed again in order to be executed.
Gintautas' answer is correct, but maybe an example is in order. Suppose you've developed a Scheme dialect that sports a loop construct
(do-n-times n expr)
with the obvious semantics. Now, when you call the naive eval to evaluate a loop that runs ten times
(eval '(do-n-times 10 (print 'hello)))
then it will analyze the loop body ten times. With the version of eval that separates analysis from evaluation, the loop body is analyzed once, then evaluated ten times.
The analysis phase returns a procedure, which may or not be fast in your Scheme interpreter. However, it could conceivably do all kinds of optimizations (dead code analysis, JIT compilation to machine code, etc.).
larsmans's answers is extremely good.
As a complementary answer, one can also view analyze(environ) as a curried form of eval(expr, environ) where the parameter expr has been passed in ahead of time. In SICP, you can read the example code like:
(define (analyze-assignment exp)
(let ((var (assignment-variable exp))
(vproc (analyze (assignment-value exp))))
(lambda (env)
(set-variable-value! var (vproc env) env)
'ok)))
When you see a let (([var] [preprocessed stuff])), that is preprocessing being stored in a closure until it is needed later when environ is passed in.
This is mostly a follow-up to this question. I decided to just keep YAGNI in mind and created a global variable (libpython). I set it to #f initially, then set! it when init is called. I added a function that should handle checking if that value has been initialized:
(define (get-cpyfunc name type)
(lambda args
(if libpython
(apply (get-ffi-obj name libpython type) args)
(error "Call init before using any Python C functions"))))
So now here's what I want to do. I want to define a macro that will take the following:
(define-cpyfunc Py_Initialize (_fun -> _void))
And convert it into this:
(define Py_Initialize (get-cpyfunc "Py_Initialize" (_fun -> _void)))
I've been reading through the macro documentation to try figuring this out, but I can't seem to figure out a way to make it work. Can anyone help me with this (or at least give me a general idea of what the macro would look like)? Or is there a way to do this without macros?
I've answered most of this question in the other one (I didn't see this one). It's fine to use a function that pulls out the bindings like this, but one possible problem here is that since you generate the binding only when the resulting function is called, this binding is re-created on each and every call. An easy way to solve this quickly is using promises, something like this:
(require scheme/promise)
(define (get-cpyfunc name type)
(define the-function
(delay (if libpython
(get-ffi-obj name libpython type)
(error "Call init before using any Python C functions"))))
(lambda args (apply (force the-function) args)))
But this is essentially almost the same as the code I posted in your previous question.
More random notes:
get-ffi-obj will accept a symbol as the name to bind to -- this is intentional, to make such macros (as in the last question) easy.
Using (symbol->string 'name) in a macro is fine. As I noted above in my comment reply to Nathan's comment, this means that it gets called at runtime, but mzscheme should be able to optimize that anyway, so there's no need to try and write some sophisticated macro that does the job at compile time.
Look inside the PLT directory -- you will find a collection called ffi. This is a collection of examples of bindings with various styles. Macros that create the bindings are very common in these examples.
Why don't you change the generated code to
(define Py_Initialize (get-cpyfunc 'Py_Initialize (_fun -> _void)))
and then have get-cpyfunc run (symbol->string name)?
Granted, there is probably a way to do this with syntax-case (I can never remember its syntax though), and definitely if you're using a Scheme with CL-esque define-macro.
It's not the complete answer, but I came up with a macro that meets both requirements (defines a variable and a string with the name of that variable):
> (define-syntax (my-syntax stx)
(syntax-case stx ()
[(_ id)
#'(define-values (id) (values (symbol->string (quote id))))]))
> (my-syntax y)
> y
"y"