I'm trying to create a new language in Racket where statements are on separate lines. A newline defines the end of a statement and the start of a new one.
I read through the Create Languages chapter of the guide which was very useful but the examples were focused on extending s-exp-like languages. The only option I see is manually writing my own parser for read and read-syntax.
I was hoping to use readtables but I don't know if I can. I tried:
(make-readtable #f #f 'non-terminating-macro my-read-line-fn)
but I don't know if this is much help. I guess I could create a sub-readtable that does things like read-word, read-string which I dispatch to based on what character my-read-line-fn gets.
Is that the best strategy or is there a predefined way of reading until the end of line?
I don't think you need to do anything with the readtable. Your lang/reader.rkt can provide your own read-syntax that can read/parse however it wants, and presumably stop when it encounters EOL.
One interesting example is Brainfudge. Its concept of a "statement" is a single character, but IIUC also [ brackets ].
See its lang/reader.rkt and parser.rkt for the low-level bits, and then try to understand how that is ultimately evaluated as Racket expressions.
You do indeed need to write version of read and read-syntax that parse your language. The readtable is only meant to modify the builtin read, so I suggest that you take a look at Parser Tools (http://docs.racket-lang.org/parser-tools/index.html), which is tool for writing parsers in the lex/yacc style.
An alternative is to use ragg:
http://www.hashcollision.org/ragg/
Install Ragg using the package manager in DrRacket. Search for ragg in the list of available packages.
Make your own reader.rkt:
#lang s-exp syntax/module-reader
(test test)
#:read-syntax my-read-syntax
#:read my-read
;; override default read (won't be used but is required)
(define (my-read in) (read-line in))
;; override read-syntax by reading in one string at a time and
;; pass it to statement-string->code to get code as dara and
;; make it syntax with datum->syntax
(define (my-read-syntax in)
(datum->syntax #f (statement-string->code (read-line in))))
;; This is actually how you want your code
;; to become s-expressions. I imagine that my
;; module has a primitive called code that
;; interprets the string as is
(define (statement-string->code str)
(list 'code str))
Racket doesn't have "statements", so the concept of newlines ending "statements" is nonsensical.
If your motivation is to reduce or do away with parentheses, I encourage you to use a "standard alternative" reader like sweet-expressions, rather than making something home-grown.
Related
In Lisp, a function's arguments are evaluated first before entering the function body. Macro arguments stay not evaluated.
But sometimes, one wants to inject code pieces stored in variables into a macro. This means evaluating the argument for the macro first, and then apply the macro-of-choice on this evaluated result.
One has to resort to
(eval `(macro ,arg))
To achieve this - but eval does not behave correctly in different environments.
The best thing would be, if one could do:
(apply macro (list arg))
or
(funcall macro arg)
But since the macro is not a function this doesn't work.
Is it possible to achieve something like this? - To circumvent that problem oder to make the macro available in the functions namespace?
Or am I missing some other ways to solve such problems?
I came to this question while trying to answer How to produce HTML from a list. but also in Generate TYPECASE with macro in common lisp, Evaluate arguments passed to a macro that generates functions in lisp, and How to convert a list to code/lambda in scheme?. But I always thought while answering them it would be good to have an apply or funcall-like function which can take macros.
It is not clear what you are trying to do, although it is almost certain that you are confused about something. In particular if you are calling eval inside macroexpansions then in almost all cases you are doing something both seriously wrong and seriously dangerous. I can't ever think of a case where I've wanted macros which expand to things including eval and I have written Lisp for a very very long time.
That being said, here is how you call the function associated with a macro, and why it is very seldom what you want to do.
Macros are simply functions whose domain and range is source code: they are compilers from a language to another language. It is perfectly possible to call the function associated with a macro, but what that function will return is source code, and what you will then need to do with that source code is evaluate it. If you want a function which deals with run-time data which is not source code, then you need that function, and you can't turn a macro into that function by some magic trick which seems to be what you want to do: that magic trick does not, and can not, exist.
So for instance if I have a macro
(defmacro with-x (&body forms)
`(let ((x 1))
,#forms))
Then I can call its macro function on a bit of source code:
> (funcall (macro-function 'with-x)
'(with-x (print "foo")) nil)
(let ((x 1)) (print "foo"))
But the result of this is another bit of source code: I need to compile or evaluate it, and nothing I can do will get around this.
Indeed in (almost?) all cases this is just the same as macroexpand-1):
> (macroexpand-1 '(with-x (print "foo")))
(let ((x 1)) (print "foo"))
t
And you can probably write macroexpand-1 in terms of macro-function:
(defun macroexpand-1/equivalent (form &optional (env nil))
(if (and (consp form)
(symbolp (first form))
(macro-function (first form)))
(values (funcall (macro-function (first form)) form env)
t)
(values form nil)))
So, if the result of calling a macro is source code, what do you do with that source code to get a result which is not source code? Well, you must evaluate it. And then, well, since the evaluator expands macros for you anyway, you might as well just write something like
(defun evaluate-with-x (code)
(funcall (compile nil `(lambda ()
(with-x ,#code)))))
So you didn't need to call the macro's function in any case. And this is not the magic trick which turns macros into functions dealing with data which is not source code: it is a terrible horror which is entirely made of exploding parts.
A concrete example: CL-WHO
It looks like this question might have its origins in this one and the underlying problem there is that that's not what CL-WHO does. In particular it is a confusion to think that something like CL-WHO is a tool for taking some kind of list and turning it into HTML. It's not: it's a tool for taking the source code of a language which is built on CL but includes a way of expressing HTML output mingled with CL code, and compiles it into CL code which will do the same thing. It happens to be the case that CL source code is expressed as lists & symbols, but CL-WHO isn't really about that: it's a compiler from, if you like, 'the CL-WHO language' to CL.
So, let's try the trick we tried above and see why it's a disaster:
(defun form->html/insane (form)
(funcall
(compile nil `(lambda ()
(with-html-output-to-string (,(make-symbol "O"))
,#form)))))
And you might, if you did not look at this too closely, think that this function does in fact do the magic trick:
> (form->html/insane '(:p ((:a :href "foo") "the foo")))
"<p></p><a href='foo'>the foo</a>"
But it doesn't. What happens if we call form->html/insane on this perfectly innocuous list:
(:p (uiop/run-program:run-program "rm -rf $HOME" :output t))
Hint: don't call form->html/insane on this list if you don't have very good backups.
CL-WHO is an implementation of a programming language which is a strict superset of CL: if you try to turn it into a function to turn lists into HTML you end up with something involving the same nuclear weapon you tinker with every time you call eval, except that nuclear weapon is hidden inside a locked cupboard where you can't see it. But it doesn't care about that: if you set it off it will still reduce everything within a few miles to radioactive ash and rubble.
So if you want a tool which will turn lists – lists which aren't source code – into HTML then write that tool. CL-WHO might have the guts of such a tool in its implemenentation, but you can't use it as it is.
And this is the same problem you face whenever you are trying to abuse macros this way: the result of calling a macro's function is Lisp source code, and to evaluate that source code you need eval or an equivalent of eval. And eval is not only not a terrible solution to almost any problem: it's also a nuclear weapon. There are, perhaps problems for which nuclear weapons are good solutions, but they are few and far between.
I love Racket's #;. I want to see it in every language that I ever use again. Can it be added to other Lisps via their macro systems? Or does the commenting character break the macro system's ability to read the code?
A sufficient answer will demonstrate a macro being built in any Lisp other than Racket that allows for a change in the commenting system. You need not actually implement Racket's #;, but I would like it if you do. Lisps with the least similarity to Racket, e.g. Clojure or any non-Scheme will be particularity nice to see.
#; isn't a macro, it's what Common lisp would call a readmacro: what it does is defined at read time, not later than that. Read macros which aim to completely suppress input are mildly perilous because there needs to be a way of saying 'read the following thing, but ignore it', and that's only possible if any other readmacros behave well: there's nothing to stop someone defining a readmacro which produces some side-effect even if reading is suppressed.
However, well-behaved readmacros (which includes all of the standard ones and the rest of the standard reader) in CL won't do that: they'll listen to whether reading is being suppressed, and behave accordingly.
CL allows you to do this as standard by using its conditionalisation on features, and in particular #+(or) <expr> will always skip <expr>.
But you can define your own: #; is not predefined so you can define it:
(set-dispatch-macro-character
#\# #\;
(lambda (stream char n)
(declare (ignore char))
(let ((*read-suppress* t))
(dotimes (i (or n 1) (values))
(read stream)))))
After this, or at least with a better tested version of this, then #; <expr> (or obviously #;<expr>) will read as whitespace, and #2; ... ... will skip two following expressions:
> (let ((x #;1 #2; 2 3 4)) x)
4
What you are looking for is #+(or) reader macro.
Since (or) evaluates to nil, the condition is always false the following form is never evaluated.
I would like to create a new language that has the same syntax as typed racket, but when executed will do two things:
run the given program as typed racket
if type checks, then translates the input into another language (say python). I am planning to write a translator from typed racket forms to python syntax.
Any suggestions on how to start or pointers to some skeleton code? I have read this tutorial but it mostly talks about creating new syntax which I don't need in my case.
I understand how I can write racket code to translate from one language to another, what I don't get is how I can do both of the above, i.e., first run it as another language, and then do something else with the same input.
It sounds like you want to make a language with a new #%module-begin form. This form is either inserted by the reader (when you do a #lang .... line at the top of your file, or by the language if you wrote the module out by hand. Either way, its generally associated with your language definition. This macro has full access to the entire module's unexpanded syntax. So for example, a macro like this:
(provide (rename-out [-module-begin #%module-begin]))
(define-simple-macro (-module-begin body ...)
(#%module-begin
(writeln '(body ...))
body ...))
Would create a language that does two things:
Print out the body of the code (as an s-expression), and
Run the body with the #%module-begin that the language definition was written in.
You can see how you can use this technique to grab the body of your program twice, and do two different things with it. So let's try running this example. First, lets take the sample from above, and put it in a file "mylang.rkt":
#lang racket
(provide (rename-out [-module-begin #%module-begin])
(except-out (all-from-out racket) #%module-begin))
(require syntax/parse/define)
(define-simple-macro (-module-begin body ...)
(#%module-begin
(writeln '(body ...))
body ...))
And now we can write a program in mylang like this:
#lang s-exp "mylang.rkt"
(+ 1 2)
And when you run it, you'll get something like this:
((+ 1 2))
3
First its printing out the program text, then running it.
You can read more about this process in a paper I wrote discussing this aspect of the Video language. You might also find the book Beautiful Racket to have some examples you might find useful.
I'm trying to write an emacs LISP function to un-indent the region
(rigidly). I can pass prefix arguments to indent-code-rigidly or
indent-rigidly or indent-region and they all work fine, but I don't
want to always have to pass a negative prefix argument to shift things
left.
My current code is as below but it seems to do nothing:
(defun undent ()
"un-indent rigidly."
(interactive)
(list
(setq fline (line-number-at-pos (region-beginning)))
(setq lline (line-number-at-pos (region-end)))
(setq curIndent (current-indentation))
;;(indent-rigidly fline lline (- curIndent 1))
(indent-region fline lline 2)
;;(message "%d %d" curIndent (- curIndent 1))
)
)
I gather that (current-indentation) won't get me the indentation of the first line
of the region, but of the first line following the region (so a second quesiton is
how to get that!). But even when I just use a constant for the column (as shown,
I don't see this function do any change.
Though if I uncomment the (message) call, it displays reasonable numbers.
GNU Emacs 24.3.1, on Ubuntu. And in case it matters, I use
(setq-default indent-tabs-mode nil) and (cua-mode).
I must be missing something obvious... ?
All of what Tim X said is true, but if you just need something that works, or an example to show you what direction to take your own code, I think you're looking for something like this:
(defun unindent-rigidly (start end arg &optional interactive)
"As `indent-rigidly', but reversed."
(interactive "r\np\np")
(indent-rigidly start end (- arg) interactive))
All this does is call indent-rigidly with an appropriately transformed prefix argument. If you call this with a prefix argument n, it will act as if you had called indent-rigidly with the argument -n. If you omit the prefix argument, it will behave as if you called indent-rigidly with the argument -1 (instead of going into indent-rigidly's interactive mode).
There are a number of problems with your function, including some vary
fundamental elisp requirements. Highly recommend reading the Emacs Lisp
Reference Manual (bundled with emacs). If you are new to programming and lisp,
you may also find An Introduction to Emacs Lisp useful (also bundled with
Emacs).
A few things to read about which will probably help
Read the section on the command loop from the elisp reference. In particular,
look at the node which describes how to define a new command and the use of
'interactive', which you will need if you want to bind your function to a key
or call it with M-x.
Read the section on variables from the lisp reference
and understand variable scope (local v global). Look at using 'let' rather
than 'setq' and what the difference is.
Read the section on 'positions' in the elisp reference. In particular, look at
'save-excursion' and 'save-restriction'. Understanding how to define and use
the region is also important.
It isn't clear if your writing this function just as a learning exercise or
not. However, just in case you are doing it because it is something you need to
do rather than just something to learn elisp, be sure to go through the Emacs
manual and index. What you appear to need is a common and fairly well supported
requirement. It can get a little complicated if programming modes are involved
(as opposed to plain text). However, with emacs, if what you need seems like
something which would be a common requirement, you can be fairly confident it is
already there - you just need to find it (which can be a challenge at first).
A common convention is for functions/commands to be defined which act 'in
reverse' when supplied with a negative or universal argument. Any command which
has this ability can also be called as a function in elisp code with the
argument necessary to get that behaviour, so understanding the inter-play
between commands, functions and calling conventions is important.
I'm writing an emacs major mode for a programming environment which supports two slightly different programming languages. Both are lisps (one is Scheme), so both use s-expressions. The two languages are differentiated by their function definition keyword: scheme uses define, while the other language (which is called xtlang) uses bind-func
So, in the same buffer, I have
(define foo ; this is scheme
(lambda (a)
(+ scheme-fn a))
(bind-func bar ; this is xtlang
(lambda (b)
(* xtlang-fn b))
I've got the font-locking working, using something like
(font-lock-add-keywords nil
'(("(\\(define\\)\\>"
(1 font-lock-keyword-face))
("(\\(bind-func\\)\\>"
(1 font-lock-keyword-face))
("\\<scheme-fn\\>"
(0 font-lock-function-name-face))
("\\<xtlang-fn\\>"
(0 font-lock-function-name-face))
))
What I'd like to be able to do is to be able to colour the parens differently based on the language (scheme/xtlang).
So, colour all the parens in the form red if the top-level defun is a define, and blue if it's a bind-func, while still highlighting all the keywords/functions within the forms as normal.
This may require multi-line font locking, since the define/bind-func will probably be on a previous line to the keywords to be highlighted. This thread suggests that font-lock-multiline in conjunction with match-anchored in font-lock-keywords may be the answer, but then goes on to suggest that font-lock-multiline should only be used in situations where the multi-line aspect is the exception rather than the rule.
My other option seems to be to use syntax-propertize, but I'm a bit confused about how that works - the documentation is a bit sparse.
The easiest way to handle this is utilizing the following fact:
MATCHER can be either the regexp to search for, or the function name
to call to make the search (called with one argument, the limit of the
search;
In other words, you could replace regexps like "\\<scheme-fn\\>" with a function that repeatedly search for the regexp using re-search-forward, and return if a match is found in the correct context.
For a concrete example of a package that use this technique, look at cwarn-mode, which is part of the standard Emacs distribution.