When parsing syntax from a string input port, all line/column numbers get set to #f. Why is that?
(let* ([port (open-input-string "123")]
[stx (read-syntax "my-input" port)])
(printf "line ~a column ~a" (syntax-line stx) (syntax-column stx))
)
Expected output:
line 1 column 0
Actual output:
line #f column #f
You have to explicitly turn on line based location tracking for ports, for example by setting the port-count-lines-enabled parameter:
(parameterize ([port-count-lines-enabled #t])
(let* ([port (open-input-string "123")]
[stx (read-syntax "my-input" port)])
(printf "line ~a column ~a" (syntax-line stx) (syntax-column stx))))
which prints
line 1 column 0
There's also port-count-lines! for enabling it on existing ports.
Related
I am trying to define a reader macro that reads the length of a string. The string will be enclosed in vertical bars (pipes). For example:
|yes| -> 3
|| -> 0
|The quick brown fox| -> 19
|\|| -> 1 — The pipe character can be escaped by preceding it with a backslash.
(let ((x |world|)) (+ |hello| x)) -> 10
I managed to write this:
#lang racket
(define (handle-pipe in [count 0])
(define cur-char (read-char in))
(cond [(eqv? cur-char #\|)
count]
[else
(when (and (eqv? cur-char #\\) ; Handle escape ("\|").
(eqv? (peek-char in) #\|))
(read-char in)) ; Consume |.
(handle-pipe in (+ count 1))]))
(parameterize ([current-readtable
(make-readtable (current-readtable)
#\|
'terminating-macro
(lambda (char in src-name line col pos)
(handle-pipe in)))])
(eval (read (open-input-string "(let ((x |world|)) (+ |hello| x))"))
(make-base-namespace)))
This returns 10, as expected.
The problem now is: I would like to use the reader macro directly in my Racket code instead of having to feed a string into (eval (read (open-input-string ...))). For example, I would like to use the reader macro like this:
#lang racket
(define (handle-pipe in [count 0])
(define cur-char (read-char in))
(cond [(eqv? cur-char #\|)
count]
[else
(when (and (eqv? cur-char #\\) ; Handle escape ("\|").
(eqv? (peek-char in) #\|))
(read-char in)) ; Consume |.
(handle-pipe in (+ count 1))]))
(current-readtable
(make-readtable (current-readtable)
#\|
'terminating-macro
(lambda (char in src-name line col pos)
(handle-pipe in))))
(let ((x |world|)) ; Using the reader macro directly in Racket.
(+ |hello| x))
However, there is an error message when I run the program above:
my-program.rkt:20:9: world: unbound identifier
in: world
location...:
my-program.rkt:20:9
context...:
do-raise-syntax-error
for-loop
[repeats 1 more time]
finish-bodys
lambda-clause-expander
for-loop
loop
[repeats 3 more times]
module-begin-k
expand-module16
expand-capturing-lifts
temp118_0
temp91_0
compile15
temp85_0
standard-module-name-resolver
What did I do wrong? How do I use the reader macro in my code?
That's not possible. The pipeline of compilation/evaluation in Racket is:
Read
Expand macros
Evaluate expanded program
Your configuration of current-readtable is done in step 3, so it cannot influence things that happened already in step 1.
The reason your first code works is that eval starts the pipeline again on the datum you provided it to.
Note that the reader macro is actually intended to be used when you create a new #lang. But since you want it to work with #lang racket, it's not applicable.
This question is about sbcl -- or so I thought originally. The question: When is a character not a character? Consider the following code:
(defconstant +asc-lf+ #\Newline)
(defconstant +asc-space+ #\Space)
(prin1 (type-of #\Newline )) (terpri)
(prin1 (type-of #\Space )) (terpri)
(prin1 (type-of +asc-lf+ )) (terpri)
(prin1 (type-of +asc-space+)) (terpri)
As expected, it produces:
STANDARD-CHAR
STANDARD-CHAR
STANDARD-CHAR
STANDARD-CHAR
Now consider this code:
(defun st (the-string)
(string-trim '(#\Newline #\Space) the-string))
(princ "\"")
(princ (st " abcdefgh "))
(princ "\"")
(terpri)
It produces:
"abcdefgh"
But consider this code:
(defconstant +asc-lf+ #\Newline)
(defconstant +asc-space+ #\Space)
(defun st (the-string)
(string-trim '(+asc-lf+ +asc-space+) the-string))
(princ "\"")
(princ (st " abcdefgh "))
(princ "\"")
(terpri)
When you load it using sbcl, it gives you:
While evaluating the form starting at line 6, column 0
of #P"/u/home/sbcl/experiments/type-conflict.d/2.lisp":"
debugger invoked on a TYPE-ERROR:
The value
+ASC-LF+
is not of type
CHARACTER
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [RETRY ] Retry EVAL of current toplevel form.
1: [CONTINUE] Ignore error and continue loading file "/u/home/sbcl/experiments/type-conflict.d/2.lisp".
2: [ABORT ] Abort loading file "/u/home/sbcl/experiments/type-conflict.d/2.lisp".
3: Exit debugger, returning to top level.
((FLET SB-IMPL::TRIM-CHAR-P :IN SB-IMPL::GENERIC-STRING-TRIM) #\ )
0]
At first, I was anticipating being able to report that clisp does the appropriate call to #'string-trim, with the anticipated returned value, or maybe errors out. But it does neither of these. The function returns the same string that was passed to it, without any trimming.
Is this what should be happening? What am I missing?
EDIT approx. 2017-10-21 08:50 UTC
The fine answer by PuercoPop inspires a follow-up question. If I should post this as a separate question, just give the word and I will.
Why is it that (at least with sbcl and clisp) this:
(defconstant +asc-lf+ #\Newline)
(defconstant +asc-space+ #\Space)
(prin1 (type-of (first (list #\Newline #\Space))))
(terpri)
(prin1 (type-of (first '(#\Newline #\Space))))
(terpri)
yields this?
STANDARD-CHAR
STANDARD-CHAR
With PuercoPop's answer, I would have expected it to yield something about a symbol, not a character, for the second expression.
The main confusion comes from
the dual purpose of lists: data and code. For evaluation (+ a b) is code, here a function call. Both (quote (+ a b)) and '(+ a b) is data, as they evaluate to the quoted literal data.
reading already creates objects. #\newline is already read as a character object. It is built-in syntax: Sharpsign Backslash It is not a string, not a symbol and not some yet unknown piece of data. It is read as an object of type character (I use the wording character object for that here, one could also just say character).
These are symbols:
foo
bar
+foo+
*the-foo*
When symbols get evaluated, they evaluate to their value.
These are character objects:
#\f
#\O
#\o
#\newline
When character objects get evaluated, they evaluate to themselves.
Thus '#\foo, (quote #\foo) and #\foo evaluate all to the same object.
These are lists
(newline #\newline) ; the first item is a symbol, the second a character object
(#\a #\b #\c) ; a list of character objects
(a b c) ; a list of symbols
What happens if we evaluate lists:
(+ a b) ; the sum of the values of A and B
(list a b) ; a list gets computed, with the values of variables a and b
(list 'a 'b) ; a list gets computed, with the symbols A and B
'(a b) ; a literal list of the symbols A and B
'(#\a #\b) ; a literal list of the character objects #\a and #\b
'(a #\a) ; a literal list of the symbol A and the character object #\a
(#\a #\b) ; an error, #\a is not a function/macro/special-form
(+ a 'b) ; an error, a symbol B is not a number
Evaluating backquoted lists:
`(a ,a #\a ,#\a) ; a list of the symbol a, the value of the variable a,
; the character object a and again the character object a
Your error:
'(+asc-lf+ +asc-space+) evaluates to a list of symbols.
The function STRING-TRIM expects a sequence of characters.
You need to write something like this:
(list +asc-lf+ +asc-space+) ; calling the function list
`(,+asc-lf+ ,+asc-space+) ; a backquoted list with comma for evaluation
(vector +asc-lf+ +asc-space+) ; the constructed vector is also a sequence
Also:
(list #\Newline #\Space) and '(#\Newline #\Space) evaluate both to a list of characters. The #\ syntax is a built-in feature of the Lisp reader to construct character objects. Thus #\newline is converted at read-time into a character-object:
CL-USER 82 > (describe (read))
#\Newline ; we type the nine characters #\Newline
#\Newline is a CHARACTER
Name "Newline"
Code 10
The problem is that you are quoting the "character list". So instead of a list of characters it is a list of symbols. That is
(defun st (the-string)
(string-trim (list +asc-lf+ +asc-space+) the-string))
The error message hints at this when it says
The value
+ASC-LF+ is not of type
CHARACTER
and not
The value
#\Newline is not of type
CHARACTER
I'm writing a grammar which I intend to implement in a Lisp read procedure, i.e. reading one expression at a time from an input source which is i.e. mutable. Most of the grammar is just like Lisp, but the two pertinent changes are:
Whitespace is read and is part of the resulting syntax. Contiguous whitespace is grouped together like contiguous non-whitespace characters are grouped as identifiers, and the result of reading such a string is a "whitespace object", which stores the exact sequence of characters read. The evaluator ignores whitespace objects when they appear in a list (in other words, if foo is a whitespace object then (eval '(+ 3 foo 4)) is equivalent to (eval '(+ 3 4))), and if it is asked to evaluate one directly, it is self-evaluating.
Secondly, if several tokens other than whitespace tokens appear on the same line, those tokens are collected into a list and that list is the result of the read.
e.g.,
+ 3 4 5
(+ 3 4 5)
+ 3 4 (+ 1 4)
(+ 3 4 (+ 1 4))
all produce the value 12.
Is it possible to implement this reader as a Lisp read procedure that follows the typical expectations of a read procedure? If so, how? (I'm at a loss.)
Edit: Clarification on whitespace:
If we say that a "whitespace object" is simply a string and read, then reading the following segment:
(foo bar baz)
produces a syntax object like:
'(foo " " bar " " baz)
In other words, the whitespace between tokens is stored in the resultant syntax object.
Suppose I write a macro named ->, which takes a syntax object (scheme style macro), and whitespace? is a predicate identifying whitespace syntax objects
(define-macro (-> stx)
(let* ((stxl (syntax-object->list stx))
(obj (car stxl))
(let proc ((res empty))
(lst (cdr stxl)))
(let ((method (car lst)))
(if (whitespace? method)
; skip whitespace, recur immediately
(proc res (cdr lst))
; Insert obj as the second element in method
(let ((modified-method (cons (car method)
(cons obj (cdr method)))))
; recur
(proc (cons res modified-method) (cdr lst))))))))
The reading part of this is pretty easy. You just need a whitespace test, and then your reading function will install a custom reader character macro that detects whitespace and reads consecutive sequences of whitespace into a single object. First, the whitespace test and a whitespace object; these are pretty simple:
(defparameter *whitespace*
#(#\space #\tab #\return #\newline)
"A vector of whitespace characters.")
(defun whitespace-p (char)
"Returns true if CHAR is in *WHITESPACE*."
(find char *whitespace* :test 'char=))
(defstruct whitespace-object
characters)
Now the macro character function:
(defun whitespace-macro-char (stream char)
"A macro character function that consumes characters from
stream (including CHAR), until a non-whitespace character (or end of
file) is encountered. Returns a whitespace-object whose characters
slot contains a string of the whitespace characters."
(let ((chars (loop for c = (peek-char nil stream nil #\a)
while (whitespace-p c)
collect (read-char stream))))
(make-whitespace-object
:characters (coerce (list* char chars) 'string))))
Now the read function just has the same signature as the normal read, but copies the readtable, then installs the macro function, and calls read. The result from read is returned, and the readtable is restored:
(defun xread (&optional (stream *standard-input*) (eof-error-p t) eof-value recursive-p)
"Like READ, but called with *READTABLE* bound to a readtable in
which each whitespace characters (that is, each character in
*WHITESPACE*) is a macro characters whose macro function is
WHITESPACE-MACRO-CHAR."
(let ((rt (copy-readtable)))
(map nil (lambda (wchar)
(set-macro-character wchar #'whitespace-macro-char))
*whitespace*)
(unwind-protect (read stream eof-error-p eof-value recursive-p)
(setf *readtable* rt))))
Example:
(with-input-from-string (in "(+ 1 2 (* 3
4))")
(xread in))
(+ #S(WHITESPACE-OBJECT :CHARACTERS " ") 1
#S(WHITESPACE-OBJECT :CHARACTERS " ") 2
#S(WHITESPACE-OBJECT :CHARACTERS " ")
(* #S(WHITESPACE-OBJECT :CHARACTERS " ") 3
#S(WHITESPACE-OBJECT
:CHARACTERS "
")
4))
Now, to implement the eval counterpart that you want, you need to be able to remove whitespace objects from lists. This isn't too hard, and we can write a slightly more general utility function to do it for us:
(defun remove-element-if (predicate tree)
"Returns a new tree like TREE, but which contains no elements in an
element position which ssatisfy PREDICATE. An element is in element
position if it is the car of some cons cell in TREE."
(if (not (consp tree))
tree
(if (funcall predicate (car tree))
(remove-element-if predicate (cdr tree))
(cons (remove-element-if predicate (car tree))
(remove-element-if predicate (cdr tree))))))
CL-USER> (remove-element-if (lambda (x) (and (numberp x) (evenp x))) '(+ 1 2 3 4))
(+ 1 3)
CL-USER> (with-input-from-string (in "(+ 1 2 (* 3
4))")
(remove-element-if 'whitespace-object-p (xread in)))
(+ 1 2 (* 3 4))
So now the evaluation function is a simple wrapper around eval:
(defun xeval (form)
(eval (remove-element-if 'whitespace-object-p form)))
CL-USER> (with-input-from-string (in "(+ 1 2 (* 3
4))")
(xeval (xread in)))
15
Let's make sure that standalone whitespace objects still appear as expected:
CL-USER> (with-input-from-string (in " ")
(let* ((exp (xread in))
(val (xeval exp)))
(values exp val)))
#S(WHITESPACE-OBJECT :CHARACTERS " ")
#S(WHITESPACE-OBJECT :CHARACTERS " ")
I've been learning racket for a few days and I'm puzzled with this task, I'm trying to find empty lines in a text file and select a random empty line to INSERT the text "calculation here", this is as far as I have gotten so far.
for example: myfile.txt has the contents:
line1
line2
line3
line4
after the script is run, myfile.txt should now look like:
line1
calculation here
line2
line3
line4
or:
line1
line2
line3
calculation here
line4
un-working code below:
#lang racket
(define (write-to-file host text) (
with-output-to-file host (
lambda () (
write text))
#:exists 'replace))
(define empty-lines '()) ;store line number of empty line (if any)
(define (file-lines text-file)
(file->lines text-file))
(define (add-to-list line-num)
(set! empty-lines (cons line-num empty-lines)))
(let loop ((l (file-lines "myfile.txt")))
(cond ((null? l) #f)
(else
(printf "~s\n" (first l)) ; just for debugging
(cond ((equal? (first l) "") (add-to-list (first l)))(else #f))
(loop (rest l)))))
;now i need to select a random line from the list of empty-lines.
;and write "calculations here" to that line
there's no problem with the read lines method i am using, the problem is detecting and selecting a random empty space to insert my text.
Given a file name, you can read it into a list of lines using file->lines. So for instance:
(for ([line (in-list (file->lines "some-file"))])
(displayln (cond [(zero? (string-length line)) (make-random-line)]
[else line])))
Where make-random-line is some function you define to return a
random string, as you said you wanted to do.
The above reads the entire file into a list in memory. For larger files, it would be better to process things line by line. You can do this using the in-lines sequence:
(with-input-from-file "some-file"
(thunk
(for ([line (in-lines)])
(displayln (cond [(zero? (string-length line)) (make-random-line)]
[else line])))))
Update
Now that I understand your question:
#lang racket
(define lines (file->lines "some-file-name"))
(define empty-line-numbers (for/list ([line (in-list lines)]
[n (in-naturals)]
#:when (zero? (string-length line)))
n))
(define random-line-number (list-ref empty-line-numbers
(random (length empty-line-numbers))))
(for ([line (in-list lines)]
[n (in-naturals)])
(displayln (cond [(= n random-line-number) "SOME NEW STRING"]
[else line])))
(define (read-next-line-iter file)
(let ((line (read-line file)))
(unless (eof-object? line)
(display line)
(newline)
(read-next-line-iter file))))
(call-with-input-file "foobar.txt" read-next-line-iter)
http://rosettacode.org/wiki/Read_a_file_line_by_line#Racket
this function can help you read a file line by line.
check if the length is 0. and replace that line with the comment
look for 2 concurrent \n in the file. I am pretty sure there is a way in racket to do that. store those indices in a list select a pair randomly and replace the second \n with "calculation here\n".
I am having some trouble working out how to return a line of text if a match is found.
(set 'wireshark "http://anonsvn.wireshark.org/wireshark/trunk/manuf")
(set 'arptable (map (fn (x) (parse x " ")) (exec "arp -a")))
(define (cleanIPaddress x)
(slice x 1 -1))
(define (cleanMACaddress x)
(upper-case (join (slice (parse x ":") 0 3) ":")))
(define (addIPandMACaddress x)
(list (cleanIPaddress (nth 1 x)) (cleanMACaddress (nth 3 x))))
(set 'arplist (map addIPandMACaddress arptable))
(set 'routerMAC (last (assoc (exec "ipconfig getoption en1 router") arplist)))
(find-all routerMAC (get-url wireshark))
returns
("20:AA:4B")
so I know that the code "works"
but I would like to retrieve the full line of text
"20:AA:4B Cisco-Li # Cisco-Linksys, LLC"
This can be performed simply by using a string-split procedure that allows us to use remove-if (the Common Lisp version of filter) to search through a string split by newlines removing any lines that do not contain the string we are searching for. That would result in a list of every line containing the string. The functions we will define here are already available via various Common Lisp libraries, but for the education purposes, we will define them all ourselves. The code you need works like so:
; First we need a function to split a string by character
(defun string-split (split-string string)
(loop with l = (length split-string)
for n = 0 then (+ pos l)
for pos = (search split-string string :start2 n)
if pos collect (subseq string n pos)
else collect (subseq string n)
while pos))
; Now we will make a function based on string-split to split by newlines
(defun newline-split (string)
(string-split "
" string))
; Finally, we go through our text searching for lines that match our string.
; Make sure to replace 'needle' with the string you wish to search for.
(remove-if #'(lambda (x)
(equal 'nil (search (string-upcase "needle")
(string-upcase x))))
(newline-split haystack))
You should be able to apply this strategy to the code you posted with a few small modifications. This code was tested on SBCL 1.0.55.0-abb03f9, an implementation of ANSI Common Lisp, on Mac OS X 10.7.5.
In the end I used:
(find-all (string routerMAC ".*") (get-url wireshark))