I'm in the process of reading a flat file - to use the characters read I want to convert them into numbers. I wrote a little function that converts a string to a vector:
(defun string-to-vec (strng)
(setf strng (remove #\Space strng))
(let ((vec (make-array (length strng))))
(dotimes (i (length strng) vec)
(setf (svref vec i) (char strng i)))))
However this returns a vector with character entries. Short of using char-code to convert unit number chars to numbers in a function, is there a simple way to read numbers as numbers from a file?
In addition to Rainer's answer, let me mention read-from-string (note that Rainer's code is more efficient than repeated application of read-from-string because it only creates a stream once) and parse-integer (alas, there is no parse-float).
Note that if you are reading a CSV file, you should probably use an off-the-shelf library instead of writing your own.
Above is shorter:
? (map 'vector #'identity (remove #\Space "123"))
#(#\1 #\2 #\3)
You can convert a string:
(defun string-to-vector-of-numbers (string)
(coerce
(with-input-from-string (s string)
(loop with end = '#:end
for n = (read s nil end)
until (eql n end)
unless (numberp n) do (error "Input ~a is not a number." n)
collect n))
'vector))
But it would be easier to read the numbers directly form the file. Use READ, which can read numbers.
Note that read-like functions are affected by reader macros.
Pick an example:
* (defvar *foo* 'bar)
*FOO*
* (read-from-string "#.(setq *foo* 'baz)")
BAZ
19
* *foo*
BAZ
As you can see read-from-string can implicitly set a variable. You can disable the #. reader macro by setting *read-eval* to nil but anyway if you have only integers on the input then consider using parse-integer instead.
Related
I'm reading a file char by char and constructing a list which is consist of list of letters of words. I did that but when it comes to testing it prints out NIL. Also outside of test function when i print out list, it prints nicely. What is the problem here? Is there any other meaning of LET keyword?
This is my read fucntion:
(defun read-and-parse (filename)
(with-open-file (s filename)
(let (words)
(let (letter)
(loop for c = (read-char s nil)
while c
do(when (char/= c #\Space)
(if (char/= c #\Newline) (push c letter)))
do(when (or (char= c #\Space) (char= c #\Newline) )
(push (reverse letter) words)
(setf letter '())))
(reverse words)
))))
This is test function:
(defun test_on_test_data ()
(let (doc (read-and-parse "document2.txt"))
(print doc)
))
This is input text:
hello
this is a test
You're not using let properly. The syntax is:
(let ((var1 val1)
(var2 val2)
...)
body)
If the initial value of the variable is NIL, you can abbreviate (varN nil) as just varN.
You wrote:
(let (doc
(read-and-parse "document2.txt"))
(print doc))
Based on the above, this is using the abbreviation, and it's equivalent to:
(let ((doc nil)
(read-and-parse "document2.txt"))
(print doc))
Now you can see that this binds doc to NIL, and binds the variable read-and-parse to "document2.txt". It never calls the function. The correct syntax is:
(let ((doc (read-and-parse "document2.txt")))
(print doc))
Barmar's answer is the right one. For interest, here is a version of read-and-parse which makes possibly-more-idiomatic use of loop, and also abstracts out the 'is the character white' decision since this is something which is really not usefully possible in portable CL as the standard character repertoire is absurdly poor (there's no tab for instance!). I'm sure there is some library available via Quicklisp which deals with this better than the below.
I think this is fairly readable: there's an outer loop which collects words, and an inner loop which collects characters into a word, skipping over whitespace until it finds the next word. Both use loop's collect feature to collect lists forwards. On the other hand, I feel kind of bad every time I use loop (I know there are alternatives).
By default this collects the words as lists of characters: if you tell it to it will collect them as strings.
(defun char-white-p (c)
;; Is a character white? The fallback for this is horrid, since
;; tab &c are not a standard characters. There must be a portability
;; library with a function which does this.
#+LispWorks (lw:whitespace-char-p c)
#+CCL (ccl:whitespacep c) ;?
#-(or LispWorks CCL)
(member char (load-time-value
(mapcan (lambda (n)
(let ((c (name-char n)))
(and c (list c))))
'("Space" "Newline" "Page" "Tab" "Return" "Linefeed"
;; and I am not sure about the following, but, well
"Backspace" "Rubout")))))
(defun read-and-parse (filename &key (as-strings nil))
"Parse a file into a list of words, splitting on whitespace.
By default the words are returned as lists of characters. If
AS-STRINGS is T then they are coerced to strings"
(with-open-file (s filename)
(loop for maybe-word = (loop with collecting = nil
for c = (read-char s nil)
;; carry on until we hit EOF, or we
;; hit whitespace while collecting a
;; word
until (or (not c) ;EOF
(and collecting (char-white-p c)))
;; if we're not collecting and we see
;; a non-white character, then we're
;; now collecting
when (and (not collecting) (not (char-white-p c)))
do (setf collecting t)
when collecting
collect c)
while (not (null maybe-word))
collect (if as-strings
(coerce maybe-word 'string)
maybe-word))))
I'm running into a name collision with iterate and count standard function in the example below:
(defun svs-to-images (file)
(with-open-file (stream file)
(iterate:iter
(iterate:for line #:= (read-line stream nil nil))
(iterate:while line)
(line-to-image
(iterate:iter
(iterate:for c #:in-string line)
(iterate:with word)
(iterate:with pos #:= 0)
(iterate:with result #:= ; ---------\/ here
(make-array (list (1+ (count #\, line)))
:element-type 'fixnum))
(if (char= c #\,)
(setf (aref result pos)
(parse-integer
(coerce (reverse word) 'string))
pos (1+ pos)
word nil)
(setf word (cons c word)))
(iterate:finally result)) 28))))
The error I'm getting is:
csv-parser.lisp:19:5:
error:
during macroexpansion of
(ITERATE:ITER
(ITERATE:FOR LINE #:= ...)
(ITERATE:WHILE LINE)
...).
Use *BREAK-ON-SIGNALS* to intercept:
Iterate, in (COUNT , LINE):
Missing value for LINE keyword
Compilation failed.
And, if I understood it correctly, it is trying to use count as if it was the count driver from iterate, instead of the original function. How would I make it so that the correct count is used?
In comp.lang.lisp Chris Riesbeck offered this as a workaround for a similar question a few years ago:
(remprop 'count 'iter::synonym)
From then you need to use COUNTING as the iterate clause. CL:COUNT then should refer to the Common Lisp function. You would need to recompile the code.
This is a bug/feature of how iterate processes its body.
You can use a version of iterate from rutils - it uses keywords instead of plain symbols, so there will be no symbol clashes.
I am having some trouble working out how to return a line of text if a match is found.
(set 'wireshark "http://anonsvn.wireshark.org/wireshark/trunk/manuf")
(set 'arptable (map (fn (x) (parse x " ")) (exec "arp -a")))
(define (cleanIPaddress x)
(slice x 1 -1))
(define (cleanMACaddress x)
(upper-case (join (slice (parse x ":") 0 3) ":")))
(define (addIPandMACaddress x)
(list (cleanIPaddress (nth 1 x)) (cleanMACaddress (nth 3 x))))
(set 'arplist (map addIPandMACaddress arptable))
(set 'routerMAC (last (assoc (exec "ipconfig getoption en1 router") arplist)))
(find-all routerMAC (get-url wireshark))
returns
("20:AA:4B")
so I know that the code "works"
but I would like to retrieve the full line of text
"20:AA:4B Cisco-Li # Cisco-Linksys, LLC"
This can be performed simply by using a string-split procedure that allows us to use remove-if (the Common Lisp version of filter) to search through a string split by newlines removing any lines that do not contain the string we are searching for. That would result in a list of every line containing the string. The functions we will define here are already available via various Common Lisp libraries, but for the education purposes, we will define them all ourselves. The code you need works like so:
; First we need a function to split a string by character
(defun string-split (split-string string)
(loop with l = (length split-string)
for n = 0 then (+ pos l)
for pos = (search split-string string :start2 n)
if pos collect (subseq string n pos)
else collect (subseq string n)
while pos))
; Now we will make a function based on string-split to split by newlines
(defun newline-split (string)
(string-split "
" string))
; Finally, we go through our text searching for lines that match our string.
; Make sure to replace 'needle' with the string you wish to search for.
(remove-if #'(lambda (x)
(equal 'nil (search (string-upcase "needle")
(string-upcase x))))
(newline-split haystack))
You should be able to apply this strategy to the code you posted with a few small modifications. This code was tested on SBCL 1.0.55.0-abb03f9, an implementation of ANSI Common Lisp, on Mac OS X 10.7.5.
In the end I used:
(find-all (string routerMAC ".*") (get-url wireshark))
How can I remove a certain character from a string in Nyquist (which is very similar to xlisp) and have the result returned?
I want to count how many "A" there are in a string like "ABBAAAABBBAABAAAB". (Yes, there are only 'A's and 'B's in the string.)
Since there is no (count) function in Nyquist I tried something like
(length (remove #\B mystring))
or
(length (remove #\B mystring :test equal))
But it doesn't work.
Forgetting the character count for a moment, how can I remove the 'B's from the string?
Will there always be only As and Bs in the string? If not, you might want to do something like
(remove #\A yourstring :test-not 'char=)
According to the XLISP reference for remove, the Nyquist remove doesn't deal with strings, only lists. You need to convert a string to a list in order to operate on it this way, but there's no coerce either. It's a touch hacky, but the easiest way around it I see is to stream a string and read-char it. This will produce a list of chars that you can then manipulate with remove.
(defun string->list (a-string)
(let ((collector nil)
(stream (make-string-input-stream a-string)))
(dotimes (c (length a-string) (reverse collector))
(setf collector (cons (read-char stream) collector)))))
It should now be possible to
(remove #\A (string->list yourstring) :test-not 'char=)
I see this is an old question, but since it has over 800 views, it's perhaps worth having the simple answer:
(defun remove-char (character sequence)
(let ((out ""))
(dotimes (i (length sequence) out)
(setf ch (char sequence i))
(unless (char= ch character)
(setf out (format nil "~a~a" out ch))))))
(setf mystring "ABBAABABCCCCBBCCCCAAA")
(remove-char #\B mystring) ;returns "AAAACCCCCCCCAAA"
Database resources, that can be accessed from webpage that I'm currently working on, have unique id number with auto_increment set. So url would have to look like some.web.page.com/resource/id-number.
It would be kinda easy for user to notice that he can simply increase or decrease number at the end to get anything he pleases and while security isn't big concern in this case, I would really like to prevent that kind of behavior.
I was trying to find some function that would convert the number to random string look-a-like, but I failed (didn't really know what to put in that field on google.com ;) ). I also have my own ideas, but I prefer to use method that is already working well somewhere. The function needs to be symmetrical so I can easily generate string, and get number from that string. Any advice?
Ray Morgan gives an algorithm and an implementation in PHP. The algorithm has a few nice properties, namely:
the algorithm is deterministic, i.e., always produces the same obfuscated string for a given numeric ID value.
the obfuscation is fully invertible, i.e., if you know (only) the obfuscated value, you can extract the underlying numeric ID
doesn't yield any recognizable patterns (such as simple increasing sequences of integers)
it can detect, whether an obfuscated ID string has been tampered with
The author itself explains the basic steps as follows
Create a random number ($segment1) based on a hash of $id.
Create a second random number ($segment2) based on a hash of $segment1.
Alter $segment2 by adding or subtracting the value of $id.
Make a third hash ($segment3) from $segment1 and the altered $segment2. This hash makes it possible to detect any alteration of the encoded ID.
Concatenate the three segments into a string,
and voilà – you have your obfuscated ID.
For those like me not comfortable with PHP, a working Common Lisp port of the algorithm could look like:
#-(and) (ql:quickload "ironclad")
#-(and) (ql:quickload "trivial-utf-8")
(defpackage "HASHID"
(:use "COMMON-LISP" "IRONCLAD" "TRIVIAL-UTF-8")
(:shadowing-import-from "COMMON-LISP" "NULL"))
(in-package "HASHID")
(defparameter +secret+ "Secret Password")
(defun sha1-hex-digest (string &optional (secret +secret+))
(let ((digest (make-digest :sha1)))
(update-digest digest (string-to-utf-8-bytes string))
(update-digest digest (string-to-utf-8-bytes secret))
(let* ((result (produce-digest digest))
(length (length result))
(char-length (* length 2))
(buffer (make-array char-length :element-type 'character))
(digits "0123456789ABCDEF"))
(loop
:with wp := 0
:for byte :across result
:do (setf (char buffer (prog1 wp (incf wp))) (char digits (ash byte -4)))
(setf (char buffer (prog1 wp (incf wp))) (char digits (logand byte 15)))
:finally (return buffer)))))
(defun obfuscate-id (identifier)
(let* ((segment-1 (subseq (sha1-hex-digest (format nil "~D" identifier)) 0 16))
(segment-2 (subseq (sha1-hex-digest (concatenate 'string segment-1)) 0 8))
(decimal (parse-integer segment-2 :radix 16))
(buried-id (if (< identifier decimal) (- decimal identifier) (+ decimal identifier)))
(new-segment-2 (format nil "~8,'0X" buried-id))
(segment-3 (subseq (sha1-hex-digest (concatenate 'string segment-1 new-segment-2)) 0 8)))
(concatenate 'string segment-1 new-segment-2 segment-3)))
(defun deobfuscate-id (string)
(let* ((segment-1 (subseq string 0 16))
(segment-2 (subseq string 16 24))
(segment-3 (subseq string 24))
(expected-2 (subseq (sha1-hex-digest segment-1) 0 8))
(expected-3 (subseq (sha1-hex-digest (concatenate 'string segment-1 segment-2)) 0 8)))
(and (string-equal segment-3 expected-3)
(let* ((v1 (parse-integer segment-2 :radix 16))
(v2 (parse-integer expected-2 :radix 16)))
(abs (- v1 v2))))))
Note, that the original implementation generated a base-64 encoded string from the obfuscated ID and used that as the actual value. I did omit this step here, but it should be simple to add, in particular, if your programming language of choice comes with base-64 support.