Simple Clojure clarification re: hashes / values - hash

I'm just getting started with Clojure, trying to wrap my brain around functional/immutable programming.
My simple problem is this - I have a map with two values, which I want to transfer from one, to another (inside the same map). Can this be done with a simple function? Or do I have to get into refs and atoms?
e.g.
(def bucket {:volume 100 :rate 10 :poured 0})
How do I move volume to poured at rate?
(defn pour
[bucket]
?
)
; -> {:volume 90 :rate 10 :poured 10}

It's a bit of both. The bucket is a value and a function will take that value and produce a new value that is based on the original (and shared unchanged values):
user> (def bucket {:volume 100 :rate 10 :poured 0})
#'user/bucket
user> (assoc bucket
:volume (- (:volume bucket) (:rate bucket))
:poured (+ (:poured bucket) (:rate bucket)))
{:rate 10, :volume 90, :poured 10}
it leaves the original bucket in the var named bucket
user> bucket
{:rate 10, :volume 100, :poured 0}
so we can define a function on bucket values:
user> (defn pour [bucket]
(assoc bucket
:volume (- (:volume bucket) (:rate bucket))
:poured (+ (:poured bucket) (:rate bucket))))
#'user/pour
user> (pour bucket)
{:rate 10, :volume 90, :poured 10}
user> bucket
{:rate 10, :volume 100, :poured 0}
This will be very useful when we want to express the idea of a bucket "identity". An identity carries values from state to state in a very useful way. I'll use an atom because I want synchronous updates to a single identiy/thing at a time.
user> (def bucket (atom {:volume 100 :rate 10 :poured 0}))
#'user/bucket
user> (swap! bucket pour)
{:rate 10, :volume 90, :poured 10}
user> (swap! bucket pour)
{:rate 10, :volume 80, :poured 20}
user> (swap! bucket pour)
{:rate 10, :volume 70, :poured 30}
and now our bucket is changing over time, and these changes are viable to everyone. It's worth noting that the function that produces the new values is totally independent of the fact that it's being used to change an atom and can be reused elsewhere. Some Clojurians refer to this as "simple"
Often you will want to compute values like "what is the result of pouring the bucket three times, which is nicely expressed by nesting each call to pour inside the next like so:
user> (pour (pour (pour bucket)))
{:rate 10, :volume 70, :poured 30}
This pattern is so common there is a two character macro to express it: ->
user> (-> bucket pour pour pour)
{:rate 10, :volume 70, :poured 30}
In practice I see clojure compose function via nesting and threadding much more often than through the mutable state sytems (atoms, refs, agents, vars)

Related

How to loop through dates in Racket?

I want to loop through dates from the current date to some stop date, like the following example in Python:
import datetime
def count_dates(stop_date):
step = datetime.date.today()
while step >= stop_date:
yield step
step = step - datetime.timedelta(days=1)
def main():
for step in count_dates(datetime.date(2018, 1, 1)):
print(step.isoformat())
I don't see any date calculation functions in Racket, however. How can this be done?
If you want to use racket/date, you can accomplish your goal with find-seconds, seconds->date, current-seconds, and a simple arithmetic:
#lang racket
(require racket/date
racket/generator)
(define 1day (* 60 60 24))
(define (count-dates year month day)
(define target (find-seconds 0 0 0 day month year))
(in-generator
(let loop ([current (current-seconds)])
(when (>= current target)
(yield (seconds->date current))
(loop (- current 1day))))))
(date-display-format 'iso-8601)
(for ([dt (count-dates 2019 2 1)])
(displayln (date->string dt)))
This outputs:
2019-02-11
2019-02-10
2019-02-09
2019-02-08
2019-02-07
2019-02-06
2019-02-05
2019-02-04
2019-02-03
2019-02-02
2019-02-01
That is, from today (Feb 11, 2019) to Feb 1, 2019.
You can also use gregor or srfi-19 which are alternative date libraries that have a notion of time duration/difference.
Here is how you can do it using the date arithmetic provided by gregor. Specifically the +days and -days functions are helpful here:
#lang racket
(require gregor
racket/generator)
;; Date -> [Sequenceof Date]
(define (count-dates stop-date)
(in-generator
(let loop ([step (today)])
(when (date>=? step stop-date)
(yield step)
(loop (-days step 1))))))
(for ([step (count-dates (date 2018 1 1))])
(displayln (date->iso8601 step)))
And if you need something more general than "some number of days", there's also functions like +date-period and -date-period.

Clojure <-> Monger <-> MongoDB: reading and writing a set

I'm using Monger to store data in MongoDB. I would like to store a Clojure set. Reading and writing the set does work, but it is returned as a list. I suspect MongoDB doesn't actually support the set data type, so the Monger client doesn't either, but hopefully I'm wrong.
Is there some way to have Monger return the set I stored with the correct type?
Minimal Leiningen example is below (which is just the most basic Monger example):
> lein new mongo-test
> cd mongo-test
Edit project.clj to add Monger dependency:
(defproject mongo-test "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.5.1"]
[com.novemberain/monger "2.0.0-rc1"]])
Start the REPL:
> lein repl
Insert and read back a set - note the found value is a list, not a set:
user=> (require '[monger.core :as mg])
nil
user=> (require '[monger.collection :as mc])
nil
user=> (def conn (mg/connect))
#'user/conn
user=> (def db (mg/get-db conn "monger-test"))
#'user/db
user=> (mc/remove db "things")
#<WriteResult { "serverUsed" : "127.0.0.1:27017" , "n" : 1 , "connectionId" : 248 , "err" : null , "ok" : 1.0}>
user=> (mc/insert db "things" {:set #{"A" 1}})
#<WriteResult { "serverUsed" : "127.0.0.1:27017" , "n" : 0 , "connectionId" : 248 , "err" : null , "ok" : 1.0}>
user=> (.next (mc/find db "things"))
{"_id" #<ObjectId 537ce43130045df5b9ff1102>, "set" [1 "A"]}
user=> (get (.next (mc/find db "things")) "set")
[1 "A"]
user=> (type (get (.next (mc/find db "things")) "set"))
com.mongodb.BasicDBList
user=> (set (get (.next (mc/find db "things")) "set"))
#{1 "A"}
user=> (type (set (get (.next (mc/find db "things")) "set")))
clojure.lang.PersistentHashSet
Obviously I can pass the result to set, as in the last two lines, but I don't want to have to call that for each specific key. Is there a way to have this happen transparently?
You are unfortunatly not wrong, MongoDB does not have a build in data type for sets. Your last example is a perfetly valid way of handling this, as would calling
(update-in results [..path.here.. "set"] set)
in the cases where you really need to use sets.

Using FuzzyJess: fuzzy-match is not defined

I have downloaded an installed Jess plugin for eclipse and FuzzyJ Toolkit with it. Now I am trying to run the following simple code:
(import nrc.fuzzy.*)
(bind ?temperature (new FuzzyVariable "temperature" 0 100 "°C"))
(bind ?pressure (new FuzzyVariable "pressure" 0.1 50 "MPa"))
(?temperature addTerm "cold" (new RightLinearFuzzySet 6 20))
(?temperature addTerm "warm" (new TrapezoidFuzzySet 15 20 25 30))
(?temperature addTerm "hot" (new LeftLinearFuzzySet 25 50))
(?pressure addTerm "low" (new RightLinearFuzzySet 0.9 5))
(?pressure addTerm "medium" (new TrapezoidFuzzySet 2 8 14 28))
(?pressure addTerm "high" (new LeftLinearFuzzySet 22 50))
(deffunction about (?number ?fuzzyVariable)
(bind ?delta (- (?fuzzyVariable getMaxUOD) (?fuzzyVariable getMinUOD)))
(new FuzzyValue ?fuzzyVariable
(new TriangleFuzzySet
(- ?number (* ?delta 0.01))
?number
(+ ?number (* ?delta 0.01))
)
)
)
; (printout t ((about 34 ?temperature) plotFuzzyValue "*"))
(assert (Temperature (about 43 ?temperature)))
(defrule trivial-rule
(Temperature ?t & :(fuzzy-match ?t "hot"))
=>
(assert (Pressure (new FuzzyValue ?pressure "low")))
)
(defrule printing
(Pressure ?p)
=>
(printout t (?p plotFuzzyValue "*"))
)
(run)
However when I run the code, I only get an exception saying that function fuzzy-match is not defined. I already changed the main class to FuzzyMain. I also tried to run FuzzyConsole and load the code from there, as well as trying to add FuzzyFunctions to the engine using addPackage, but in any case I still get that error.
Although Jess in Action unfortunately doesn't tell you this, you have to explicitly load the fuzzy functions, even when launching with FuzzyMain. Add the following line near the top of your Jess code and it will work as desired:
(load-package nrc.fuzzy.jess.FuzzyFunctions)

Best way to read (MATLAB) structures from a text file in Clojure / Incanter

I exported my matlab structs to text files so that I can read them in clojure.
I have a text file like:
name
Ali
age
33
friends-ages
30
31
25
47
know I can read this file, but what's the clojure way to convert it into something like:
(def person1
{:name "Ali"
:age 33
:friends-ages [30 31 25 47]})
or lets make it easier:
name
Ali
age
33
to:
(def person1
{:name "Ali"
:age 33})
Assuming each file has a single record,
(defn parse [f]
(let [[_ name _ age _ & friends] (.split (slurp f) "\n")]
{:name name :age age :friends (map read-string friends)}))
(parse "../../../Desktop/t.txt")
you get,
{:name "Ali", :age "33", :friends-ages (30 31 25 47)}
I guess you would need to manually do this conversion from text file (which isn't in any standard serialization formats) to the required set format. If possible, I would suggest that you should extract the MATLAB data in JSON format and that would be easy to read in clojure using the json library.
Here's a solution for your easier problem:
(ns example
(:require [clojure.string :as str]))
(def input "name\nAli\nage\n33\n") ; you'd get this by reading the file
(defn process [data]
(->> (str/split data #"\n")
(partition 2)
(map (fn [[k v]]
[(keyword k)
(try
(Integer/valueOf v)
(catch NumberFormatException _
v))]))
(into {})))
(prn (process input))
The full problem requires better specification of the format: how do you know that the list of ages is done?
I ended up writing an .m file to export my .mat files to text, then imported them by something like the following:
(use '[clojure.string :only (split join)])
(defn kv [[k v]]
[(keyword k) v])
(let [a (split (slurp (java.io.FileReader. fileName)) #"\n")]
(def testNeuron (into {} (map kv (partition 2 a)))))
Thanks to the answers by Hamza and Jouni.
The actual code is more like this:
(def dataPath "/home/ali/Dropbox/Projects/Neurojure/")
(def testFileName "dae062.c1.cylinder.DID.txt")
(defn kv [[k v]]
[(keyword k)
(if (< 0 (count (re-seq #"\d\d\d\dSpikes|\d\d\d\dOcodes|\d\d\d\dOSpikes" k)))
(into [] (map (fn [x] (Integer/valueOf x)) (split v #",")))
(try (Integer/valueOf v)
(catch NumberFormatException _ v)))])
(let [a (split (slurp (java.io.FileReader. (str dataPath testFileName))) #"\n")]
(def testNeuron (into {} (map kv (partition 2 a)))))
and the converted data file looks like this:
ConvertDate
18-Nov-2011
Name
/bgc/data/dae/062/dae062.mat
frameperiod
83.2500000037253
Spike2Version
1.27
unstored
167
rfstr
cm=rf-3.10,9.61:0.00x0.00,-135deg pe20 -2.0,-3.0 fx=0.00,fy=0.00cm=rf-3.10,9.61:0.00x0.00,-135deg pe20 -2.0,-3.0 fx=0.00,fy=0.00
rf
-3.1,9.61,0,0,-135,20,-2,-3,0,0
trange
47371169.75,100193717.5
psych
1
rc
0
Options
expname
cylinder.dxXIdD
Start
94745610.75
End
100193717.5
Area
MT
Ex
perimenter Ali
Trial=>0001Start
47377343
Trial=>0001TrialStart
47377343
Trial=>0001End
47397224.25
Trial=>0001dur
19881.2500000075
Trial=>0001uStim
0
Trial=>0001op
40980
Trial=>0001Trial
1860
Trial=>0001id
15897
Trial=>0001Startev
47377149.5
Trial=>0001serdelay
-46.25
Trial=>0001bstimes
47377195.75
Trial=>0001delay
147.25
Trial=>0001TrueEnd
47397224.25
Trial=>0001CorLoop
0
Trial=>0001dx
0
Trial=>0001Spikes
-1402,-1232,1577,1931,2165,2222,2478,2773,2903,3229,3745,3820,4071,4588,4920,5141,5752,6440,6490,6664,6770,7042,7958,8081,8307,8622,8732,9021,9082,9343,9619,9695,9877,10357,10668,10943,11105,11364,11720,12272,12499,12762,12907,13621,14121,14351,14542,14588,15104,15420,15501,16331,16596,16843,17476,17698,17996,18169,18401,18532,18706,19029,19081,19418,19603,19750,20222
Trial=>0001count
65
Trial=>0001OptionCode
+do+aa+sq+72+se+fc+fb+ap+wt+cf+ws+ts+sw+bw+cs+2a+bm+gm+x2+sp+da+ao+cS+C3+CF
Trial=>0001me
0
Trial=>0001OSpikes
-1976,-1802,-1423,-1360,-1268,-1248,-1140,-244,-220,-164,632,681,730,760,779,786,867,879,924,1062,1161,1252,1268,1431,1533,1632,1946,2210,2235,2273,2285,2296,2305,2496,2532,2541,2732,2787,2806,2822,2840,3095,3292,3431,3598,3614,3837,4100,4482,4504,4515,4651,4768,4794,4936,5020,5160,5184,5300,5314,5710,5764,6431,6453,6471,6553,6561,6584,6791,7018,7124,7880,7905,7940,7968,8011,8315,8330,8352,8568,8666,8748,8756,8766,8797,8836,9038,9297,9328,9360,9471,9632,9639,9721,9939,10196,10363,10375,10387,10410,10931,10953,10969,10986,11038,11118,11135,11405,11692,12018,12163,12258,12492,12512,12525,12884,12899,12919,13156,13183,13638,13674,13842,13988,14110,14298,14310,14321,14606,14617,15124,15132,15150,15289,15341,15620,16293,16305,16342,16364,16441,16604,16692,16932,16997,17059,17086,17210,17368,17495,17626,17639,17651,17677,17718,18013,18247,18353,18553,18691,18722,18887,18941,19438,19774,19938,19959,19967,20004,20240,20306,20500,20623
Trial=>0002Start
47406914
Trial=>0002TrialStart
47406914
Trial=>0002End
47426795.25
Trial=>0002dur
19881.2499999925
...

Racket Server & PostgreSQL - BLOB upload/download without saving to memory or disk

I am trying to make a servlet for the Racket Web Server that would allow a user to upload pictures to the site and display the already uploaded files as images on the same page. I would like to stream the pictures directly in and out of a PostgreSQL database, rather than saving them to a temporary file on disk or in memory. Is it possible? If so, what is the best way to do it? Can it be done with a stateless servlet? Any help is greatly appreciated!
Should be. I recommend the db package from PLaneT (because I wrote it). You can read the docs online.
The PostgreSQL table should have a bytea field for the image contents; on the Racket side it will be represented as a byte string.
In your servlet, you should probably return a response/full structure with the image contents. You'll have to deal with the return code, MIME type, etc yourself. (See the example in the documentation.)
In the name of science, I am posting one half of the answer to my own question. This page will show images that are already in the database. The upload page is still an open question.
Ryan Culpepper helped me in private correspondence beyond of what is posted here. I thank him for his help. All things that may look like black magic come from him, and all clumsy goofs are mine. I will be grateful for all suggestions on how to improve the code.
#lang racket
#|
================================================================================================================
We are assuming that the PostgreSQL database we are connecting to
has a table "person" with columns
"id", "firstname", "lastname" and "portrait".
The "portrait" column contains the OID of a BLOB
that stores the image file we want to display.
Suppose further that the table "person" has a legitimate entry with
id=22, firstname="John", lastname="Doe"
Then the page
http://127.0.0.1/page/22
should display greetings "Hello, John Doe!"
and show the portrait of the person below the greeting.
The portrait itself should be at
http://127.0.0.1/portrait/22.jpg
The program should be run via Racket -t "<filename>"
after defining the environment variables
"DB_USER", "DB_NAME", "DB_PORT", "DB_PASSWORD".
================================================================================================================
|#
(require
web-server/servlet
web-server/servlet-env
web-server/dispatch
web-server/stuffers/hmac-sha1
web-server/http
web-server/http/response-structs
(planet ryanc/db:1:4)
(planet ryanc/db:1:4/util/connect)
net/base64)
;---------------------------------------------------------------------------------------------------------------
; response
;---------------------------------------------------------------------------------------------------------------
(define (start given-request)
(site-dispatch given-request))
(define-values (site-dispatch given-request)
(dispatch-rules
[("page" (integer-arg)) show-page]
[("portrait" (string-arg)) show-portrait]))
(define (show-page given-request given-person-id)
(let* ( [db-person_firstname_lastname
(query-maybe-row my-connection
"SELECT firstname, lastname FROM person WHERE id = $1"
given-person-id)]
[my-firstname (vector-ref db-person_firstname_lastname 0)]
[my-lastname (vector-ref db-person_firstname_lastname 1)])
(response/xexpr
`(html ([xmlns "http://www.w3.org/1999/xhtml"])
(head
(title "Page with a portrait"))
(body
(div ([id "greetings"])
,(string-append
"Hello, " my-firstname " " my-lastname "! "))
(img ( [src ,(string-append "/portrait/"
(number->string given-person-id) ".jpg")])))))))
(define (show-portrait given-request given-portrait-file)
(let* ( [my-user-id (car (regexp-match #rx"^([0-9]+)"
given-portrait-file))]
[my-portrait-oid (query-value my-connection
"SELECT portrait FROM person WHERE id = $1"
(string->number my-user-id))]
[STREAMOUT_CHUNK_SIZE 1000]
[INV_READ #x00040000])
(response
200 ; code
#"Okay" ; message
(current-seconds) ; seconds
#"image/jpeg" ; mime type
empty ; headers
(lambda (given-output-stream) ; body generator
(start-transaction my-connection)
(define object-descriptor
(query-value my-connection
"SELECT LO_OPEN( $1, $2 )" my-portrait-oid INV_READ))
(define (stream-next-chunk)
(begin
(define my-next-chunk
(query-value my-connection
"SELECT LOREAD( $1, $2 )"
object-descriptor STREAMOUT_CHUNK_SIZE))
(if (> (bytes-length my-next-chunk) 0)
(begin
(write-bytes my-next-chunk given-output-stream)
(stream-next-chunk)
#t)
#f)))
(stream-next-chunk)
(commit-transaction my-connection)))))
;---------------------------------------------------------------------------------------------------------------
; database connection
;---------------------------------------------------------------------------------------------------------------
(define my-connection
(virtual-connection
(connection-pool
(lambda ()
(eprintf "(Re)establishing database connection...\n")
(postgresql-connect
#:user (getenv "DB_USER")
#:database (getenv "DB_NAME")
#:port (string->number (getenv "DB_PORT"))
#:socket #f
#:password (getenv "DB_PASSWORD")
#:allow-cleartext-password? #f
#:ssl 'optional ; other choices: 'yes 'no
)))))
;---------------------------------------------------------------------------------------------------------------
; servlet parameters
;---------------------------------------------------------------------------------------------------------------
(serve/servlet start
#:command-line? #t ; #t to use serve/servlet in a start up script for a Web application, and don't want a browser opened or the DrRacket banner printed
#:connection-close? #f ; #t to close every connection after one request. (Otherwise, the client decides based on what HTTP version it uses.)
#:launch-browser? #f
#:quit? #f ; #t makes the URL "/quit" end the server
#:banner? #t ; #t to print an informative banner
#:listen-ip #f ; give an IP to accept connections from external machines
#:port 80 ; 443 is the default for SSL, 80 - for open connections
#:servlet-regexp #rx"" ; #rx"" captures top-level requests
#:stateless? #t
#:server-root-path ; where the server files are rooted, default=(the distribution root)
(build-path ".")
#:ssl? #f
#:log-file (build-path "server.log"))