Failing To Upload Items To Database Using Congomongo - mongodb

I'm using the congomongo library in two processes. One of the processes interfaces with the databases without error. The other process is having more trouble. If I try to interact with the database from that processes slime repl, it too is has no problem, but if I try to interact through telnet things work badly.
Command Line:
joshua#joshua-Aspire-5251:/data/db$ telnet localhost 4576
Trying ::1...
Connected to localhost.localdomain.
Escape character is '^]'.
http://news.ycombinator.com/item?id=2286907
Connection closed by foreign host.
Server's Reaction
http://news.ycombinator.com/item?id=2286907
Processing link.
Validating link.
Scraping item.
Wikifying items.
Uploading items.
From the REPL:
(process-link "http://news.ycombinator.com/item?id=2286907")
Server's Reaction:
http://news.ycombinator.com/item?id=2286907
Processing link.
Validating link.
Scraping item.
Wikifying items.
Uploading items.
The only functional difference? One actually puts the items in the database and the other doesn't. Both seem to be accomplishing everything else.
(ns hnparser.main
(:use somnium.congomongo
[hnparser.core :only [scrape-item]]))
(mongo! :db "hacker-archives")
(defn valid-link?
[link]
(do
(println "Validating link.")
(not= nil (re-matches #"http://news\.ycombinator\.com/item\?id=\d+" link))))
(defn wikify [item]
(dissoc (assoc item :wiki [{:title (:title item)
:author (:user item)
:body (:body item)
:date (:date item)
:reason "Original post."}])
:title :user :body :date))
(defn wikify-items [items]
(do
(println "Wikifying items.")
(map wikify items)))
(defn upload-item
[item]
(if (nil? (fetch-one :items :where {:id (:id item)}))
(insert! :items (assoc item :scrape-date (java.util.Date.)))))
(defn upload-items
[items]
(do
(println "Uploading items.")
(map upload-item items)))
(defn process-link
[link]
(do
(println "Processing link.")
(if (valid-link? link)
(upload-items (wikify-items (scrape-item link))))))
(ns hnparser.server
(:use clojure.contrib.server-socket
hnparser.main)
(:import [java.io PushbackReader InputStreamReader]))
(defn read-links
[is os]
(let [in-reader (PushbackReader. (InputStreamReader. is))]
(let [input (str (read in-reader))]
(do
(println input)
(process-link input)))))
(defn -main
[]
(def *server* (create-server 4576 read-links)))
Note that this successfully scrapes (verified through print statements). It just doesn't upload to the database.

Map is lazy: you should never use map when you want side effects instead of return values. You can wrap a map with dorun, but in most cases it's easier to use doseq.
Also defn has an implicit do, so (fn [x] (do (println x) (inc x))) is the same as (fn [x] (println x) (inc x)).

Related

How to return the result of FileReader. in clojurescript

I have a function that reads a file using js/FileReader.:
(defn read-file [file]
(let [js-file-reader (js/FileReader.)]
(set! (.-onload js-file-reader)
(fn [evt]
(let [result (-> evt .-target .-result)
array (js/Uint8Array. result)]
{:content array}))) ; <- This is the value that 'read-file' should return
(.readAsArrayBuffer js-file-reader file)))
The problem is that I would like it to return the value of the .-onload method of the FileReader, but I only get (of course) the value of (.readAsArrayBuffer js-file-reader file) which, naturally, is undefined.
Thank you very much!
Edit
After trying with Martin Půda's answer, I think that the problem has to do with an asyncrhonous thing. I tested this code:
(defn read-file [file]
(let [js-file-reader (js/FileReader.)
reading-result (atom)
done? (atom false)]
(set! (.-onload js-file-reader)
(fn [evt]
(let [result (-> evt .-target .-result)
array (js/Uint8Array. result)]
(reset! reading-result {:content array})
(reset! done? true)
(js/console.log "in: " (:content #reading-result)))))
(.readAsArrayBuffer js-file-reader file)
;; (while (not #done?) (js/console.log (.-readyState js-file-reader)))
(js/console.log "out: " #reading-result)
#reading-result))
I get first the log of out: undefined , and then the log of in: (with the desired result).
When I uncomment the line (while...), I get an infinite loop of 1's... So I think that the function never notices that the FileReader was done... I don't know how to solve this...
Try this:
(defn read-file [file]
(let [js-file-reader (js/FileReader.)
reading-result (atom)]
(set! (.-onload js-file-reader)
(fn [evt]
(let [result (-> evt .-target .-result)
array (js/Uint8Array. result)]
(reset! reading-result {:content array}))))
(.readAsArrayBuffer js-file-reader file)
#reading-result))
See docs for atom and reset!.
EDIT: See HTML5 FileReader how to return result?- yes, it's asynchronous. I see two possibilities:
You will write all function calls, which work with returned value, inside event listeners.
You will follow this CLJS guide for Promises and use JS Promises or clojure.core.async.
After much reading, I solved it using a callback function:
(ns lopezsolerluis.fits)
(defn read-file [file callback]
(let [js-file-reader (js/FileReader.)]
(set! (.-onload js-file-reader)
(fn [evt]
(let [result (-> evt .-target .-result)
array (js/Uint8Array. result)]
(callback array))))
(.readAsArrayBuffer js-file-reader file)))
Then read-file is called from this:
(ns lopezsolerluis.annie-web)
(defn procesar-archivo [result]
(js/console.log "Bloques: " result))
(defn input-file []
[:input {:type "file" :id "fits" :name "imagenFits" :accept "image/fits"
:on-change (fn [this]
(if (not (= "" (-> this .-target .-value)))
(let [^js/File file (-> this .-target .-files (aget 0))]
(fits/read-file file procesar-archivo)))
(set! (-> this .-target .-value) ""))}])
I added the namespaces because it surprised me that the callback machinery worked even across namespaces. Well, maybe it shouldn't surprise me; but I am learning and was a new concept for me. :)
I answer my question in case it's useful for others (it costed me a lot! :))

event handling in enfocus (clojurescript) not working

I am using enfocus for dom manipulation in ClojuseScript.
I have an event handler:
(ns numeros-linguas.script
(:require [enfocus.core :as ef]
[enfocus.events :as ev])
(:require-macros [enfocus.macros :as em]))
(defn aviso-seleciona []
(ef/at "select" (ev/listen :change
(fn [evt]
(let [lingua-id (ef/from [:select] (ef/get-prop :value))
lingua-selector (str "#" lingua-id)
cores (array "azul" "vermelho" "laranja" "verde")
linguas-visiveis (when-let [r (ef/from [:#resultado :> :div.visivel] (ef/get-attr :id))]
(-> r
list
flatten))]
(ef/at "#resultado" (ef/append (ef/from lingua-selector identity)))
(ef/at lingua-selector (ef/remove-class "invisivel"))
(ef/at lingua-selector (ef/add-class "visivel"))
(map #(do
(ef/at (str "#" %)
(ef/add-class %2))
(map (fn [cor]
(ef/at (str "#" %)
(ef/remove-class cor)))
(remove #{%2} cores)))
linguas-visiveis cores))))))
(set! (.-onload js/window)
aviso-seleciona)
The tricky part is in the map. I want to remove some css classes from some div elements and add some others at these elements. But it is not working. All the other elements in the form are working except this one. But if I copy and paste the form to a repl connected to that page and run the code it works as expected. I tryed to comment parts of the code to understand what is going on but with no hope.
I've noticed you are doing side-effects in map, which is not going to work because map is lazy, i.e. it won't evaluate unless you try to use sequence it produces, which explains why it works in the REPL, but not in the browser.
For quickest and dirtiest fix, try using dorun to force evaluation of your sequence: (dorun (map #(do ...... ).
or, idiomatically, rewrite your code to something resembling this (which is not very idiomatic):
(doseq [[el cor-to-add] (map vector linguas-visiveis cores)]
(ef/at (str "#" el)
(ef/add-class cor-to-add))
(doseq [cor-to-remove (remove #{cor} cores)]
(ef/at (str "#" el)
(ef/remove-class cor-to-remove)))
)
or this, removing all classes from every element, first:
(doseq [el linguas-visiveis
cor cores]
(ef/at (str "#" el)
(ef/remove-class cor)))
and adding only required classes later:
(doseq [[el cor] (map vector linguas-visiveis cores)]
(ef/at (str "#" el)
(ef/add-class cor)))

Check functions for equality inside a macro

I have a macro where I need to check whether first element of the passed form is exactly the clojure.core/not function. How could I achieve that?
(defmacro foo [form]
(println (= clojure.core/not (first form)))) ; expect true here, but its false
And I use that macro like so:
(foo (not smthng))
I've heard about resolving symbols with #' but how could it help me in this case (if it even could be helpful)?
Any thoughts are welcome.
In the macro above there is no syntax quoting/namespacing of symbols going on, so the first thing in the form would just be 'not.
(defmacro foo [form]
(println (first form)))
(foo (not nil))
;=> not
So a simple macro to check for the first symbol being not would be:
(defmacro foo [form]
(println (= 'not (first form))))
EDIT
To ensure that you have the correct clojure.core/not (and not some other not that might have been referred), you can use the resolve function to get a fully qualified var.
(defmacro foo [form]
(if (= #'clojure.core/not (resolve (first form)))
(println "(first form) is clojure.core/not")
(println "(first form) is not clojure.core/not")))
(foo (not nil))
;;=> "(first form) is clojure.core/not"
(foo (+ 3 4))
;;=> "(first form) is not clojure.core/not"

Clojure Korma: Cannot run an aggregate count

I'm trying to run a simple query using Clojure+Korma to extract the number of records.
This is what I'm trying to do:
(defmacro number-of [ref & filter]
`(let [basetmp# (-> (kc/select* ~ref)
(kc/aggregate (count :*) :cnt))]
(if ~filter
(-> basetmp#
(kc/where ~filter))
basetmp#)))
However if I try to use this macro I get an error message saying:
Wrong number of args (2) passed to: core$count
The query would perfectly work if executed in a function but there's something wrong/missing
in the macro I cannot spot :(
Thanks,
Nico
As noted by ponzao you are picking up the wrong count.
Looking at the macroexpansion
(number-of 'foo) ;; expands to....
(clojure.core/let [basetmp__9167__auto__ (clojure.core/->
(korma.core/select* 'foo)
(korma.core/aggregate
(clojure.core/count :*)
:cnt))]
(if nil
(clojure.core/-> basetmp__9167__auto__ (korma.core/where nil))
basetmp__9167__auto__))
So you need to prevent the count in your macro being expanded as clojure.core/count, you can do this with a unquote/quote thus:
(defmacro number-of [ref & filter]
`(let [basetmp# (-> (kc/select* ~ref)
(kc/aggregate (~'count :*) :cnt))]
(if ~filter
(-> basetmp#
(kc/where ~filter))
basetmp#)))
Which then expands as expected...
(clojure.core/let [basetmp__9137__auto__ (clojure.core/->
(korma.core/select* 'foo)
(korma.core/aggregate
(count :*)
:cnt))]
(if nil
(clojure.core/-> basetmp__9137__auto__ (korma.core/where nil))
basetmp__9137__auto__))
The resultant SQL looks reasonable:
(kc/as-sql (number-of 'foo))
"SELECT COUNT(*) \"cnt\" FROM \"foo\""
UPDATE:
From the comments "What does the count actually represent?" - If you realize that kc/aggregate is also a macro and that the arguments are a 'SQL aggregate' DSL of sorts, then you can expand the kc/aggregate call too. You find there's a function, parse-aggregate in engine.clj where it eventually maps to korma.sql.fn/agg-count:
(clojure.core/let [q__2640__auto__ (kc/select* 'foo)]
(korma.sql.engine/bind-query
q__2640__auto__
(clojure.core/let [res__2641__auto__ (korma.core/fields
q__2640__auto__
[(clojure.core/->
q__2640__auto__
(korma.sql.fns/agg-count
:*))
:cnt])]
(if nil
(korma.core/group res__2641__auto__ nil)
res__2641__auto__))))

Clojure/postgresql: How do I access the enum values from the Jdbc4Array results?

I have an enum in postgresql defined like so:
create type color as enum ('yellow', 'purple', 'white', 'black');
And I can get to the Jdbc4Array like so:
(def colors
((first (sql/with-connection db/db
(sql/with-query-results res
["select enum_range(null::color)"]
(doall res)))) :enum_range))
This shows an object like this:
#<Jdbc4Array {yellow,purple,white,black}>
But trying the usual things throws an exception:
(.getArray colors) => stream closed
So I figure I need to access the array before the connection is closed:
(def colors
((sql/with-connection db/db
(sql/with-query-results res
["select enum_range(null::color)"]
(.getArray ((first (doall res)) :enum_range))))))
But in this case I get this exception:
Method org.postgresql.jdbc4.Jdbc4Array.getArrayImpl(long,int,Map)
is not yet implemented.
Sinister. What can I do here?
There is something very strange going on with the Postgresql Jdbc4Array.getArray() implementation, I couldn't get it to work. But, I have some success with .getResultSet() :
user=> (with-connection db (with-query-results rs ["select enum_range(null::color)"]
(.getResultSet (get (first(doall rs)) :enum_range))))
#<Jdbc4ResultSet org.postgresql.jdbc4.Jdbc4ResultSet#17cff66>
Now, array's contents could be accessed via standard ResultSet interface. I've copied some code from clojure.contrib.sql to do it:
(defn resultset-seq
[^java.sql.ResultSet rs]
(let [rsmeta (. rs (getMetaData))
idxs (range 1 (inc (. rsmeta (getColumnCount))))
keys (map (fn [i] (. rsmeta (getColumnLabel i))) idxs)
check-keys (or (apply distinct? keys)
(throw (Exception. "ResultSet must have unique column labels")))
row-struct (apply create-struct keys)
row-values (fn [] (map (fn [^Integer i] (. rs (getObject i))) idxs))
rows (fn thisfn []
(when (. rs (next))
(cons (apply struct row-struct (row-values)) (lazy-seq (thisfn)))))]
(rows)))
which gives (sorry for quick-hack style code)
user=> (with-connection db
(with-query-results rs ["select enum_range(null::color)"]
(get (first (resultset-seq
(.getResultSet (get (first(doall rs))
:enum_range))))
"VALUE")))
#<PGobject yellow>