core.async: async java.jdbc

[Alice]

(defonce ^ExecutorService db-thread-pool (Executors/newFixedThreadPool 8))

(defn db-thread-call

  [f]

  (let [c (chan 1)]

    (.execute db-thread-pool

              (fn []

                (let [ret (try

                            (f)

                            (catch Throwable t

                              nil))]

                  (when-not (nil? ret)

                    (>!! c ret))

                  (close! c))))

    c))

(defmacro db-thread

  [& body]

  `(db-thread-call (fn [] ~@body)))

(defn insert-async!

  [& args]

  (db-thread

    (try

      (apply jdbc/insert! args)

      (catch Throwable t

        t))))

(go

  (jdbc/db-transaction [t-con db-spec]

    (<! (insert-async! t-con :fruit {:name "apple"}))))

Exception in thread "async-dispatch-5" java.lang.AssertionError: Assert failed: <! used not in (go ...) block

So, do we need a db-transaction that does all the work inside a macro instead of wrapping the body into a function and passing it to another function?

[Sean Corfield]

On Wed, Jul 31, 2013 at 10:29 AM, Alice <doff...@gmail.com> wrote:
> (go
> (jdbc/db-transaction [t-con db-spec]
> (<! (insert-async! t-con :fruit {:name "apple"}))))

Does this work:

(jdbc/db-transaction [t-con db-spec]
(go

(<! (insert-async! t-con :fruit {:name "apple"}))))

--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Alice]

It doesn't produce a compile time error but I think it's not the correct code because the transaction can be committed while insert-async! is still executing.

[Timothy Baldridge]

Why not use <!! ?

Timothy

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Alice]

I have an async http handler and I don't want it to consume a thread.

[Timothy Baldridge]

DB work is a IO operation, and IO shouldn't be done inside a go block (go blocks use limited thread pools). 

So what about something like this?

(defn handle-db [chan]
 (thread
    (while true
      (let [[data result] (<! c)]
         (run-db-transaction-as-normal data)
         (>! result :done)))))

Now spin up a few of these:

(def db-chan (let [c (chan 4)]
                         (dotimes [x 4] 

                           (handle-db c))
                         c))

Now it's trivial to use this from a go block:

(defn db-async [data]

  (go
    (let [c (chan)]
      (>! db-chan [data c])
      (<! c))))

So this is the pattern: create processors using threads, then send data to those processors via a shared channel, passing in a response channel. 

Anyways, that's the way I'm thinking these days. 

Timothy

[Alice]

(defn insert-async!

  [& args]

  (db-thread

    (try

      (apply jdbc/insert! args)

      (catch Throwable t

        t))))

db-thread is a macro that runs the body in a fixed thread pool, so IO isn't done inside a go block.

[Sean Corfield]

On Wed, Jul 31, 2013 at 10:58 AM, Alice <doff...@gmail.com> wrote:
> It doesn't produce a compile time error but I think it's not the correct
> code because the transaction can be committed while insert-async! is still
> executing.

Right. I was just showing how to avoid the compile error (because you
need <! in the context of the go block.

Timothy provided a good solution I think.

[Alice]

What if I want to run a select query in the middle of a transaction?

On Thursday, August 1, 2013 3:33:29 AM UTC+9, tbc++ wrote:

[Jonah Benton]

These are interesting code listings and questions, but they conflate the syntactic detail that core.async's go is implemented as a macro, with the semantic intention to support designs around lightweight data flow between independent code blocks. 

It might be of value to check out Go, the language, with its channels and go-routines, which were part of the inspiration for core.async. In Go, channels and go-routines are language rather than library features. Go also lacks macros. In that context, answers to these questions are clear: e.g. if you want to run an insert asynchronously and then run a select following the insert in the context of the same transaction, the only way to do it looks like Timothy's example, where the code implements a data flow- the particular data to insert and select is sent over a channel, handled by a function running in a go routine whose semantic purpose is to wait on an input channel, do the database operation, and stick the result on an output channel.

From on the other thread, Go doesn't have exceptions, but error handling between go routines is usually handled with a separate error channel that again serves to support data flow, rather than control flow.  Similarly, the reason that a special channel-aware throw wasn't included in core.async is that the idea isn't to use go blocks in a lexical, control-flow sense- even though that's what the implementation expands to- it's to support completely different data-flow oriented designs.

Hope that helps...