Promise/Deliver use cases
Baishampayan Ghose
I am trying to understand the use-cases of the new promise/deliver
feature in Clojure. I have tried using them, and they seem to be
pretty straight-forward to use, but unfortunately I haven't been able
to understand its use-cases.
It would be great if someone pointed out some example usage of promise/deliver.
Regards,
BG
--
Baishampayan Ghose
Laurent PETIT
programming paradigm".
It's an easy way to make a computation done in thread A (and using a
pre-declared promise) block until thread B has delivered the promise
(given it its value).
The book CTM covers dataflow programming :
http://www.info.ucl.ac.be/~pvr/book.html
Now, to be honest, i still haven't read the related parts of the book,
and I'm unable to give more concrete examples yet :-)
2010/1/21 Baishampayan Ghose <b.g...@ocricket.com>:
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>
Sean Devlin
interacting with physical devices, especially robots. Suppose you
have a 200 ton press, with a safety sensor. You have the following
code:
(def is-it-safe? (promise))
(defn activate-press []
(if @is-it-safe?
(go!)))
(defn poll-sensor []
(loop []
(if (= (read-sensor) :safe)
(deliver is-it-safe :safe)
(do (Thread/sleep 50) recur))))
Now, the poll-sensor fn is usually running in a daemon thread
somewhere else. The call to activate-press will happen at a seemingly
random time, at least as far as poll-sensor is concerned. The promise
forces the press to wait unit it's safe to activate.
There are all sort of other conditions where this makes sense when
dealing with mechanics. This is just one.
HTH,
Sean
On Jan 21, 6:40 am, Laurent PETIT <laurent.pe...@gmail.com> wrote:
> In a nutshell, those are the building blocks for the "dataflow
> programming paradigm".
>
> It's an easy way to make a computation done in thread A (and using a
> pre-declared promise) block until thread B has delivered the promise
> (given it its value).
>
> The book CTM covers dataflow programming :http://www.info.ucl.ac.be/~pvr/book.html
>
> Now, to be honest, i still haven't read the related parts of the book,
> and I'm unable to give more concrete examples yet :-)
>
> 2010/1/21 Baishampayan Ghose <b.gh...@ocricket.com>:
Dave Jack
that uses `promise`:
http://map-str.appspot.com/2010/01/Channels-in-Clojure
David Nolen
Laurent PETIT
maybe I don't understand your example well, but anyway, here are my
remarks / concerns:
* You're example is not at all exempt from multithreading: you use
futures, and in the real scenario, the calls to deliver would have to
occur from a thread able to pull from or be notified by the progress
of the tests
* If one wants all the graphical values to be update "on the fly",
then one will have to place the calls to deref in independent threads,
wouldn't it be too heavy to have so many threads (perhaps 100, 10000
?) just for updating a single node of a tree in the overall
application ?
2010/1/22 David Nolen <dnolen...@gmail.com>:
Chouser
I've used them to convert a callback-based (continuation-passing
style, if you will) API into a blocking one. The lib I was using
provides something you can call like:
(rpc-call destination method-args done)
Where 'done' is a function that gets called with the results of
the remote procedure call. But I want to write a function that
does rpc but *returns* the result, so...
(let [p (promise)]
(rpc-call destination method-args #(deliver p %))
@p)
This will work just fine whether rpc-call calls 'done'
synchronously, or if it returns right away and 'done' is called
by some other thread later.
--Chouser
http://joyofclojure.com/
David Nolen
David,
maybe I don't understand your example well, but anyway, here are my
remarks / concerns:
* You're example is not at all exempt from multithreading: you use
futures, and in the real scenario, the calls to deliver would have to
occur from a thread able to pull from or be notified by the progress
of the tests
* If one wants all the graphical values to be update "on the fly",
then one will have to place the calls to deref in independent threads,
wouldn't it be too heavy to have so many threads (perhaps 100, 10000
?) just for updating a single node of a tree in the overall
application ?
Steven E. Harris
> It would be great if someone pointed out some example usage of
> promise/deliver.
I've used them in cases where I think I need a future, but I don't want
to wrap the future around some function just to catch and propagate its
result elsewhere. In Clojure, the `future' and `future' call functions
include the spawning of the function's execution in another
thread. That's really several separate concerns:
future = function + promise + async + deliver
Exposing `promise' and `deliver' directly allows one to use other means
of spawning asynchronous work (or not) while still retaining the
block-on-a-latching-result capability.
--
Steven E. Harris
Jeff Rose
using Open Sound Control (OSC) messages, I use promises in combination
with futures to provide synchronous callbacks with an optional
timeout. For example, you might send a status request message to a
device and you expect an immediate response as long as the device is
live. If the device is off or it has crashed then you don't want to
hang forever waiting for a response. The promise/deliver pair acts as
the communication mechanism between a handler that gets called
asynchronously by another thread listening on a socket and the
original thread that is waiting for the response.
In the function below osc-handle registers a handler function for a
specific OSC path, which acts as a basic message dispatch. All the
handler does is deliver the promise and remove itself. The future is
used because they have a timeout feature when using the .get method.
(defn osc-recv
"Receive a single message on an osc path (node) with an optional
timeout."
[peer path & [timeout]]
(let [p (promise)]
(osc-handle peer path (fn [msg]
(deliver p msg)
(osc-remove-handler)))
(let [res (try
(if timeout
(.get (future @p) timeout TimeUnit/MILLISECONDS)
@p)
(catch TimeoutException t
nil))]
res)))
The library is available on github, in case anyone wants to talk OSC:
http://github.com/rosejn/osc-clj
-Jeff
Steven E. Harris
> The future is used because they have a timeout feature when using the
> .get method.
Should there be a corresponding timeout-based `deref' function¹
("deref-within"?) for promises? Having to close a function over your
"@p" form and spawn a job on a separate thread seems like way too much
work just to get a timeout-based wait on the promise's delivery
arriving.
Footnotes:
¹ http://richhickey.github.com/clojure/clojure.core-api.html#clojure.core/deref
--
Steven E. Harris
Richard Newman
> twice. Once to render it for the user and a second time to show the
> results. With promises you only need to traverse the data structure
> one time. In the traversal you create promises for the test results
> as well as created a flattened list of the tests to run.
Isn't this a case for delay/force, not promise/deliver?
David Nolen
Isn't this a case for delay/force, not promise/deliver?With out promises you would have to traverse this data structure twice. Once to render it for the user and a second time to show the results. With promises you only need to traverse the data structure one time. In the traversal you create promises for the test results as well as created a flattened list of the tests to run.
Richard Newman
> delay/force?
You said:
> With out promises you would have to traverse this data structure
> twice. Once to render it for the user and a second time to show the
> results. With promises you only need to traverse the data structure
> one time. In the traversal you create promises for the test results
> as well as created a flattened list of the tests to run.
You can do that with delay, too: traverse the data structure,
collecting delayed executions. Rather than making a bunch of promises,
delivering on them during your walk, and derefing the promises during
output, you create a bunch of delayed executions during your walk, and
force them during output.
Just as with the promise approach, you walk the list only once, and
execution happens after the walk is done.
Unlike promises, the execution of each test happens within the call
stack of the printing call (when the delay is forced). Unlike your
promise-based approach, you don't need to use Futures to work around
the blocking caused by dereferencing the promise; all of the work
using delay/force happens on the same thread.
E.g.,
(def test-input [[1 1 2] [1 -1 0]])
(defn do-tests []
(let [delays (map (fn [[a b e]]
(do
(println "First test expects" e)
(delay (= e (+ a b)))))
test-input)]
(doseq [d
;; doall effects printing.
(doall delays)]
(println "Test output:" (force d)))))
user=> (do-tests)
First test expects 2
First test expects 0
Test output: true
Test output: true
I'm not sure *why* you'd want to do what you're describing, but delays
will do it.
-R
David Nolen
(def test-input [[1 1 2] [1 -1 0]])
(defn do-tests []
(let [delays (map (fn [[a b e]]
(do
(println "First test expects" e)
(delay (= e (+ a b)))))
test-input)]
(doseq [d
;; doall effects printing.
(doall delays)]
(println "Test output:" (force d)))))
user=> (do-tests)
First test expects 2
First test expects 0
Test output: true
Test output: true
I'm not sure *why* you'd want to do what you're describing, but delays will do it.
-R
Richard Newman
> is something covered in my example.
The delays are still around, so you can do it whenever you want
without re-running the tests:
(defn do-tests []
(let [delays (map (fn [[a b e]]
(do
(println "First test expects" e)
(delay (= e (+ a b)))))
test-input)]
(doseq [d
;; doall effects printing.
(doall delays)]
(println "Test output:" (force d)))
;; Now walk the *already run* tests, adding up true results.
(reduce + (map (comp {true 1 false 0} force) delays))))
David Nolen
The delays are still around, so you can do it whenever you want without re-running the tests:How do you handle producing the sum of how many tests passed? This is something covered in my example.
Richard Newman
> to is something like a continuation as mentioned by Chouser. We want
> to block the thread of execution until some point in the future when
> the values become available.
I'm not sure that what you described describes what you want :)
Yes, if you want to block the current thread to wait for a value
computed by a background thread, you need promises and futures.
However, you were explicitly talking about a use "outside the context
of concurrency". If you don't want concurrency, only delayed
execution, then use delay instead.
> In your example you have to *know* that all the tests have run in
> order to accumulate results.
No you don't — force is synchronous. By the time that `doall` has
finished, all the tests have run and their results have been
accumulated inside the delays.
As I said, if you want to run tests in parallel, then you're not
talking "outside the context of concurrency". Futures are awesome, but
you can't argue that they're not a parallel construct.
> By using promise/deliver you can remove yourself from caring about
> that at all.
>
> With a recursive data structure this becomes annoying very quickly,
> you have to bookkeep where you are if you use delay/force.
No you don't — simply always call `force`, just as you always call
`deref`, and the delayed computation will be done if necessary. (In
fact, you can use "@", just as you can with refs and promises.)
Here's an example of a trivial recursive arithmetic evaluator that
prints the whole tree, returning a delay which captures the execution.
When forced, it prints the arithmetic results back up the tree.
Execution occurs once.
(defn form->delay [v]
(if (number? v)
(do
(println " Saw number" v)
(delay v))
(let [[op & args] v]
(println "Seen form with operator" op ", args" args)
(let [delays (map form->delay args)]
(delay
(let [res (eval `(~op ~@(map force delays)))]
(println "Result is" res)
res))))))
user=> (let [de (form->delay `(+ 5 (- 3 2)))]
(println "First run: " @de)
(println "No bookkeeping: " @de)
(println "Delay: " de)
@de)
Seen form with operator clojure.core/+ , args (5 (clojure.core/- 3 2))
Saw number 5
Seen form with operator clojure.core/- , args (3 2)
Saw number 3
Saw number 2
Result is 1
Result is 6
First run: 6
No bookkeeping: 6
Delay: #<Delay@58e22f2b: 6>
6
You'll see that:
* It runs on one thread
* There is no bookkeeping of what's been forced and what has not; you
can force multiple times without re-execution
* It can do arbitrary work during the tree walk (printing "Seen
form..."), and during evaluation (printing "Result is"), and the two
phases are separate.
> By creating the future computations to be performed during the
> traversal you later only need to deliver each individual test
> result. As tests come in they automatically trigger the next level
> of computation (the aggregate result). You don't need track
> relationships between tests at all because that was determined by
> the first traversal.
There are only two places where futures might apply in what you're
talking about:
* To run tests in parallel, which you explicitly disregarded
* To have the tests depend on each other's values (which doesn't seem
very smart to me).
Use futures and promises for parallelism or dataflow-style work. Use
delay for non-parallel, synchronous delayed execution.
David Nolen
Use futures and promises for parallelism or dataflow-style work. Use delay for non-parallel, synchronous delayed execution.
Richard Newman
You're right. Here's a recursive example of what I was trying to do with promise/deliver:And with delay/force:
I suppose the one thing I like about promise/deliver is that promises can be "watched" making it perhaps more extensible? Anyone can get just attempt to deref the value inside a future to "watch it". With delay/force it's harder to add new functionality without mucking directly with the code. Or is there a better way to handle this that I haven't thought of?
Thanks for the insights Richard.
Jeff Rose
Maybe (await-for ...) could be modified to work on both agents and
promises?
Note though, that getting a promise with a timeout doesn't require
explicitly closing over the promise. The anonymous function that is
created in the osc-recv function I pasted above is necessary because
it gets run asynchronously by a server socket thread. Getting with a
timeout versus without one is the difference of:
; blocking deref
@p
; deref with 100ms timeout
(.get (future @p) 100 TimeUnit/MILLISECONDS)
-Jeff
> ¹http://richhickey.github.com/clojure/clojure.core-api.html#clojure.co...
>
> --
> Steven E. Harris
Steven E. Harris
>Getting with a timeout versus without one is the difference of:
>
> ; blocking deref
> @p
>
> ; deref with 100ms timeout
> (.get (future @p) 100 TimeUnit/MILLISECONDS)
But the former just blocks on the promise being delivered, while the
latter creates an anonymous function, creates a proxy Future around it,
submits it for execution on a separate thread, and then blocks on the
function completing and (possibly) yielding a return value. That's much
more than a syntactic difference.
I tried to rewrite `promise' in terms of an AbstractQueuedSynchronizer
so that I could expose timed waits on it, but I got hung up with lack of
access to protected methods in the `proxy' macro.
--
Steven E. Harris