Add closure support to blocking operations to allow them to be non-blocking operations.

[Luis Majano]

If we can compare to other languages in this age, we must provide some type of non-blocking IO operations natively into the language.  I propose adding closure callback support to the following functions/operations so they can be processed natively in the background (non-blocking) and then call back when done.

FileRead

FileWrite

cfquery

http

cffeed/feed

cfmail/mail

storedproc

ftp

etc

Basically any operation that can block a network or local resource.  Imagine doing this:

fileRead( file, function(contents){

  process file here in the background once it is read.

});

fileWrite( file, contents, function(){

 // process file here once it is written.

});

new http( url="xxx", method="post", results=function(content){ process here the results })

[Andrew Scott]

+1

Regards,

Andrew Scott

WebSite: http://www.andyscott.id.au/

Google+:  http://plus.google.com/113032480415921517411

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/railo/390304ad-7817-404a-8cde-7c3f184b9f22%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Adam Cameron]

Interesting suggestion, but not the way I'd approach it. I've written up my thoughts here: http://cfmlblog.adamcameron.me/2013/10/a-possible-variation-on-thread-idea.html

Cheers for the inspiration for today's blog article, Luis.

-- 

Adam

[Michael Offner]

i like the idea, question is could we do this in a more general approach.

You can do today the same as follows.

<cfthread name="susi">

<cfset fileWrite( file, contents)>

// process file here once it is written.

</cfthread>

(perhaps a little offtopic) 

my problem with cfthread is, that i always have to check how to pass data to it, when i use it, it is not clear how to use exactly out of my mind.

i would like to have a easier approach to use threading in Railo. Before we and ACF has added cfthread, my idea about threading was as follows.

<cfscript>

function test(arg1) async=true {

   ....

}

test(123);

</cfscript>

based on this you could turn around your example as

function () async="true"{

   f=fileRead( file);

   // process file here in the background once it is read.

};

Micha

2013/10/11 Luis Majano <lma...@gmail.com>

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/railo/390304ad-7817-404a-8cde-7c3f184b9f22%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

/micha

Michael Offner CTO Railo Technologies GmbH

[Andrew Scott]

Actually I like Luis's idea much better Adam, as it opens the possibility of opening closures to the rest of the language. However, if you don't wish to do a closure like Luis suggested, then the next option would be to add a callback function like you suggested Adam.

That is why closures are fun..

Regards,

Andrew Scott

WebSite: http://www.andyscott.id.au/

Google+:  http://plus.google.com/113032480415921517411

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/34504009-012e-4ee8-bf05-b9f9962235a1%40googlegroups.com.

[Andrew Scott]

hehe ColdBox adds that feature already with AOP...

Regards,

Andrew Scott

WebSite: http://www.andyscott.id.au/

Google+:  http://plus.google.com/113032480415921517411

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/CAJVB4jdUSVxNXe-P0qEcaa3218xQ7LV1Z7tN7BTbWL7KCdfVrw%40mail.gmail.com.

[Adam Cameron]

On Friday, 11 October 2013 09:10:11 UTC+1, Andrew Scott wrote:

Actually I like Luis's idea much better Adam, as it opens the possibility of opening closures to the rest of the language. However, if you don't wish to do a closure like Luis suggested, then the next option would be to add a callback function like you suggested Adam.

That is why closures are fun..

You are making an invalid distinction between a callback, which is a function that is passed to another function to do some work; and closure, which is an approach to variable binding which function-expressions can be implemented to use. They are two different conceptual things, and can't be compared sensibly. It's like comparing oranges (the fruit) to orange (the colour). It makes no sense.

A callback can be any sort of function: it might be a function expression which implements closure; or it might be from a function declaration which doesn't implement closure.

-- 

Adam

 

[Andrew Scott]

Thanks Adam I am very well aware of what the differences are, it was a slip and I thought I had deleted the word callback when I rewrote that section of text.

Regards,

Andrew Scott

WebSite: http://www.andyscott.id.au/

Google+:  http://plus.google.com/113032480415921517411

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/cf01899a-3f40-44c0-99e7-f911c9c4c5fd%40googlegroups.com.

[Adam Cameron]

On Friday, 11 October 2013 09:22:59 UTC+1, Andrew Scott wrote:

Thanks Adam I am very well aware of what the differences are, it was a slip and I thought I had deleted the word callback when I rewrote that section of text.

But the entire basis for your comment was predicated there being a difference between a closure and a callback, Andrew. How big was this typo of yours? Everything between pressing "reply to this" and "send"? ;-)

-- 

Adam

 

[Andrew Scott]

Not at all Adam, what I am saying is a preference to code like Luis suggested and as a fallback to preference would be your suggestion.

Regards,

Andrew Scott

WebSite: http://www.andyscott.id.au/

Google+:  http://plus.google.com/113032480415921517411

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/447c21c5-c8f5-4d0e-bdf5-9600afdafc69%40googlegroups.com.

[Andrew Penhorwood]

+1 on the idea of having some type of callback (Luis Majano's idea) or function processing (see Adam Cameron's blog post).

I have used cfthread in the past to handle some of these types of processes.  It works but like Micha said it seems complicated to get data in and out of the cfthread context.

My use case, my framework has stats collection built into the code base.  Since this type of information is of no value to the end user (person request the page) I don't want to delay the page being sent to the browser

processing these stats.  So I created a stats collection component that uses cfthread to process them async.

Andrew Penhorwood

[Luis Majano]

I think adding the concept of closures on the callback makes it really easy to write, look and adapt.

However, why stop there.  Using Adam's approach you can use the concept of Java Future's which is part of the JCF.  Working lately with Couchbase and most of its operations being asynchronous you get to appreciate it.

Some of their operations give you a Future object you can use to check the status of the thread operation,etc. Much how we using cfthread get a struct back with its data.  I think the problem with the cfthread approach is that you have to either poll or wait for the operation to finish to do something.  I like Micha's idea as well, but if a function is tagged with an "async=true" then maybe it should return to you a Future object or struct by default.  So you can monitor its progress.

I just think that sometimes you don't want to poll for results, you want them to be done completely in the background once a blocking operation is finalized.

[Michael Offner]

like you say the cfthread "scope" is very similar to the the Future object, you get a status (running,completed,teminated ...) and also the internal scope ...

i think the problem of cfthread is, it is to complicated.

i think for a "run and forget" the async function is perfect.

the supported function 

each([1,..,100000], closure:function(){}, parallel:true, maxThreads:20);

 is perfect to handle the problem of the right balance of concurrent threads.

so you are able to restrict the concurrent threads to a certain number.

railo takes care that max 20 concurrent threads are running and waits for all threads to finish at the end of the function call.

(cfloop-parallel will follow soon)

but, what about the future/cfthread object, of course a async/parallel function could return this as well

function test() parallel=true {

     return "done";

}

thread=test();

dump(thread); // gives you all info about the thread an the doc for the possible actions

dump(thread.status);// running,waiting ...

dump(thread.result); // in this case "done" when completed

dump(thread.output); // data writen with echo (out writer)

thread.terminate(); // terminates the thread

thread.join(); // waits for completion

micha

2013/10/12 Luis Majano <lma...@gmail.com>

--
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/82c73a1a-643b-44bf-9c17-b2cdd522d381%40googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

[AJ Mercer]

this is a very interesting and thought provoking thread here.

I like the concept of a future object - in AngularJS it is called a promise and is very powerful for asynchronous processes.

As I understand it, Railo would require a mechanism to raise events for the future objects that be listened to by code.

There has been mention of adding more events to Railo - eg, email sent.

I am thinking futures call tie into this architecture.

And to tie into another (mail) thread - web.cfc / server.cfc

One contexts could raise and event, and another could handle it.

The thought of event based CFML is making me giddy with possibilities.

--

AJ Mercer
<webonix:net strength="Industrial" /> | <webonix:org community="Open" />
http://twitter.com/webonix

Railo Community Manager

[dajester2012]

Maybe this is overkill for the discussion at hand, but I still wanted to point out this really nice project...

We have used the cfconcurrent library in our shop for threading.  It is basically CF wrappers around the java concurrency library, and is pretty awesome.  You can write a CFC that implements the java runnable or callable interface (via createDynamicProxy) and pass it off to an executor in return for a Future.  Full disclosure: we did run into memory leak issues when attempting to use shared objects for job queues.  However, this was several versions of Railo back and it was a fairly (overly?) complex system.  I think we're going to take another stab at it in the future using caches instead of shared objects.

[Adam Cameron]

On Thursday, 10 October 2013 23:54:49 UTC+1, Luis Majano wrote:

Basically any operation that can block a network or local resource.  Imagine doing this:

fileRead( file, function(contents){

  process file here in the background once it is read.

});

This suggestion continued to bug me, and it occurred to me at CFCamp why it bugs me

The issue here is that there are some possibly long-running "blocking" operations which could be expedited via some background running. Fair enough, but I thought Luis's suggestion here treated the symptom, not the issue. Hence my suggestion of having a more all-encompassing solution.

But on reflection the suggestion above is not only sub-optimal, but not even a solution to the problem being observed. In the example above, the actual file I/O is still being being done in the "main" (blocking) process, whereas the subsequent processing is being passed off to another non-blocking process of some description. But it's the I/O that is likely to be the slow bit! Either way, it's only dealing with half the perceived issue.

So - as it stands - I'm reasonably against this approach to dealing with the identified problem, on the basis that it doesn't really achieve very well what it sets out to do.

Sorry, Luis ;-)

-- 

Adam

[Bruce Kirkpatrick]

Luis: Did you try to implement non-blocking I/O with existing CFML yet?   

You could maintain a queue that contains the callback obj info and then in onRequestEnd, you call a function that finds a completed queue unit and executes the callback method for it.   This would allow everything to happen out of order at the end of the request similar to how node.js does it.  I already made an asynchronous query system like this before, but I didn't think to implement a callback function as well, it would be pretty easy to do that as well.

Your framework could also have a way of setting up a group of these queued work units and the group would have its own callback function in addition to the individual work units having an optional callback.  

At the end of all these callbacks, you'd execute the rest of onRequestEnd to finish off the request.

In some requests, it may be possible to pre-process some of the returned data before you need to have all of them ready at the same time, so you would improve performance in these areas as you begin to work in this style.   Since I love performance, I will probably try to implement what I'm saying for my app.

You also need to build a system that limits the number of concurrent access to a shared resource in a specific way.  Like you don't want to have 100 requests connect via CFHTTP to the same domain at the same time probably.  You could make it part of the framework to limit threads globally.  A framework is probably not going to want to ask its user to directly call CFHTTP anyway even if it did have the closure features, because you're going to want to augment its features somehow.

I wrote out how to do some of these things in detail with CFML in the last few days.

File I/O api was already improved in the past to waste less memory and block less if you want to do that, but we still don't have a detailed API for working with database and http in a streaming way. I don't like how much memory / cpu time is wasted from that either.    cfquery lazy="true" almost fixed the database access, but I think it still stores a complete resultset after the loop is complete.  It would be more efficient if it could discard the memory for the rows you already looped somehow.  CFML database access uses more memory then many other languages just so you can access the data out of order?  We almost never need the database data to be looped backwards or something...

[Bruce Kirkpatrick]

I realize you asked for CFML to have new features, but if anyone is interested, I was inspired to build a solution for asynchronous functions with callbacks based on the CFML we have today without having to write cfthread manually.

https://github.com/jetendo/async-dot-cfc

async.cfc can do asynchronous I/O or run any slow function and then it will call the callback function on the object that you assigned to the work unit.  

If you want to limit the number of threads globally. For example, you could limit all requests so only 10 threads can run against the resource even if 100 concurrent Railo requests are being made.  This is the key difference that makes async.cfc act like more node.js threadpool for different resources.   My approach uses a thread name prefix so you can create as many different kinds of threadpools as you wish without the names conflicting with each other.  I.e. "memcache|read" or "database1write" or "http|google.com".  Theoretically, an entire app built like this would be able to get higher # of requests per second because each resource is being managed more carefully.

You can also define a group of dis-similar functions to run in separate threads that each have their own callbacks, but also a callback for the group, so you can create a set of dependencies before allowing further execution of the app.  Like you could run a query that gets the total count of records and a query that gets the first 10 records at the same time with this feature like when you do pagination of a list of records.  When the queries are both done, the group callback function would be called.

I choose not to use closures, but it could be adapted for closures.

You can queue other work immediately after the first work is queued.  CFML execution doesn't get blocked, which makes this more similar to node.js style programming then the "each" function which blocks until all threads are done and joined.    

I don't think each()'s maxThreads setting is server-wide, it is just specific to that specific call, right? If so, each() is potentially more dangerous since you'd have a lot more threads being generated by just a few concurrent users then you may actually want.

cfthread has around 1 millisecond of overhead on my machine, so you probably want to make sure you're doing more then a few milliseconds of work before you create a thread for something.

[Luis Majano]

Adam

I think you are missing the point that when a callback is sent then the engine should do the io operation in a separate thread thus continuing operation from the file read. Thus anything after would execute immediately and the engine spawns a thread to read the file and then call your callback once done. Therefore it won't block.

Bruce. That's a great project. Maybe a ColdBox module would be awesome for it?

However the concept of tasks is already available via the jcf ( java concurrent framework ) in all jdks. Marc esher has a cfconcurrent project already.

However, why not expose this java functionality directly as well. Where you can create task CFCs and send them to cf for processing like the executor and scheduler service in java already.

Now I know we can leverage cfthread and dynamic proxies to accomplish some of these things. However concurrency is such a complex topic that I think it is an area where CFML can come and save the day. Maybe even provide special concurrent locking shared scopes, scheduling, monitoring, thread and task pools, timeouts, etc.

I would love to see something like

Function operation() timeout=20{}

How cool would it be if the engine allowed us to provide timeouts for function calls.

[Ryan Hinton]

Very nice, Bruce! Thanks for sharing and giving us great analysis.

---

From: Bruce Kirkpatrick <br...@farbeyondcode.com>

Sender: ra...@googlegroups.com

Date: Sat, 19 Oct 2013 16:58:11 -0700 (PDT)

To: <ra...@googlegroups.com>

ReplyTo: ra...@googlegroups.com

Subject: [railo] Re: Add closure support to blocking operations to allow them to be non-blocking operations.

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/3bb6ccd9-481e-4b4a-969e-6fa98d3652b1%40googlegroups.com.

[Baz]

It would be nice having non-blocking closures and/or callbacks as syntactic sugar to threads, but I hope no-one expects that this would make railo an alternative to node. The cool/important part about node is that it achieves concurrency without parallelism - it's single threaded. Operations that are "waiting" are consuming near zero resources - not spawning ginormous threads, with context switching, and state management, etc. That's how node is able to achieve orders of magnitude better concurrency than any/most threaded systems. 

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/414075544-1382246889-cardhu_decombobulator_blackberry.rim.net-1288356286-%40b12.c20.bise6.blackberry.

[Mark Drew]

So railo kinda already has the idea of tasks. You can create a thread type=task and set the retries etc. 

Regards

Mark Drew

Sent from a mobile device

--
Did you find this reply useful?  Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/6ee6cb9d-18cb-45d7-9d55-475d302a4a01%40googlegroups.com.

[Adam Cameron]

On Sunday, 20 October 2013 05:19:30 UTC+1, Luis Majano wrote:

Adam

I think you are missing the point that when a callback is sent then the engine should do the io operation in a separate thread thus continuing operation from the file read. Thus anything after would execute immediately and the engine spawns a thread to read the file and then call your callback once done. Therefore it won't block.

OK, so you're saying that if the function receives a callback argument, then it should infer that different threading behaviour of the function being called should take place?

That's making me grimace.

Code should be predictable. Passing a callback should imply a callback will be used for [something]. It does not (and accordingly should not) suggest "the callback will be called and the main operation will now run in a separate thread". If you're adding an "and" into the mix like that, you're doing too much. 

I'm not intrinsically against the proposal (although you're not yet convincing me this function-by-function way is a good approach), but this suggested implementation is a bit incoherent, I think.

-- 

Adam

[Bruce Kirkpatrick]

I didn't understand what thread type="task" was until now.  That's an awesome Railo exclusive feature.   I actually wanted to have asynchronous queries that would finish without making the request wait for them to complete so you could improve concurrency for the public URLs.  When I found I couldn't do that with cfthread action="run", I removed that feature and made it rely on session variables to be faster.  

However, if I use type="task", I can actually finish the request without waiting for the query to finish, and the user can get an instant response. This would be good for reading and writing out a large file or a logging routine like a database driven hit counter.  You don't always need to display a result.

[Gert Franz]

Especially the thing is that CFTHREAD type=”task” allows you to check your task manager inside the Railo Administrator in order to see whether your tasks have completed successfully (nothing to see) or whether they have failed. There is an additional attribute called retryinterval which takes either an array of structs or a struct containing the two keys interval (timespan) and tries (numeric) so that you can define a custom retry interval schedule if the task fails.

This is a perfect solution for stuff you need to make sure works…

 

Greetings from Switzerland

Gert Franz

 

The Railo Company Ltd.      Professional Open Source

skype: gert.franz                   ge...@getrailo.com

+41 76 5680 231                     www.getrailo.com

 

The Railo Company – Poland Street 15 – London W1F 8QE – United Kingdom

Company No. 8055066 – VAT Registration No. 150 4060 58

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/33e26e41-fce7-44c3-ac77-1ecda0e96ea2%40googlegroups.com.

[Adam Cameron]

On Thursday, 10 October 2013 23:54:49 UTC+1, Luis Majano wrote:

If we can compare to other languages in this age, we must provide some type of non-blocking IO operations natively into the language.  I propose adding closure callback support to the following functions/operations so they can be processed natively in the background (non-blocking) and then call back when done.

FileRead

FileWrite

cfquery

http

cffeed/feed

cfmail/mail

storedproc

ftp

Luis, did you ever raise a ticket for this? I can't find it, if so (I did not look very hard).

Brad's also just raised this: https://issues.jboss.org/browse/RAILO-3131

-- 

Adam 

[Michael Offner]

We will also add the tag cftask to Railo 5 (light way version of cfthread-task).

Then we plan to do <cfloop ... parallel=true maxthreads=10> that work the same way as the closure functions (run the body parallel with limited threads and in the end Railo joins back all threads to the main thread.)

Micha

--

Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/railo/4158c114-6746-4201-a37b-4cf5788284b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Adam Cameron]

On Tuesday, 22 July 2014 07:25:44 UTC+1, Michael Offner wrote:

We will also add the tag cftask to Railo 5 (light way version of cfthread-task).

Please don't add it as a *tag* :-/

-- 

Adam