The Request Pattern

[Marcus Ottosson]

Hey guys,

I'm looking for feedback on a programming pattern I've been tinkering with lately; its a simplification of signalling an event and getting a result back via another signal, such as when a GUI requests an event to be performed, but the event it performed elsewhere - most likely a separate object - and gets an event back from the separate object upon completion.

The Request Pattern

Let me know what you think.

Best,

Marcus

[Justin Israel]

Just wondering, are you developing a completely new framework based on new concepts, or are you abstracting a layer over existing frameworks with existing concepts?

In your diagrams they seem to imply that it is an asynchronous request-reply, but in your code examples they imply a synchronous call. Have you found faults with existing frameworks such as Qt (QCoreApplication) or Twisted? These frameworks achieve the same results where you can dispatch an async call, and have a result returned to you similar to a callback. 

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/022c3632-4637-4cc5-bd37-8fe1a4a91127%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marcus Ottosson]

Just wondering, are you developing a completely new framework based on new concepts, or are you abstracting a layer over existing frameworks with existing concepts?

I'd have to go with both, and neither. :) See below.

In your diagrams they seem to imply that it is an asynchronous request-reply, but in your code examples they imply a synchronous call.

This pattern would be highly synchronous. I think any asynchronisation would have to reside external to the Request object, unless you can think of a way?

Have you found faults with existing frameworks such as Qt (QCoreApplication) or Twisted? These frameworks achieve the same results where you can dispatch an async call, and have a result returned to you similar to a callback. 

Quite the contrary. :) I've found many uses for concepts within Qt, some of which I'd love to use outside of Qt as well, hence the implementation into a more widely accessible object. I'm more interested in the actual concepts than I am in particular implementations like the ones listed above.

I have no doubts the Request Pattern already exists. The reason for me defining it is to find out more about it as it has become quite useful in my code. Could you point me to where Twisted does this?

Thanks Justin.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA2x2DcD%2BH20eN95-WqbzErLFKAsrBueYvyy1F1tzsx71A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Matt Chambers]

Hey there, this is already a thing. No need to reinvent the wheel.

http://en.wikipedia.org/wiki/Futures_and_promises

[Marcus Ottosson]

Hi Matt, thanks for the link.

I think the future and promises pattern is mostly for concurrent operations, whereas the pattern I'm attempting to describe (although it would seem I'm not doing a very good job in doing so :) is a pattern for de-coupling components; it is actually synchronous, much like signals and slots.

Since both you and Justin find the Request Pattern about asynchronous and concurrent operations, I'd like to refine the RFC to highlight that it is not. Could you point out to me which part of the document made you come to this conclusion?

Thanks a lot

On 17 April 2014 08:57, Matt Chambers <mvcha...@me.com> wrote:

Hey there, this is already a thing.  No need to reinvent the wheel.

http://en.wikipedia.org/wiki/Futures_and_promises

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/3f753ec7-1cc8-4d59-8671-8bf7ebdb3ccd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Justin Israel]

I'm just finding the diagrams don't seem to match the code examples. The examples do suggest a 100% synchronous call. You make the function call and block until you have a result. In that case, what you have is basically an abstraction over one function calling and returning the value of another function in the form of a closure. 

def other():

    return "foo"

def this():

    return other()

print this()

# "foo"

That is a synchronous call, and you block until you get your value. 

But your diagrams refer to two phases, which implies the call is asynchronous:

def callmeback(val):

    print val

def other():

    return "foo"

def this(callback):

    enqueue(other, callback)

this(callmeback)

# do some other stuff.

# eventually this happens:

# "foo"

Or in the terms of a future:

future = this()

# do some stuff.

print future.get()

# "foo"

So is it an async request-reply pattern where you send off a request and at some point later you expect to retrieve a reply? Or is it a synchronous request-reply where you send off a request and block until you get your reply? Because my impression is that either the code examples don't match the diagrams, or the code examples are correct and its an abstraction around closures where one function calls another function whose implementation can be switched out (my function calls something that contains switching logic to figure out what to really call and return). 

Regarding the twisted thing, I don't think it is what you are after since it does focus on solving async problems using an event loop and callbacks as opposed to threading. 

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOC0O9cVh29_ic7O95NLC21PqRVvyDY265%2BhuHBZbxZGbw%40mail.gmail.com.

[Marcus Ottosson]

Here we go. That is an excellent observation, Justin, I see what you mean. I'll try and adjust the diagrams to fit the code examples better.

I'm thinking of ways to make it clear from the get-go. I'm thinking of using terms such as "inter-process RPC", since:

1. Calls act internal to the object

2. Calls reside elsewhere

3. Calls are synchronous

Just like RPC. :)

However a Request is not a RPC, because they are subscribed to by external objects (observers), like Qt Signals.

In fact, the calls act identical to calling upon the local functionality of an object. The Request object might be referred to as a "proxy" around an external object; again, similar to RPCs and services.

Ultimately, from the objects point of view, calling a method it owns and calling a method through a Request is identical. The only difference is the external methods doesn't have to exist to begin with, but can come and go, just like with Qt Signals and Slots.

As for the diagrams, are you referring to first one? The "graphics" to "logic" using 2 signals? That is indeed an asynchronous design, and I should clarify that this is a synchronous use of an asynchronous method. :) Signals may be use for async calls, (i.e. one signal could emit, but not necessarily expect one in return, at least not immediately) The use-case I'm attempting to illustrate is for situations where you want a response back immediately; when you want something happening remotely to respond to the GUI immediately.

My particular use-case is for GUI and logic sitting right next to each other, not leaving any room for delay. Can you see any other use?

Thanks again!

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA11R-UEnWf5DUCHz%2B_KuYJn1Sx66K7CPD3uZFbsJR7VqQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[engen...@gmail.com]

You will have to excuse my brevity and any typos in advance, as I am writing this on my cell phone.

If I understand correctly, you want to define how a particular object (the subject, using your vocabulary) can notify another (observer) of an event, have the observer take some action, and then return some thing as a result.

I understand that decoupling the subject from the observer could appealing, so that you can call any function provided by observer when an event occurs. (And possibly provide logic inside the request to route the event to one of many possible functions inside the observer)

However, I think this particular implementation would cause more issues than it solves.

For instance, in the case that a third object wires a subjects event to the wrong method the observer, and some exception thrown the observers method. In this case, my stack trace will show that the subject called the observer, and that an exception was shown.

A developer would then spend time tracking down where these calls happened, only to not find anything wrong. They would have to know that object 3 wired the event, and only upon looking at the code in object 3 would they find the bug.

This makes the code a maintenance nightmare. Especially if the maintainer didn't write the code.

And I'm all for making maintainers lives easier.

Secondly, this means that an object does not fully control what it's response is to an event.

I am of the mind that an object should explicitly define its behaviour to events.

Have you considered using event handlers?

Observers must satisfy an interface that the subject can call directly, making maintenance easier.

Also, the observer then is in full control over what happens in response to the event.

Finally, a subject is then able to provide subscribe/unsubscribe functionality to allow dynamic routing of events to different observers.

Turns out that wasn't so brief after all.

[Marcus Ottosson]

Hi Niall,

That is an interesting point, thanks for that. I'll keep this in mind as I stretch the leg of this pattern over the coming few days. I can already see your point causing harm, the same thing happens currently with regular signals. Since a signal is the one that ultimately calls upon subscribed methods and functions, it is in the signal class that errors usually gather. I've been looking for a way to solve that, but haven't succeeded yet.

By event handler, you mean have a method that filters all incoming events? Similar to the Qt Event System? I may be wrong, but it seems to me that it would simply move the problem to the event handler, no?

For reference, let me post a Python implementation here, so we're all on the same page.

The one I'm using, with weak references

https://gist.github.com/mottosso/11008554

And a stripped one without fuss

https://gist.github.com/mottosso/11008652

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/585f7e11-45ba-401a-a533-6811a3c0069d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Matt Chambers]

Signals and slots use the event loop, so I would't consider them synchronous. You can easily achieve async behavior.

Like in this case, do a network request and then notify me when its done,
QNetworkReply *reply = manager->get(QUrl("http://foo.com"));
connect(reply, SIGNAL(finished()), this, SLOT(some_slot()) );

Same with future, promise, you can achieve both async and sync behavior, its really up to caller to decide. I often use future/promise just to limit a particular operation on a server to only a couple threads. The end result is people call a completely synchronous function.

It seems like this pattern is just a poor mans version of something that already exists. I guess my confusion comes from all the time/energy your putting into documenting it.

[Matt Chambers]

Here is another source of a sync messaging API that is core to iOS, invented in the 1970s.

http://www.apeth.com/iOSBook/ch03.html

[Marcus Ottosson]

Matt, I'm sorry, but this isn't about synchronous versus asynchronous calls, I'm not sure why you keep persisting. :( I take it you are neck deep in asynchronous programming at this point. If I may, please keep an open mind and be careful not to get tunnel-vision.

On 17 April 2014 21:43, Matt Chambers <mvcha...@me.com> wrote:

Here is another source of a sync messaging API that is core to iOS, invented in the 1970s.

http://www.apeth.com/iOSBook/ch03.html

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/afaf0ed7-58e1-4379-9b2f-d3828fb7f75e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Matt Chambers]

Sorry, I'm just confused as to why your taking some concept people have been doing for 40 years and making an "RFC" for it. I applaud your enthusiasm though. I wish I put as much effort into documenting.

[Marcus Ottosson]

I had to turn to your LinkedIn for this one. Clearly you are a very educated man, and thus I would expect someone in your position to know when he is trolling. You strike me as bitter, Matt, and I'm having trouble absorbing your opinion on this matter.

On 18 April 2014 08:50, Matt Chambers <mvcha...@me.com> wrote:

Sorry, I'm just confused as to why your taking some concept people have been doing for 40 years and making an "RFC" for it. I applaud your enthusiasm though.  I wish I put as much effort into documenting.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/3a3fd118-0723-4d87-b9a8-2d3095c8f193%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com

[Justin Israel]

In an attempt to reinterpret Matt's comments to reflect my own thoughts, the way it appears to me is that the RFC is a very abstract description of an implementation that wraps existing concepts. It positions itself as if it is proposing something new, but I am not clear on the existing problems that are not properly solved, and what your idea is really aiming to change. That would be in terms of real concrete problems that need to be solved, instead of saying a button might need to trigger something that may or may not exist and it may or may not require a response either now or at some point in the future. Otherwise I am not sure what the RFC is aiming to do. It comes across more as a brainstorm for how to write your own implementation of some Python program, as opposed to saying there is a new way to do something. What specific problem do you want to solve? What have you tried that has failed to meet your needs?
That is why I feel you are receiving criticism on the topic. I don't see it as trolling. I see it as people challenging that it is a solid topic. Not all criticism has to include contributions that will further your design. Sometimes some people just don't get the goal and the intent. And that's ok. While you say sync/async is not the focus, maybe then it's not as clear as you had hoped, if commentors are focusing on the topics you didn't intend to discuss.

My 2 cents.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmOCq8SuanVGOAuuVhVntTn8wXL9vpw3kQSJFxQOxGLru_Q%40mail.gmail.com.

[Marcus Ottosson]

I don't see it as trolling. I see it as people challenging that it is a solid topic.

People? I'm referring to Matt, no one else is trolling.

I'm truly sorry if I have offended anyone. All I am asking is for feedback on a pattern. I care little about whether it is new or when it was invented. I'm using this right now and am reaching out to you in an attempt to understand it better, but it seems we've somehow gotten into the merits of whether or not one should define a problem in a document before doing so. This is no good.

I'd rather not land myself in a similar situation again, what could I have done to better illustrate my intention to you?

Now, if we could move on from this, I do have some work to do in refining the document in regards to illustrations not matching the code examples, but hopefully the definition and goal should be rather clear in this thread; i.e. it is about reducing complexity in a specific type of situation, like the one described in the document. Do also have a look at the code examples posted above for further definition.

So, what about this pattern would be amazingly helpful, less helpful, and down-right unhelpful? :)

Thanks.

Best,

Marcus

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA2yAtNG5UM%2Bnvc22Rs29TmUwzgxuij1Um21_eU5rx%2BnfQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com