long running workers and timeout

Raffaele Bianco

unread,

Jun 15, 2015, 11:18:26 AM6/15/15

to cross...@googlegroups.com

Hi,

I need run a background process either periodically or triggered by a separate component. I figured that the easier way to go about is:

def __iter__(self):
yield self.subscribe(self.run, "a.topic.name")

while True:
yield self.publish("a.topic.name")
yield sleep(5)

Unfortunately I've just discovered that after around 15 seconds the 'wamp.session.on_leave' session is called and a separate component will change global state that causes my background process to fail.

I tried playing with the components->transport->timeout settings but it doesn't seem to work (also the default is 10 seconds and I get a on_leave after 15 secs).

Any clue?

Thanks,

Raffaele

Nahuel Defosse

unread,

Jun 15, 2015, 11:22:45 AM6/15/15

to cross...@googlegroups.com

Hi Raffaele,

Is that sleep async aware? Maybe you’re blocking the reactor.

Regards

--
You received this message because you are subscribed to the Google Groups "Crossbar" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crossbario+...@googlegroups.com.
To post to this group, send email to cross...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Raffaele Bianco

unread,

Jun 15, 2015, 11:27:21 AM6/15/15

to cross...@googlegroups.com

Hi,

yes, it's "from autobahn.twisted.util import sleep".

The problem occurs if the self.run method lasts more than 15 seconds. Everything works well if self.run lasts less.

Thanks :)

/Raffaele

Tobias Oberstein

unread,

Jun 15, 2015, 1:01:52 PM6/15/15

to cross...@googlegroups.com

Hi Raffaele,

this works for me:

https://gist.github.com/oberstet/70901f556271eeceec0e

```python
from datetime import datetime
from twisted.internet.defer import inlineCallbacks

from autobahn.wamp.types import PublishOptions
from autobahn.twisted.wamp import ApplicationSession
from autobahn.twisted.wamp import ApplicationRunner
from autobahn.twisted.util import sleep

class MyComponent(ApplicationSession):

@inlineCallbacks
def onJoin(self, details):
print("session ready")
yield self.subscribe(self.run, u"a.topic.name")

while True:
yield self.publish(u"a.topic.name",
options=PublishOptions(exclude_me=False))
yield sleep(5)

def run(self):
print("run at {}".format(datetime.now()))

runner = ApplicationRunner(url=u"ws://localhost:8080/ws", realm=u"realm1")
runner.run(MyComponent)
```

and prints this

```console
(python279_1)oberstet@thinkpad-t430s:~/scm/crossbar$ python test.py
session ready
run at 2015-06-15 18:59:39.518969
run at 2015-06-15 18:59:44.519833
run at 2015-06-15 18:59:49.525260
```

Does that work for you?

Cheers,
/Tobias

> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+...@googlegroups.com>.

> To post to this group, send email to cross...@googlegroups.com

> <mailto:cross...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com

> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com?utm_medium=email&utm_source=footer>.

Raffaele Bianco

unread,

Jun 17, 2015, 10:57:14 AM6/17/15

to cross...@googlegroups.com

Hi Tobias,

Thanks for the example, that works for me too. The issue arises when the run method takes more than 15 seconds to execute (you can simulate that by adding a "time.time".sleep(15) in it)

I anyway found the culprit. The issue is causes by the options I have in :

"ws": {
  "type": "websocket",
  "options": {
    "auto_ping_interval": 10000,
    "auto_ping_timeout": 5000,
    "auto_ping_size": 4
  }

I presume that being my component busy churning DB records for more than 15 secs (10+5) obviously cannot reply to the ping and crossbar kicks it out.

I'll change the intervals or remove the auto_ping completely.

Thanks for your prompt help,

Raffaele

> <mailto:crossbario+unsub...@googlegroups.com>.

Tobias Oberstein

unread,

Jun 17, 2015, 11:05:36 AM6/17/15

to cross...@googlegroups.com

Am 17.06.2015 um 16:57 schrieb Raffaele Bianco:
> Hi Tobias,
>
> Thanks for the example, that works for me too. The issue arises when the
> run method takes more than 15 seconds to execute (you can simulate that
> by adding a "time.time".sleep(15) in it)

Using time.sleep() will block the reactor. Don't do that.

You need to sleep without blocking.

E.g. for Twisted:

https://github.com/tavendo/AutobahnPython/blob/master/autobahn/twisted/util.py#L44

and to use

yield autobahn.twisted.util.sleep(15)

>
> I anyway found the culprit. The issue is causes by the options I have in :
>
> "ws":{
> "type":"websocket",
> "options":{
> "auto_ping_interval":10000,
> "auto_ping_timeout":5000,
> "auto_ping_size":4
> }
>

No, this is perfectly fine.

>
>
> I presume that being my component busy churning DB records for more than
> 15 secs (10+5) obviously cannot reply to the ping and crossbar kicks it out.

No, the reason is that you are blocking the reactor. Also note, that
WebSocket ping/pong is completely handled by Crossbar.io already - not
your component (it runs in the router worker).

You must not block the reactor: whether by doing time.sleep() or by
doing blocking DB access.

E.g. for PostgreSQL, we have a doc page here

http://crossbar.io/docs/Database-Programming-with-PostgreSQL/

>
> I'll change the intervals or remove the auto_ping completely.

No, this isn't the solution ..

>
> Thanks for your prompt help,
> Raffaele
>
> On Monday, June 15, 2015 at 7:01:52 PM UTC+2, Tobias Oberstein wrote:
>
> Hi Raffaele,
>
> this works for me:
>
> https://gist.github.com/oberstet/70901f556271eeceec0e
> <https://gist.github.com/oberstet/70901f556271eeceec0e>
>
> ```python
> from datetime import datetime
> from twisted.internet.defer import inlineCallbacks
>
> from autobahn.wamp.types import PublishOptions
> from autobahn.twisted.wamp import ApplicationSession
> from autobahn.twisted.wamp import ApplicationRunner
> from autobahn.twisted.util import sleep
>
>
> class MyComponent(ApplicationSession):
>
> @inlineCallbacks
> def onJoin(self, details):
> print("session ready")
> yield self.subscribe(self.run, u"a.topic.name

> <http://a.topic.name>")
>
> while True:
> yield self.publish(u"a.topic.name <http://a.topic.name>",

> options=PublishOptions(exclude_me=False))
> yield sleep(5)
>
> def run(self):
> print("run at {}".format(datetime.now()))
>
> runner = ApplicationRunner(url=u"ws://localhost:8080/ws",
> realm=u"realm1")
> runner.run(MyComponent)
> ```
>
> and prints this
>
>
> ```console
> (python279_1)oberstet@thinkpad-t430s:~/scm/crossbar$ python test.py
> session ready
> run at 2015-06-15 18:59:39.518969
> run at 2015-06-15 18:59:44.519833
> run at 2015-06-15 18:59:49.525260
> ```
>
> Does that work for you?
>
> Cheers,
> /Tobias
>
> Am 15.06.2015 um 17:18 schrieb Raffaele Bianco:
> > Hi,
> >
> > I need run a background process either periodically or triggered
> by a
> > separate component. I figured that the easier way to go about is:
> >
> > def __iter__(self):

> > yield self.subscribe(self.run, "a.topic.name <http://a.topic.name>")
> >
> > while True:
> > yield self.publish("a.topic.name <http://a.topic.name>")

> > yield sleep(5)
> >
> >
> > Unfortunately I've just discovered that after around 15 seconds the
> > 'wamp.session.on_leave' session is called and a separate
> component will
> > change global state that causes my background process to fail.
> >
> > I tried playing with the components->transport->timeout settings
> but it
> > doesn't seem to work (also the default is 10 seconds and I get a
> > on_leave after 15 secs).
> >
> > Any clue?
> >
> > Thanks,
> > Raffaele
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Crossbar" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send

> > an email to crossbario+...@googlegroups.com <javascript:>
> > <mailto:crossbario+...@googlegroups.com <javascript:>>.

> > To post to this group, send email to cross...@googlegroups.com

> <javascript:>
> > <mailto:cross...@googlegroups.com <javascript:>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+...@googlegroups.com>.

> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/crossbario/c915873f-330c-43ea-9832-48b1e0cb123b%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/c915873f-330c-43ea-9832-48b1e0cb123b%40googlegroups.com?utm_medium=email&utm_source=footer>.

Raffaele Bianco

unread,

Jun 17, 2015, 11:12:47 AM6/17/15

to cross...@googlegroups.com

Hi Tobias,

Yes, you're perfectly right but I'm using SQLAlchemy and I'm stuck with sync. Alternatively I might offload the work using deferToThread, would that work?

Thanks,

Raffaele

> > <mailto:crossbario+unsub...@googlegroups.com <javascript:>>.

> > To post to this group, send email to cross...@googlegroups.com
> <javascript:>
> > <mailto:cross...@googlegroups.com <javascript:>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/crossbario/7907cfdd-936a-44da-81f1-a6d02c67ee8c%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+unsub...@googlegroups.com>.

Tobias Oberstein

unread,

Jun 17, 2015, 11:38:02 AM6/17/15

to cross...@googlegroups.com

Hi Raffaele,

Am 17.06.2015 um 17:12 schrieb Raffaele Bianco:
> Hi Tobias,
>

> Yes, you're perfectly right but I'm using SQLAlchemy and I'm stuck with
> sync. Alternatively I might offload the work using deferToThread, would
> that work?

That should work in principle, as then only the background worker thread
is blocked.

Also, there is an asynch ORM here http://findingscience.com/twistar/

I haven't use any of above approaches as I am avoiding ORMs, and use
database stored procedures to encapsulate the DB.

Well, and lately I've been using the direct database integration of
Crossbar.io;)

Cheers,
/Tobias

> <http://a.topic.name> <http://a.topic.name>",

> <http://a.topic.name> <http://a.topic.name>")

> > > <mailto:crossbario+...@googlegroups.com
> <javascript:> <javascript:>>.

> > an email to crossbario+...@googlegroups.com <javascript:>
> > <mailto:crossbario+...@googlegroups.com <javascript:>>.

> > To post to this group, send email to cross...@googlegroups.com
> <javascript:>
> > <mailto:cross...@googlegroups.com <javascript:>>.
> > To view this discussion on the web visit
> >

> https://groups.google.com/d/msgid/crossbario/c915873f-330c-43ea-9832-48b1e0cb123b%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/c915873f-330c-43ea-9832-48b1e0cb123b%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/crossbario/c915873f-330c-43ea-9832-48b1e0cb123b%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/crossbario/c915873f-330c-43ea-9832-48b1e0cb123b%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>

> > For more options, visit https://groups.google.com/d/optout
> <https://groups.google.com/d/optout>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Crossbar" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to crossbario+...@googlegroups.com

> <mailto:crossbario+...@googlegroups.com>.

> To post to this group, send email to cross...@googlegroups.com
> <mailto:cross...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/crossbario/79939983-9526-4f81-af0c-f1907311e87f%40googlegroups.com
> <https://groups.google.com/d/msgid/crossbario/79939983-9526-4f81-af0c-f1907311e87f%40googlegroups.com?utm_medium=email&utm_source=footer>.

Reply all

Reply to author

Forward