autobahn.wamp.exception.SerializationError: Unable to serialize WAMP application payload ()

[foxmask]

When I add a component in the .crossbar/config.json and start crossbar, I face that behavior once the data have been retrieve by the publisher.

autobahn.wamp.exception.SerializationError: Unable to serialize WAMP application payload (can't serialize time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0, tm_yday=61, tm_isdst=0))

When I dont add the component to .crossbar/config and start crossbar, then start the publisher all went well...

Do you know a trick that fixes that behavior or do you need the piece of code that raise that error ?

[foxmask]

once notice :
I install crossbar that way :

pip install crossbar

then

pip install autobahn[twisted,accelerate,compress,serialization]

To enable serialization

[Tobias Oberstein]

Am 02.03.2015 um 14:39 schrieb foxmask:
> When I add a component in the .crossbar/config.json and start crossbar,
> I face that behavior once the data have been retrieve by the publisher.
>
> autobahn.wamp.exception.SerializationError: Unable to serialize WAMP
> application payload (can't serialize time.struct_time(tm_year=2015,
> tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0,
> tm_yday=61, tm_isdst=0))

The error says it all: time.struct_time cannot be serialized to JSON.

/Tobias

[foxmask]

I've well understood that

but with crossbar 0.10.1 and autobahn 0.10.0 I dont have any issue like that .

This occurs with crossbar 0.10.2 + autobahn 0.10.1

[Tobias Oberstein]

Am 02.03.2015 um 16:27 schrieb foxmask:
> I've well understood that
>
> but with crossbar 0.10.1 and autobahn 0.10.0 I dont have any issue like
> that .

I doubt that.

time.struct_time can't be serialized .. and this did not change over
AutobahnPython versions.


oberstet@thinkpad-t430s:~/scm/scratchbox$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> ts = time.localtime()
>>> type(ts)
<type 'time.struct_time'>
>>> import json
>>> json.dumps(ts)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/json/__init__.py", line 243, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python2.7/json/encoder.py", line 184, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: time.struct_time(tm_year=2015, tm_mon=3, tm_mday=2,
tm_hour=16, tm_min=59, tm_sec=56, tm_wday=0, tm_yday=61, tm_isdst=0) is
not JSON serializable
>>>


/Tobias

>
> This occurs with crossbar 0.10.2 + autobahn 0.10.1
>
>
> Le lundi 2 mars 2015 16:04:48 UTC+1, Tobias Oberstein a écrit :
>
> Am 02.03.2015 um 14:39 schrieb foxmask:
> > When I add a component in the .crossbar/config.json and start
> crossbar,
> > I face that behavior once the data have been retrieve by the
> publisher.
> >
> > autobahn.wamp.exception.SerializationError: Unable to serialize WAMP
> > application payload (can't serialize time.struct_time(tm_year=2015,
> > tm_mon=3, tm_mday=2, tm_hour=11, tm_min=53, tm_sec=0, tm_wday=0,
> > tm_yday=61, tm_isdst=0))
>
> The error says it all: time.struct_time cannot be serialized to JSON.
>
> /Tobias
>

> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com
> <mailto:autobahnws+...@googlegroups.com>.
> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autobahnws/4cbcc608-0e80-4ccf-b719-44a5d5afcc2d%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/4cbcc608-0e80-4ccf-b719-44a5d5afcc2d%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[foxmask]

Ok ...
So :

I've a publisher that get feeds rss url (and date of the last handling)  from the database
the url is then loaded by feedparser ; and for each item I check the last time the handling has been done
when one item has not been handled I add it to a dict and returnValue() on it.
then for each item I publish it.
on the other side the subcriber receive the stuff.
with crossbar0.10.1 / autobahn 0.10.0 all of this is ok

you can have a look at the 2 components if you want :

1. https://github.com/foxmask/wamp-th/blob/master/components/th_rss/back.py
2. https://github.com/foxmask/wamp-th/blob/master/components/th_evernote/front.py
   

on the front.py line 255 I added print(data) and as you can see the date (published propery) of the RSS is well retrieved

2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] *************************************
2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] Internet
2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client]
2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] Flux de Strict minimum
2015-03-02 17:12:26+0100 [WampWebSocketClientProtocol,client] {u'summary_detail': {u'base': u'http://blog.timothe-leroy.com/feed/tumblog/', u'type': u'text/html', u'value': u'<p><a href="http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5" rel="bookmark" target="_blank" title="Point de vue du pr\xe9sident de MO5 sur la console Ouya">http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5</a></p>\n\t&#8230; et je partage totalement son avis<p><a href="http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/#respond" title="Commentaire sur Point de vue du pr\xe9sident de MO5 sur la console Ouya">Laisser un commentaire</a></p>', u'language': None}, u'published_parsed': {u'tm_sec': 0, u'n_sequence_fields': 9, u'n_unnamed_fields': 0, u'tm_mday': 18, u'tm_isdst': 0, u'tm_year': 2013, u'tm_mon': 7, u'tm_yday': 199, u'tm_hour': 9, u'n_fields': 9, u'tm_wday': 3, u'tm_min': 47}, u'links': [{u'href': u'http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5', u'type': u'text/html', u'rel': u'alternate'}], u'author': u'Tim', u'title': u'Point de vue du pr\xe9sident de MO5 sur la console Ouya', u'slash_comments': u'0', u'comments': u'http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/#comments', u'summary': u'<p><a href="http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5" rel="bookmark" target="_blank" title="Point de vue du pr\xe9sident de MO5 sur la console Ouya">http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5</a></p>\n\t&#8230; et je partage totalement son avis<p><a href="http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/#respond" title="Commentaire sur Point de vue du pr\xe9sident de MO5 sur la console Ouya">Laisser un commentaire</a></p>', u'content': [{u'base': u'http://blog.timothe-leroy.com/feed/tumblog/', u'type': u'text/html', u'value': u'<p><a href="http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5" rel="bookmark" target="_blank" title="Point de vue du pr\xe9sident de MO5 sur la console Ouya">http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5</a></p>\n\t<p>&#8230; et je partage totalement son avis</p>\n<p><a href="http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/#respond" title="Commentaire sur Point de vue du pr\xe9sident de MO5 sur la console Ouya">Laisser un commentaire</a></p>', u'language': None}], u'guidislink': False, u'title_detail': {u'base': u'http://blog.timothe-leroy.com/feed/tumblog/', u'type': u'text/plain', u'value': u'Point de vue du pr\xe9sident de MO5 sur la console Ouya', u'language': None}, u'link': u'http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5', u'authors': [{}], u'author_detail': {u'name': u'Tim'}, u'wfw_commentrss': u'http://blog.timothe-leroy.com/point-de-vue-du-president-de-mo5-sur-la-console-ouya/feed/', u'id': u'http://www.gameblog.fr/news/36845-ouya-le-coup-de-gueule-du-president-de-mo5', u'tags': [{u'term': u'Best of', u'scheme': None, u'label': None}], u'published': u'Thu, 18 Jul 2013 09:47:00 +0000'}

So, my understanding is ; as the data I retrieve from back.py are exchange to front.py and can be displayed ; there is no problem of serialization, isnt it ?

Or am I in a special case that bypass the serialization issue ?

> <mailto:autobahnws+unsub...@googlegroups.com>.

[Alex Grönholm]

The same happens with RPC calls and this is bad because then the caller gets no error response either, and is left waiting forever. I consider this a major bug in Autobahn. If serialization fails, the caller should get a response saying that the call failed because the result couldn't be serialized -- or something similar.

[Tobias Oberstein]

Hi Alex,

> The same happens with RPC calls and this is bad because then the caller
> gets no error response either, and is left waiting forever. I consider
> this a major bug in Autobahn. If serialization fails, the caller should
> get a response saying that the call failed because the result couldn't
> be serialized -- or something similar.

The serialization already fails in the client library of the callee. But
yes, this doesn't lead to an error being sent to the peer. This is a
bug, I agree

https://github.com/tavendo/AutobahnPython/issues/349

Further, even with this, we need proper timeouts inside the router
already (an invocation of an endpoint might not come back for other
reasons ..).

Cheers,
/Tobias

[foxmask]

So am i wrong ? If so how to handle such data ? If not is that a bug ?

[Alex Grönholm]

Don't try to send data that contains struct_times. Alternatively you can customize the msgpack serializer to serialize those to either strings or custom types. I've done something similar in my own app and I made a post about it just a while ago, though it never got any replies: https://groups.google.com/forum/#!topic/autobahnws/T8i-ldq-Piw

To integrate this with autobahn, you need code like this:

def patch_autobahn_msgpack():

    from types import ModuleType
    from autobahn.wamp import serializer
    serializer.msgpack = ModuleType('msgpack')
    serializer.msgpack.packb = partial(packb, default=default)
    serializer.msgpack.unpackb = partial(unpackb, ext_hook=ext_hook)

[Tobias Oberstein]

Am 02.03.2015 um 21:00 schrieb Alex Grönholm:
> Don't try to send data that contains struct_times. Alternatively you can

Exactly. In the case of struct_times, just convert it to a string
according to a standard (like ISO8601)

>>> import time
>>> ts = time.gmtime()
>>> time.strftime("%Y-%m-%dT%H:%M:%SZ", ts)
'2015-03-02T20:10:27Z'

> customize the msgpack serializer to serialize those to either strings or
> custom types. I've done something similar in my own app and I made a
> post about it just a while ago, though it never got any
> replies: https://groups.google.com/forum/#!topic/autobahnws/T8i-ldq-Piw

Sorry, I am still catching up with mailing list posts (answered a bunch
recently).

Custom serialization to string works, but is of limited value, since the
deserialization will not recreate the original types.

Custom serialization to custom types is only possible with msgpack, and
won't work in general, since a router will need to reserialize to other
formats (JSON) transparently.

E.g. a browser speaking WAMP-over-WebSocket/JSON to a C++ client
speaking WAMP-over-RawSocket/MsgPack.

> To integrate this with autobahn, you need code like this:
>
>
> defpatch_autobahn_msgpack():
>
> fromtypesimportModuleType
>
> fromautobahn.wampimportserializer
> serializer.msgpack = ModuleType('msgpack')
> serializer.msgpack.packb = partial(packb,default=default)
> serializer.msgpack.unpackb = partial(unpackb,ext_hook=ext_hook)
>
>
> maanantai 2. maaliskuuta 2015 21.17.31 UTC+2 foxmask kirjoitti:
>
> So am i wrong ? If so how to handle such data ? If not is that a bug ?
>

> --
> You received this message because you are subscribed to the Google
> Groups "Autobahn" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to autobahnws+...@googlegroups.com

> <mailto:autobahnws+...@googlegroups.com>.

> To post to this group, send email to autob...@googlegroups.com
> <mailto:autob...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autobahnws/a2399929-4d3a-4c01-a293-46d9459fedf0%40googlegroups.com
> <https://groups.google.com/d/msgid/autobahnws/a2399929-4d3a-4c01-a293-46d9459fedf0%40googlegroups.com?utm_medium=email&utm_source=footer>.

[foxmask]

Hi
I shouldn't have to do this as the date and time I get come :

1) from a column of a table of my database which is in expected format
2) from RSS data (usually the published node) which are also in the good format.

The fact is ; it's not my code the play with the datetime format

And once again I dont have this behavior with autobahn 0.10.0 / crossbar 0.10.1 ; all work fine.

Regards

> <mailto:autobahnws+unsub...@googlegroups.com>.