dict as parameter to a remote method ?
Edomaur
I think I need some help... I am trying to use a dict as a parameter
to a remote procedure provided by a Service class, but it doesn't
seems to word. Did I miss something ? Every time I try to use the
parameter as a dictionary, results in an exception, except for simply
printing it to the screen.
An example :
Service method :
def exposed_writeData(self, topic, data):
print data
for key, value in data.items() :
newdata[key] = value
print newdata["truc"]
Client function :
def testaction(arg):
print ">>>TASK",arg,"sleeping 3 seconds"
c = rpyc.connect_by_service("CeluiCi")
c.root.writeData("trucmuche",{"alalala":235245, "truc":"bulle"})
time.sleep(3)
print "trucmuche"
print "<<<END_TASK",arg
Resulting exception :
>>>TASK task 1 sleeping 3 seconds
ERROR DURING TASK EXECUTION cannot access 'items'
Traceback (most recent call last):
File "/opt/openconnect/ocd/schedule.py", line 469, in threadedcall
self.execute()
File "/opt/openconnect/ocd/schedule.py", line 313, in execute
self.action(*self.args, **self.kw)
File "occlientd.py", line 11, in testaction
c.root.writeData("trucmuche",{"alalala":235245, "truc":"bulle"})
File "/usr/local/lib/python2.6/dist-packages/rpyc-3.0.6-py2.6.egg/
rpyc/core/netref.py", line 123, in __call__
return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
File "/usr/local/lib/python2.6/dist-packages/rpyc-3.0.6-py2.6.egg/
rpyc/core/netref.py", line 45, in syncreq
return conn().sync_request(handler, oid, *args)
File "/usr/local/lib/python2.6/dist-packages/rpyc-3.0.6-py2.6.egg/
rpyc/core/protocol.py", line 329, in sync_request
raise obj
AttributeError: cannot access 'items'
Thank you for your help.
--
Antoine Boegli
Manuel Naranjo
> Hello !
>
> I think I need some help... I am trying to use a dict as a parameter
> to a remote procedure provided by a Service class, but it doesn't
> seems to word. Did I miss something ? Every time I try to use the
> parameter as a dictionary, results in an exception, except for simply
> printing it to the screen.
>
the stack trace:
sync_request !
You're not getting an object byValue but rather byRef, this means if you
make a change to the dict on the server it will get reflected on the
client. Even though this some might make sense for calling methods on
the other end, it doesn't for the arguments dict (which is your case).
My solution to this issue was to do pickle.dumps on the client and
pickle.loads on the other end. I know this sucks, but I figured out it's
a good solution, as I don't want modifications on the server to be
reflected on the client.
Rpyc has been designed to be very flexible and I love that, but this
kind of issues are the ones that are hard to work with. Maybe it's time
to drop sync calls and start doing all async? inLineGenerators from
twisted is a nice example on how to make the code look sync while
actually is async. A few weeks ago there was the first PyCon Argentina,
I've been there and someone showed this method, I think it's worth
studying it to make rypc even more robust. The base for this is using
yield when ever the request is going to go through the network.
Manuel
tomer filiba
rpyc/core/protocol.py", line 329, in sync_request
raise obj
AttributeError: cannot access 'items'
that's because your connection, by default, is not configured to allow access to all attributes.
to solve that, when creating the connection, pass allow_public_attrs = True, like so:
c = rpyc.connect_by_service("CeluiCi", config = {"allow_public_attrs" : True})
that should do the trick. for other configuration parameters that you might find useful, see:
http://sebulbasvn.googlecode.com/svn/tags/rpyc/3.0.6/core/protocol.py
hope this helps,
-tomer
tomer filiba
simply waiting for the result. the problem is about configuring the right permission to accessing attributes.
about @inlineCallbacks, yes, it is indeed a great way, but it doesn't fit rpyc's model.
rpyc strives to be transparent, i.e., your code should work the same on local objects as well as remote ones.
with @inlineCallbacks, an invocation like this:
c.modules.os.listdir("/")
would look like
@inlineCallbacks
def foo():
r1 = yield c.modules.os # remote-getattr to get module
r2 = yield r1.listdir # remote-getattr to get function
r3 = yield r2("/") # remote-invocation of the function
or, more succinctly:
yield (yield (yield c.modules.os).listdir)("/")
either way, it's not feasible. and even if we did work that way, a function that operates on a remote object could not operate on a local one.
for instance, today, this function is agnostic to rpyc:
def f(x):
x.append(5)
using the twistedish approach, we would have to use two versions:
def f_on_local(x):
x.append(5)
@inlineCallbacks
def f_on_remote(x):
yield (yield x.append)(5)
bottom line, mixing synchronous and async programming models do not play well together.
either you write your whole program twisted, or you stick with the "normal" sync model.
-tomer
An NCO and a Gentleman
Manuel Naranjo
> well, this has nothing to do with the type of the request -- in fact,
> all requests are async, with sync_request
> simply waiting for the result. the problem is about configuring the
> right permission to accessing attributes.
seen sync_async doing it, some how are different.
> rpyc strives to be transparent, i.e., your code should work the same
> on local objects as well as remote ones.
> with @inlineCallbacks, an invocation like this:
>
> c.modules.os.listdir("/")
>
> would look like
> @inlineCallbacks
> def foo():
> r1 = yield c.modules.os # remote-getattr to get module
> r2 = yield r1.listdir # remote-getattr to get function
> r3 = yield r2("/") # remote-invocation of the function
>
> or, more succinctly:
> yield (yield (yield c.modules.os).listdir)("/")
Not totally sure about this. That's because rpyc makes a difference
between sync and async calls. But I'm not such an expert on python to
tell how it should look like.
> bottom line, mixing synchronous and async programming models do not
> play well together.
> either you write your whole program twisted, or you stick with the
> "normal" sync model.
Maybe it's just because of GIL, but this "normal" model you talk about
is not possible on Linux so far. It eventually stops. Specially when you
have a system (which I'm not sure I designed right, sorry I'm an
electronic engineer, not an informatics guy) that uses a sort of
signaling system, where the calls goes all the time from one side to the
other.
Now I had been able to trace all the problems down to the call to
<connection>.root a sync call, that halts the thread when one side is
waiting for the other to return. Maybe rpyc needs a way to define
byReference and byValue variables? For example in a signal system I send
the signal to the remote end and I drop the variable on my side. If the
other side keeps a reference then it makes the local site headache, and
the garbace collector is getting into troubles too.
The "Hackish" solution I found is pickling, but maybe it's time for a
better way? Maybe something like a rpyc.byValue() function? by default
you pass by reference, but if you want you can do it by value? If you
think this sounds good I can try implementing this and creating a patch.
I have many uses for this.
Manuel
Antoine Boegli
For the moment, since I'm working with MongoDB anyway, I'm using it
directly through pymongo to transmit dictionaries. Rpyc is used to get
some datas, like timestamps and so on, and to synchronise operations
of the different serveur accessing collections in MongoDB.
My skill as a programmer is rusty, to say the least... But since my
program will be open, I'll give you a like to the bitbucket repository
when available. :-)
--
Antoine Boegli
software engineer & linux expert
tomer filiba
The "Hackish" solution I found is pickling, but maybe it's time for a better way? Maybe something like a rpyc.byValue() function? by default you pass by reference, but if you want you can do it by value? If you think this sounds good I can try implementing this and creating a patch. I have many uses for this.
-tomer
Manuel Naranjo
Ohh right it does it, not the same way I think on how to do it, but it
does it. What about moving this to rpyc.utils.helpers and importing into
rpyc?
Still I would change it, it transmits the remote proxy and then
pickle/unpickle on the local end, what I do is sending the pickled
object as I would transmit any string. What about adding it to brine so
that brine makes this seems less to the other end?.
Something like
A = rpyc.utils.byRef(<object>)
conn.root.somemethod(A)
On the other end you would have:
def exposed_somemethod(A):
print A
And the internals is handled by brine?
Manuel