Received: by 10.66.74.69 with SMTP id r5mr2281000pav.12.1346080344449;
Mon, 27 Aug 2012 08:12:24 -0700 (PDT)
X-BeenThere: kivy-dev@googlegroups.com
Received: by 10.68.231.225 with SMTP id tj1ls6890258pbc.4.gmail; Mon, 27 Aug
2012 08:12:23 -0700 (PDT)
Received: by 10.68.134.129 with SMTP id pk1mr1724593pbb.13.1346080343839;
Mon, 27 Aug 2012 08:12:23 -0700 (PDT)
Date: Mon, 27 Aug 2012 08:12:23 -0700 (PDT)
From: knappador <knappa...@gmail.com>
To: kivy-dev@googlegroups.com
Message-Id: <de0555db-6d66-4ec9-a273-8bdb0f1aa843@googlegroups.com>
In-Reply-To: <1F44FDA4-195A-4430-9ABB-735B452B6383@kivy.org>
References: <aff2a1b7-4383-4a0b-bdf7-0bac66afec23@googlegroups.com> <501EFE6F.5050406@gmail.com> <501F0076.3040909@gmail.com> <010d7705-6088-4d22-bbcc-f25b7d72be4e@googlegroups.com> <ca341e62-8f57-444a-85dc-15fb0c919053@googlegroups.com> <5cb2b250-a3bd-441d-b7ed-4d98b2fbed47@googlegroups.com>
<1F44FDA4-195A-4430-9ABB-735B452B6383@kivy.org>
Subject: Re: [kivy-dev] Remote Debugger
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_Part_512_11884509.1346080343230"
------=_Part_512_11884509.1346080343230
Content-Type: multipart/alternative;
boundary="----=_Part_513_20924365.1346080343232"
------=_Part_513_20924365.1346080343232
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
I found that iPython has a client-server model that can run over sockets.=
=20
If I can figure out the tornado event loop and make it cooperate with=20
Kivy, we can have an iPython shell. It's a little better and lighter than=
=20
the manhole shell IMO. When I tried Kivy Remote Shell I got some=20
half-printed exceptions quite a bit.
The most basic unit of execution in the client-server is a function even=20
for attribute lookups. This makes it possible to get the cooperative=20
locking done correctly in all cases without membranes and all the problems=
=20
they cause (leakage and type checking errors). If I can make this solution=
=20
mirror the API of InteractiveLauncher I'd like to just replace the whole=20
thing.
After looking at tons of possible solutions, the need for an ssh server=20
seemed to always point back to using twisted conch. if all target devices=
=20
can run ssh servers, conch isn't needed. in either case, IPython can be=20
forwarded over an ssh tunnel if security is desired. I imagine security is=
=20
not a bad investment because exploiting open ports on networks without=20
strong firewalls is too easy for someone who has access to the code.
On Tuesday, August 7, 2012 5:43:28 AM UTC-5, Mathieu Virbel wrote:
>
> That's just another serialization format, but doesn't resolve the main=20
> issue: kivy widget cannot be serialized from the start.
>
> I think, to make bpython/ipython/pudb/... work with introspection, we nee=
d=20
> Clock + a little wrapper (similar to SafeMembrane, but only for=20
> introspection doc/properties...)
>
> Le 6 ao=C3=BBt 2012 =C3=A0 18:37, aspidites a =C3=A9crit :
>
> Bleh. Was trying to change email settings and unsubscribed instead.=20
> Anyways, it might be worth mentioning=20
> http://rpyc.sourceforge.net/docs.html which is similar in spirit to Pyro =
(
> http://stackoverflow.com/questions/1410328/what-are-the-pros-and-cons-of-=
pyro-and-rpyc-python-libs
> );
>
> That said, whats wrong with plain-old json/xml rpc?
>
> On Monday, August 6, 2012 9:22:14 AM UTC-5, knappador wrote:
>>
>> I'm really tempted to just forward stdin,stderr, and stdout. Figuring=
=20
>> out when to lock the thread is a bit trickier without knowing that a met=
hod=20
>> returned etc. Debuggers would work better though. =20
>>
>> On Monday, August 6, 2012 6:22:40 AM UTC-5, knappador wrote:
>>>
>>> It might be worth it to take another look at eval/exec style=20
>>> implementations to verify the utility ( or lack thereof ) for alternati=
ve=20
>>> python shells. Twisted manhole as well. This is the big motivator for=
=20
>>> transferring any proxy in the first place.
>>>
>>> The two choices for the remote communication are something using=20
>>> exec/eval or transferring an object proxy. Having objects available=20
>>> locally allows IPython, BPython etc to do their thing (right?), so we g=
et=20
>>> fast API learning/remembering. Proxies themselves are problematic, not=
=20
>>> only because of bugs, but run-time type-checking. Pyro is much more ma=
ture=20
>>> than any proxy I'm going to create and I can use the communication even=
t=20
>>> itself to hook into pausing kivy.clock. The thread that runs pyro woul=
d=20
>>> lock the kivy process and no membrane objects would be necessary. All=
=20
>>> associated type problems go away.
>>>
>>> The problem at that point is how to create new objects. Instead of=20
>>> sending them over the wire, if a factory is used, (or a hack that impor=
ts=20
>>> kivy namespace objects as Pyro objects inside the already running=20
>>> application thread) then the objects never have to be transferred. Sim=
ply=20
>>> an RPC takes place and the constructor is actually run in the execution=
=20
>>> context of the paused process. No complicated serializations.
>>>
>>> If using a namespace hack, I would simply override any imports of kivy=
=20
>>> and it's modules to instead call up Pyro objects from the running App's=
=20
>>> context. Not sure on implementation, but would require no factory chan=
ges=20
>>> or other tricks. I'm half tempted to use factory just because I don't =
use=20
>>> it enough and writing imports all the time is tedious =3D)
>>>
>>> On Sunday, August 5, 2012 6:23:34 PM UTC-5, Mathieu Virbel wrote:
>>>>
>>>> > Partially. Tell me where you get an interest to copy an object=20
>>>> locally ?=20
>>>> > What things can you do that a remote execution cannot do ?=20
>>>> > Pyro sounds cool, but i think that we might have too much magic to=
=20
>>>> make=20
>>>> > it work.=20
>>>> >=20
>>>> > Just imagining to pickle an Image widget:=20
>>>> > 1. Image widget + Widget + EventDispatcher=20
>>>> > 2. All the callback associated to it (i think 6 registration for 4=
=20
>>>> > compiled callback from the style.kv). By default, code object cannot=
=20
>>>> be=20
>>>> > pickled: pickle.dumps(compile('self.center_x -=20
>>>> self.norm_image_size[0] /=20
>>>> > 2., self.center_y - self.norm_image_size[1] / 2.', '<string>',=20
>>>> 'eval'))=20
>>>> > -> TypeError: can't pickle code objects=20
>>>> > 3. All the OpenGL object: Color state, Rectangle (=3Done 1 Buffer/2=
=20
>>>> VBO) +=20
>>>> > the texture (1 Texture)=20
>>>>
>>>> Just readed on the website:=20
>>>>
>>>> The pickle protocol is very convenient to transfer Python objects=
=20
>>>> =E2=80=98over the wire=E2=80=99 but it also has a drawback: it ser=
ializes the whole=20
>>>> object graph you=E2=80=99re passing=20
>>>>
>>>> So it cannot be achieve that way :/=20
>>>>
>>>>
>>>> I'm sure you can get the same "usability" that pyro give, but in remot=
e=20
>>>> with Clock :)=20
>>>>
>>>> You could even change the main loop in EventBase.idle() to make things=
=20
>>>> like:=20
>>>>
>>>> def idle(self):=20
>>>> ...=20
>>>> while not self.debug_pause:=20
>>>> Clock.tick()=20
>>>> ...=20
>>>>
>>>> And then from the debugger, you can have a command like "stop", and th=
e=20
>>>> main thread will just iterate over the Clock, no rendering, not even=
=20
>>>> input event... until you send the command "continue".=20
>>>>
>>>> I even think that the best way would be to create a DebugClock, that=
=20
>>>> only your debug thread is using. And don't click the main clock at all=
=20
>>>> -> all the app / threads app / gl that use the main clock will just be=
=20
>>>> freezed. Ideal to debug :)=20
>>>>
>>>> (from kivy.clock import ClockBase; DebugClock =3D ClockBase() ->=20
>>>> Then in the EventBase:=20
>>>> if self._debug_clock:=20
>>>> while not self._debug_paused:=20
>>>> self._debug_clock.tick()=20
>>>> )=20
>>>>
>>>> Is that ok for you ?=20
>>>>
>>>> Mathieu=20
>>>>
>>>
>
------=_Part_513_20924365.1346080343232
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
I found that iPython has a client-server model that can run over sockets. &=
nbsp;If I can figure out the tornado event loop and make it cooperate=
with Kivy, we can have an iPython shell. It's a little better and li=
ghter than the manhole shell IMO. When I tried Kivy Remote Shell I go=
t some half-printed exceptions quite a bit.<br><br>The most basic unit of e=
xecution in the client-server is a function even for attribute lookups. &nb=
sp;This makes it possible to get the cooperative locking done correctly in =
all cases without membranes and all the problems they cause (leakage and ty=
pe checking errors). If I can make this solution mirror the API of In=
teractiveLauncher I'd like to just replace the whole thing.<div><div><br></=
div><div>After looking at tons of possible solutions, the need for an ssh s=
erver seemed to always point back to using twisted conch. if all targ=
et devices can run ssh servers, conch isn't needed. in either case, I=
Python can be forwarded over an ssh tunnel if security is desired. I =
imagine security is not a bad investment because exploiting open ports on n=
etworks without strong firewalls is too easy for someone who has access to =
the code.<br><br>On Tuesday, August 7, 2012 5:43:28 AM UTC-5, Mathieu Virbe=
l wrote:<blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0=
.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div style=3D"word-wra=
p:break-word">That's just another serialization format, but doesn't resolve=
the main issue: kivy widget cannot be serialized from the start.<div><br><=
/div><div>I think, to make bpython/ipython/pudb/... work with introspection=
, we need Clock + a little wrapper (similar to SafeMembrane, but only for i=
ntrospection doc/properties...)</div><div><br><div><div>Le 6 ao=C3=BBt 2012=
=C3=A0 18:37, aspidites a =C3=A9crit :</div><br><blockquote type=3D"cite">=
Bleh. Was trying to change email settings and unsubscribed instead. Anyways=
, it might be worth mentioning <a href=3D"http://rpyc.sourceforge.net/docs.=
html" target=3D"_blank">http://rpyc.sourceforge.net/<wbr>docs.html</a> whic=
h is similar in spirit to Pyro (<a href=3D"http://stackoverflow.com/questio=
ns/1410328/what-are-the-pros-and-cons-of-pyro-and-rpyc-python-libs" target=
=3D"_blank">http://stackoverflow.com/<wbr>questions/1410328/what-are-<wbr>t=
he-pros-and-cons-of-pyro-and-<wbr>rpyc-python-libs</a>);<br><br>That said, =
whats wrong with plain-old json/xml rpc?<br><br>On Monday, August 6, 2012 9=
:22:14 AM UTC-5, knappador wrote:<blockquote class=3D"gmail_quote" style=3D=
"margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">I'=
m really tempted to just forward stdin,stderr, and stdout. Figuring o=
ut when to lock the thread is a bit trickier without knowing that a method =
returned etc. Debuggers would work better though. <br><br>On Mo=
nday, August 6, 2012 6:22:40 AM UTC-5, knappador wrote:<blockquote class=3D=
"gmail_quote" style=3D"margin:0;margin-left:0.8ex;border-left:1px #ccc soli=
d;padding-left:1ex"><div>It might be worth it to take another look at eval/=
exec style implementations to verify the utility ( or lack thereof ) for al=
ternative python shells. Twisted manhole as well. This is the b=
ig motivator for transferring any proxy in the first place.<br></div><div><=
br></div>The two choices for the remote communication are something using e=
xec/eval or transferring an object proxy. Having objects available lo=
cally allows IPython, BPython etc to do their thing (right?), so we get fas=
t API learning/remembering. Proxies themselves are problematic, not o=
nly because of bugs, but run-time type-checking. Pyro is much more ma=
ture than any proxy I'm going to create and I can use the communication eve=
nt itself to hook into pausing kivy.clock. The thread that runs pyro =
would lock the kivy process and no membrane objects would be necessary. &nb=
sp;All associated type problems go away.<br><br>The problem at that point i=
s how to create new objects. Instead of sending them over the wire, i=
f a factory is used, (or a hack that imports kivy namespace objects as Pyro=
objects inside the already running application thread) then the objects ne=
ver have to be transferred. Simply an RPC takes place and the constru=
ctor is actually run in the execution context of the paused process. =
No complicated serializations.<div><br></div><div>If using a namespace hack=
, I would simply override any imports of kivy and it's modules to instead c=
all up Pyro objects from the running App's context. Not sure on imple=
mentation, but would require no factory changes or other tricks. I'm =
half tempted to use factory just because I don't use it enough and writing =
imports all the time is tedious =3D)<br><br>On Sunday, August 5, 2012 6:23:=
34 PM UTC-5, Mathieu Virbel wrote:<blockquote class=3D"gmail_quote" style=
=3D"margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"=
>> Partially. Tell me where you get an interest to copy an object locall=
y ?
<br>> What things can you do that a remote execution cannot do ?
<br>> Pyro sounds cool, but i think that we might have too much magic to=
make
<br>> it work.
<br>>
<br>> Just imagining to pickle an Image widget:
<br>> 1. Image widget + Widget + EventDispatcher
<br>> 2. All the callback associated to it (i think 6 registration for 4
<br>> compiled callback from the style.kv). By default, code object cann=
ot be
<br>> pickled: pickle.dumps(compile('self.<wbr>center_x - self.norm_imag=
e_size[0] /
<br>> 2., self.center_y - self.norm_image_size[1] / 2.', '<string>=
', 'eval'))
<br>> -> TypeError: can't pickle code objects
<br>> 3. All the OpenGL object: Color state, Rectangle (=3Done 1 Buffer/=
2 VBO) +
<br>> the texture (1 Texture)
<br>
<br>Just readed on the website:
<br>
<br> The pickle protocol is very convenient to transfer Python=
objects
<br> =E2=80=98over the wire=E2=80=99 but it also has a drawbac=
k: it serializes the whole
<br> object graph you=E2=80=99re passing
<br>
<br>So it cannot be achieve that way :/
<br>
<br>
<br>I'm sure you can get the same "usability" that pyro give, but in remote=
=20
<br>with Clock :)
<br>
<br>You could even change the main loop in EventBase.idle() to make things =
like:
<br>
<br> def idle(self):
<br> ...
<br> while not self.debug_pause:
<br> Clock.tick()
<br> ...
<br>
<br>And then from the debugger, you can have a command like "stop", and the=
=20
<br>main thread will just iterate over the Clock, no rendering, not even=20
<br>input event... until you send the command "continue".
<br>
<br>I even think that the best way would be to create a DebugClock, that=20
<br>only your debug thread is using. And don't click the main clock at all=
=20
<br>-> all the app / threads app / gl that use the main clock will just =
be=20
<br>freezed. Ideal to debug :)
<br>
<br>(from kivy.clock import ClockBase; DebugClock =3D ClockBase() ->
<br>Then in the EventBase:
<br>if self._debug_clock:
<br> while not self._debug_paused:
<br> self._debug_clock.tick()
<br>)
<br>
<br>Is that ok for you ?
<br>
<br>Mathieu
<br></blockquote></div></blockquote></blockquote></blockquote></div><br></d=
iv></div></blockquote></div></div>
------=_Part_513_20924365.1346080343232--
------=_Part_512_11884509.1346080343230--
Message from discussion Remote Debugger
| Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy |
| ©2013 Google |