Re: Question about PodSixNet
ch...@mccormick.cx
Sorry for the slow reply. I hope you don't mind me CC'ing the mailing list with
your query.
With regards to figuring out how much data has been sent, the best way to do
that is get the return value of the Send() method when you are sending data.
If you want to see how much data is received on the receiving end you could
patch the Channel class' collect_incoming_data() method to print out the amount
of bytes it receives each time.
Hope this helps!
Cheers,
Chris.
On Sat, Jan 22, 2011 at 06:59:08PM +0000, Dan Horn wrote:
> Hi Chris,
>
> First let me thank you for PodSixNet, it's a nice little library :)
>
> My question involves packet sizes. Following your examples, if I send a list in the way you demonstrate it, a sys.getsizeof(data) reveals the structure to be 140bytes (on both the sending and receiving end). However, when I do a sizeof on the data itself, the byte size varies appropriately based on what I actually sent.
>
> For example, if I send 1000 integers just doing a range(1000) in a list, the sizeof(data['numbers']) is like ~4000 bytes, but the sizeof on the data structure itself still is only 140 bytes. I figure this is due to weak references so the actual encapsulated data is what is sent over the wire, in which case, I'm not sure about actual network bandwidth usage. I'm still new to python though, so maybe it's something else. It seems the data itself would be the overhead plus the data it actually contains, but that doesn't seem to be the case.
>
> My next step is to sniff it out with wireshark or some such, but I was curious if you could shed any light on the size of packets/overhead while using PodSixNet. Overall, sending Python data over the wire is far more costly than say, compiled C++ (for some reason, integers in python measure around 10 to 12 bytes, depending). Is sys.getsizeof() just overly generous, or have you noticed packet sizes being quite hefty on the overhead as well?
>
> Thanks in advance,
> -Dan
-------------------
http://mccormick.cx
catch22
to report a little less than what is received on the other end of the
client, though I imagine that's probably due to protocol overhead
(it's a pretty consistent 19 byte delta).
My concerns of size were related to dictionaries. Definitely send
lists over the wire instead! A dictionary containing the same data as
a list would take twice or more the amount of bytes to send (if these
numbers are correct).
The only other oddity I've seen here is even though I'm sending the
same amount of data, the values reported by both sides steadily grow.
I can't figure out if this is an issue with PodSixNet, or if the data
sizes in python just grow as data does (or related to memory/garbage
collection?)
Here's a snip of what I'm talking about. First is the byte amount
returned by Send, then the list I'm actually sending (first item
only), the last number is the actual length of the list itself (# of
objects). Data is ID, X, Y, Angle, Health. 5 packets per second.
The fluctuation is normal since objects get dropped then recreated, so
sometimes the list is smaller. The troubling part is the fact the
data steadily grows in size the longer I let the simulation run.
The first few seconds of running the simulation:
183 [1, 359, 354, 0, 15] 13
192 [1, 369, 354, 0, 15] 14
194 [1, 377, 357, 45, 15] 14
195 [1, 385, 365, 45, 15] 14
196 [1, 392, 372, 45, 15] 14
196 [1, 394, 381, 90, 15] 14
183 [1, 394, 391, 90, 15] 13
192 [1, 394, 401, 90, 15] 14
194 [1, 394, 411, 90, 15] 14
195 [1, 394, 421, 90, 15] 14
196 [1, 394, 431, 90, 15] 14
196 [1, 399, 439, 45, 15] 14
183 [9, 145, 365, 45, 15] 13
A couple minutes later...
209 [136, 230, 237, -45, 15] 13
220 [136, 237, 237, 45, 15] 14
222 [136, 244, 244, 45, 15] 14
223 [136, 251, 251, 45, 15] 14
224 [136, 258, 258, 45, 15] 14
224 [136, 266, 265, 45, 15] 14
209 [136, 273, 272, 45, 15] 13
220 [136, 280, 280, 45, 15] 14
222 [136, 287, 287, 45, 15] 14
223 [136, 294, 294, 45, 15] 14
224 [136, 301, 301, 45, 15] 14
224 [136, 308, 308, 45, 15] 14
209 [136, 315, 315, 45, 15] 13
220 [136, 322, 322, 45, 15] 14
222 [136, 329, 329, 45, 15] 14
223 [136, 336, 336, 45, 15] 14
224 [136, 343, 343, 45, 15] 14
224 [136, 350, 350, 45, 15] 14
Any thoughts on this? It continues to grow if I let it sit there.
About every minute it jumps 10 bytes for some reason. I thought
initially it was the way python assigns memory, but I'm not so sure,
these values don't change that much. That shouldn't cause a change in
size allocation, I wouldn't think. So I'm stumped :)
-Dan
Chris McCormick
Thanks for drawing this to my attention. I will have to do some poking around
and see what is causing this. I suspect it's a memory leak somewhere, something
not being cleaned up correctly in between packets, but I will have a look.
Cheers,
Chris.
> On Jan 24, 11:19ï¿œpm, "ch...@mccormick.cx" <ch...@mccormick.cx> wrote:
> > Hi Dan,
> >
> > Sorry for the slow reply. I hope you don't mind me CC'ing the mailing list with
> > your query.
> >
> > With regards to figuring out how much data has been sent, the best way to do
> > that is get the return value of the Send() method when you are sending data.
> >
> > If you want to see how much data is received on the receiving end you could
> > patch the Channel class' collect_incoming_data() method to print out the amount
> > of bytes it receives each time.
> >
> > Hope this helps!
> >
> > Cheers,
> >
> > Chris.
> >
> > On Sat, Jan 22, 2011 at 06:59:08PM +0000, Dan Horn wrote:
> > > Hi Chris,
> >
> > > First let me thank you for PodSixNet, it's a nice little library :)
> >
> > > My question involves packet sizes. ï¿œFollowing your examples, if I send a list in the way you demonstrate it, a sys.getsizeof(data) reveals the structure to be 140bytes (on both the sending and receiving end). ï¿œHowever, when I do a sizeof on the data itself, the byte size varies appropriately based on what I actually sent.
> >
> > > For example, if I send 1000 integers just doing a range(1000) in a list, the sizeof(data['numbers']) is like ~4000 bytes, but the sizeof on the data structure itself still is only 140 bytes. ï¿œI figure this is due to weak references so the actual encapsulated data is what is sent over the wire, in which case, I'm not sure about actual network bandwidth usage. ï¿œI'm still new to python though, so maybe it's something else. ï¿œIt seems the data itself would be the overhead plus the data it actually contains, but that doesn't seem to be the case.
> >
> > > My next step is to sniff it out with wireshark or some such, but I was curious if you could shed any light on the size of packets/overhead while using PodSixNet. ï¿œOverall, sending Python data over the wire is far more costly than say, compiled C++ (for some reason, integers in python measure around 10 to 12 bytes, depending). ï¿œIs sys.getsizeof() just overly generous, or have you noticed packet sizes being quite hefty on the overhead as well?
> >
> > > Thanks in advance,
> > > -Dan
> >
> > -------------------http://mccormick.cx
>
> --
> You received this message because you are subscribed to the Google Groups "PodSixNet" group.
> To post to this group, send email to pods...@googlegroups.com.
> To unsubscribe from this group, send email to podsixnet+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/podsixnet?hl=en.
>
-------------------
http://mccormick.cx
Chris McCormick
Is there any chance you could send me the code you are using to test this so I
can reproduce it and fix it?
Cheers,
Chris.
On Thu, Jan 27, 2011 at 11:41:34AM -0800, catch22 wrote:
> On Jan 24, 11:19ï¿œpm, "ch...@mccormick.cx" <ch...@mccormick.cx> wrote:
> > Hi Dan,
> >
> > Sorry for the slow reply. I hope you don't mind me CC'ing the mailing list with
> > your query.
> >
> > With regards to figuring out how much data has been sent, the best way to do
> > that is get the return value of the Send() method when you are sending data.
> >
> > If you want to see how much data is received on the receiving end you could
> > patch the Channel class' collect_incoming_data() method to print out the amount
> > of bytes it receives each time.
> >
> > Hope this helps!
> >
> > Cheers,
> >
> > Chris.
> >
> > On Sat, Jan 22, 2011 at 06:59:08PM +0000, Dan Horn wrote:
> > > Hi Chris,
> >
> > > First let me thank you for PodSixNet, it's a nice little library :)
> >
> > > My question involves packet sizes. ï¿œFollowing your examples, if I send a list in the way you demonstrate it, a sys.getsizeof(data) reveals the structure to be 140bytes (on both the sending and receiving end). ï¿œHowever, when I do a sizeof on the data itself, the byte size varies appropriately based on what I actually sent.
> >
> > > For example, if I send 1000 integers just doing a range(1000) in a list, the sizeof(data['numbers']) is like ~4000 bytes, but the sizeof on the data structure itself still is only 140 bytes. ï¿œI figure this is due to weak references so the actual encapsulated data is what is sent over the wire, in which case, I'm not sure about actual network bandwidth usage. ï¿œI'm still new to python though, so maybe it's something else. ï¿œIt seems the data itself would be the overhead plus the data it actually contains, but that doesn't seem to be the case.
> >
> > > My next step is to sniff it out with wireshark or some such, but I was curious if you could shed any light on the size of packets/overhead while using PodSixNet. ï¿œOverall, sending Python data over the wire is far more costly than say, compiled C++ (for some reason, integers in python measure around 10 to 12 bytes, depending). ï¿œIs sys.getsizeof() just overly generous, or have you noticed packet sizes being quite hefty on the overhead as well?
> >
> > > Thanks in advance,
> > > -Dan
> >
> > -------------------http://mccormick.cx
>