Python protobuf speed and latest Protobuf version

[timothy.m]

Looking at this thread (http://groups.google.com/group/protobuf/
browse_thread/thread/9acb7638e1f5ff61), it seems like protobuf in
Python needed significant work as of July 2008. Looking at the
versions on the downloads page, it doesn't appear that there have been
any other releases since then. Has there been more development on this
issue?

I am working on a project that would really benefit from a Python
protobuf speed boost.

Regards,

Tim

[Jason Hsueh]

The last release, 2.3.0, includes significant improvments to Python: http://protobuf.googlecode.com/svn/trunk/CHANGES.txt There have also been further improvements since then (the C extension-based implementation mentioned in http://groups.google.com/group/protobuf/browse_thread/thread/6b5d6373b2391fb4/a1cb14d7baad559f), but they have not yet been released.

--
You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
To post to this group, send email to prot...@googlegroups.com.
To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.

[Peter Schuller]

> Looking at this thread (http://groups.google.com/group/protobuf/
> browse_thread/thread/9acb7638e1f5ff61), it seems like protobuf in
> Python needed significant work as of July 2008. Looking at the
> versions on the downloads page, it doesn't appear that there have been
> any other releases since then. Has there been more development on this
> issue?

While sifting through 2.2.0->2.3.0 history I saw commits with major
changes to the Python bits indicating 2.3.0 should be *significantly*
faster (due to less dynamic introspection), but I have not benchmarked
it yet. 2.3.0 was released earlier this year I believe (based on the
release dates of the files).

--
/ Peter Schuller

[Kenton Varda]

The download page says 2.3.0 was released "Jan 08".  You probably thought that meant 2008, but it actually means Jan 8th of this year.  This threw me off too, when I looked at it.  I'll file a bug against the codesite people.

On Fri, Jun 18, 2010 at 6:59 AM, timothy.m <timothy...@gmail.com> wrote:

[Kenton Varda]

http://code.google.com/p/support/issues/detail?id=4116

[Will P]

Have you done any profiling to see where you're seeing the code spend
the most time? I contributed a small python performance enhancement
patch many months ago, which was accepted and is now part of the
wire_format.py code (it wasn't much, I just unrolled a loop really).
I'm interested in helping out to speed up the python code. Do you
know where in the code you would want to see more optimization? Do
you find any specific primitive types to be worse than others? What
are the sizes and number of fields of your protobuf messages? Can you
share your .proto file and any sample code that sit in your critical
code path?

I'm definitely interested in helping out here.

Thanks!
-W

[Kenton Varda]

Version 2.3.0 introduced a complete overhaul of the internals of the Python implementation, drastically improving performance.  I don't think it can be pushed much further while still remaining pure-python.  Petar is still working on an implementation which wraps the C++ protobuf implementation, thus making it vastly faster.  It should be ready for the next release, probably sometime this summer.