Can't pickle repeated scalar fields, convert to list first

430 views
Skip to first unread message

priya_cast_23

unread,
Nov 16, 2023, 7:39:58 AM11/16/23
to Protocol Buffers
I am trying to parallelize a protobuf application
one instance in each thread but I am getting this error

/usr/local/lib/python3.10/dist-packages/google/protobuf/internal/containers.py", line 250, in __reduce__
    raise pickle.PickleError("Can't pickle repeated scalar fields, convert to list first")

I compiled protobuf
./configure CPPFLAGS="-DGOOGLE_PROTOBUF_NO_THREAD_SAFETY"

  def __reduce__(self, **kwargs) -> NoReturn:
    # Convert repeated scalar fields to lists
    raise pickle.PickleError("Can't pickle repeated scalar fields, convert to list first")


  def add(self, **kwargs: Any) -> _T:
    """Adds a new element at the end of the list and returns it. Keyword
    arguments may be used to initialize the element.
    """
    new_element = self._message_descriptor._concrete_class(**kwargs)
    new_element._SetListener(self._message_listener)
    self._values.append(new_element)
    if not self._message_listener.dirty:
      self._message_listener.Modified()
    return new_element

priya_cast_23

unread,
Nov 17, 2023, 6:48:05 AM11/17/23
to Protocol Buffers
I was wondering if someone experienced this error before.
I tried to check if the descriptor is a  repeated scalar fields and convert to a list by
 changing the new_element to a list but it did not work.

I appreciate your time and thoughtful consideration.

sh...@google.com

unread,
Nov 21, 2023, 1:34:16 PM11/21/23
to Protocol Buffers
Can file an issue on github with more details about the protobuf versions, OS, etc: https://github.com/protocolbuffers/protobuf/issues
Reply all
Reply to author
Forward
0 new messages