In Gremlin Python, when serializing a query with a big vertex ID, an error occurs saying cannot handle the big intere.

85 views
Skip to first unread message

Jason Koh

unread,
Aug 5, 2022, 7:36:36 AM8/5/22
to Gremlin-users
Hi there, I'm actively using Gremlin in my company and happy to learn more from the community.

In this project, I use gremlin-python for ingesting some data. Some vertices have a big vertex ID like 2150838352. So when I send a query to update the graph, I get the following error:
```
File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/process/traversal.py", line 80, in iterate try: self.nextTraverser() File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/process/traversal.py", line 88, in nextTraverser return self.next_traverser() File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/process/traversal.py", line 92, in next_traverser self.traversal_strategies.apply_strategies(self) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/process/traversal.py", line 682, in apply_strategies traversal_strategy.apply(traversal) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/remote_connection.py", line 78, in apply remote_traversal = self.remote_connection.submit(traversal.bytecode) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/driver_remote_connection.py", line 102, in submit result_set = self._client.submit(bytecode, request_options=self._extract_request_options(bytecode)) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/client.py", line 149, in submit return self.submit_async(message, bindings=bindings, request_options=request_options).result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 444, in result return self.__get_result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/connection.py", line 66, in cb f.result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__get_result() File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/protocol.py", line 86, in write message = self._message_serializer.serialize_message(request_id, request_message) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/serializer.py", line 225, in serialize_message args = processor_obj.get_op_args(op, args) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/serializer.py", line 48, in get_op_args return op_method(args) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/driver/serializer.py", line 88, in bytecode args['gremlin'] = self._writer.to_dict(gremlin) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/structure/io/graphbinaryV1.py", line 168, in to_dict return self.serializers[t].dictify(obj, self, to_extend) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/structure/io/graphbinaryV1.py", line 811, in dictify writer.to_dict(arg, to_extend) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/structure/io/graphbinaryV1.py", line 168, in to_dict return self.serializers[t].dictify(obj, self, to_extend) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/structure/io/graphbinaryV1.py", line 672, in dictify writer.to_dict(obj.id, to_extend) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/structure/io/graphbinaryV1.py", line 168, in to_dict return self.serializers[t].dictify(obj, self, to_extend) File "/home/jbkoh/.local/lib/python3.8/site-packages/gremlin_python/structure/io/graphbinaryV1.py", line 253, in dictify to_extend.extend(cls.byte_format_pack(obj)) struct.error: 'i' format requires -2147483648 <= number <= 2147483647 During handling of the above exception, another exception occurred:
```

So I checked the code, and realized that IntIO is a subclass of LongIO and `dictify` is not specified for IntIO, so the range check within that function of LongIO is reused for IntIO. And thus, int32_pack is applied to the number instead of either using int64_pack or raising an error in advance.

So, how can I solve this problem to send an update query with a big vertex ID? Should vertex IDs be Int other than Long? If not, what's the right place to update that in the code?

My running environment is: Ubuntu 20.04, Python 3.8.10, and gremlinpython 3.6.1.

I'd appreciate any help. Thanks!

Kelvin Lawrence

unread,
Aug 6, 2022, 3:57:25 PM8/6/22
to Gremlin-users
To help reproduce this, could you please provide an example of the Python code you were trying to run that uses the big ID?

As you say it looks like the 32-bit packer was invoked on a 64-bit value.

Cheers
Kelvin

Reply all
Reply to author
Forward
0 new messages