Max frame length of 65536 has been exceeded. (Gremlin server 3.3.3)

[Damien Seguy]

Hi all,

I'm moving from TP 3.2.9 to TP 3.3.3. So far, this has been almost a drop-in replacement. 

Currently, I'm running into a (rare) problem : 

io.netty.handler.codec.CorruptedFrameException: Max frame length of 65536 has been exceeded.

It happens when I send an array as parameter of a query. So far, 3000 elements in the query is OK,

but 4263 is not anymore. The whole query freeezes.

I found some messages suggesting to raise maxContentLength in the configuration.

It's already 81928192, and a linearized version of the above array is 140kb : that looks like well below the limit.

I upgraded the size to 819281928192, and changed maxChunkSize, maxAccumulationBufferComponents, maxHeaderSize to much larger

values too, with still the same feedback. 

I'm running on OSX 10.11, with Java 8 161 and Neo4j plug-in 3.2.3. 

Any suggestion to pass this ? 

Damien Seguy.

[Stephen Mallette]

>  I'm moving from TP 3.2.9 to TP 3.3.3. So far, this has been almost a drop-in replacement

that's good to know - thanks

as for your problem, just increase the maxContentLength in the server yaml file and the connectionPool.maxContentLength on the driver configuration.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/dbba65f0-c431-4b7b-9dac-4b6e783bbbbd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Damien Seguy]

On Wednesday, May 30, 2018 at 12:34:00 PM UTC+2, Stephen Mallette wrote:

>  I'm moving from TP 3.2.9 to TP 3.3.3. So far, this has been almost a drop-in replacement

that's good to know - thanks

as for your problem, just increase the maxContentLength in the server yaml file and the connectionPool.maxContentLength on the driver configuration.

 maxContentLength is already 81928192 (81 Mb) and the content sent is about 140 kb. 

I tried to add some more (and enlarge other buffers at the same time), but this failed.

The client is not gremlin-client. It's PHP class, and was working fine with the same or larger load on 3.2.9. 

The error I mention is on the server, and I don't even get any feedback on the client side.

Usually, I get also an exception, but this one stays on the server.

Server itself still runs, as I can run another query (may be the initial tread is stuck, though).

`io.netty.handler.codec.CorruptedFrameException: Max frame length of 65536 has been exceeded.` 

Anything else I can try ? 

Damien.

[Stephen Mallette]

I'm not aware of anything else that throws that error or that has a default of that size. could you try to quickly write a gremlin script that you run in the console to use the java driver and then send the same payload to see what happens? If it works, then the problem is somehow with the PHP code, if not, we have something else going on. i can't think of anything that changed between 3.2.x and 3.3.x that would cause this sudden problem.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/32250e7a-b3e3-4088-a611-ada08de3d296%40googlegroups.com.

[Robert Dale]

Damien, can you share your gremlin-server yaml file?  or better, a complete and reproducible test case?

Robert Dale

On Wed, May 30, 2018 at 7:45 AM, Stephen Mallette <spmal...@gmail.com> wrote:

I'm not aware of anything else that throws that error or that has a default of that size. could you try to quickly write a gremlin script that you run in the console to use the java driver and then send the same payload to see what happens? If it works, then the problem is somehow with the PHP code, if not, we have something else going on. i can't think of anything that changed between 3.2.x and 3.3.x that would cause this sudden problem.

On Wed, May 30, 2018 at 7:03 AM Damien Seguy <damien...@gmail.com> wrote:

On Wednesday, May 30, 2018 at 12:34:00 PM UTC+2, Stephen Mallette wrote:

>  I'm moving from TP 3.2.9 to TP 3.3.3. So far, this has been almost a drop-in replacement

that's good to know - thanks

as for your problem, just increase the maxContentLength in the server yaml file and the connectionPool.maxContentLength on the driver configuration.

 maxContentLength is already 81928192 (81 Mb) and the content sent is about 140 kb. 

I tried to add some more (and enlarge other buffers at the same time), but this failed.

The client is not gremlin-client. It's PHP class, and was working fine with the same or larger load on 3.2.9. 

The error I mention is on the server, and I don't even get any feedback on the client side.

Usually, I get also an exception, but this one stays on the server.

Server itself still runs, as I can run another query (may be the initial tread is stuck, though).

`io.netty.handler.codec.CorruptedFrameException: Max frame length of 65536 has been exceeded.` 

Anything else I can try ? 

Damien.

--
You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/32250e7a-b3e3-4088-a611-ada08de3d296%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Gremlin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/CAA-H438VA2BqdrN5h2rZ4DbLWPzbZbOG8JUp3Pg_FUm6bAjodg%40mail.gmail.com.

[Damien Seguy]

On Wednesday, May 30, 2018 at 2:21:26 PM UTC+2, Robert Dale wrote:

Damien, can you share your gremlin-server yaml file?  or better, a complete and reproducible test case?

yaml file is below : fairly standard stuff, with some extended buffers. Well tested, so I'm confident here.

I'll try to reproduce the error with a simple query.

-----YAML---------

host: localhost

port: 8182

scriptEvaluationTimeout: 3000000

channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer

graphs: {

  graph: conf/neo4j-empty.properties}

plugins:

  - tinkerpop.neo4j

scriptEngines: {

  gremlin-groovy: {

    imports: [java.lang.Math],

    staticImports: [java.lang.Math.PI],

    scripts: [scripts/empty-sample.groovy]}}

serializers:

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { useMapperFromGraph: graph }}            # application/vnd.gremlin-v1.0+gryo

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: { useMapperFromGraph: graph }}        # application/vnd.gremlin-v1.0+gryo-lite

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}        # application/vnd.gremlin-v1.0+gryo-stringd

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { useMapperFromGraph: graph }} # application/vnd.gremlin-v1.0+json

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV2d0] }} # application/vnd.gremlin-v2.0+json

  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { useMapperFromGraph: graph }}        # application/json

#  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV1d0]  }}        # application/json

processors:

  - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}

  - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}

metrics: {

  consoleReporter: {enabled: true, interval: 180000},

  csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},

  jmxReporter: {enabled: true},

  slf4jReporter: {enabled: true, interval: 180000},

  gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},

  graphiteReporter: {enabled: false, interval: 180000}}

strictTransactionManagement: false

maxInitialLineLength: 4096

maxHeaderSize: 8192

maxChunkSize: 8192

maxContentLength: 81928192

maxAccumulationBufferComponents: 1024

resultIterationBatchSize: 64

writeBufferLowWaterMark: 32768

writeBufferHighWaterMark: 81928192

ssl: {

  enabled: false}

 

[Robert Dale]

That file did not immediately work for me.  I had to make some changes just to get Gremlin server to start.  Which serializer do you use when encountering this error?

Robert Dale

--

You received this message because you are subscribed to the Google Groups "Gremlin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gremlin-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gremlin-users/ff4553f4-2fbb-4908-809d-ba663f0b5f6f%40googlegroups.com.

[Damien Seguy]

Guys, thanks for the feedback. I'm going to call it a PEBCAK

I started gremlin with gremlin-server.sh start conf/gsneo4.yaml, and it defaulted to gremlin-server.yaml.

Now sorted, and humming.

Thanks for your help :) 

Damien.