HTTP Connection timeout

[NicolasC]

Hi,

I have devices calling HTTP requests handled by Node-red flows.

Node-Red (V0.17.5) is  located on a VM debian 9.

I can see from the device that the connection is closed after ~10".

That is too short for the devices.

When running from a Raspberry (debian 8, node-red V0.17.4), connections are closed after 120".

Where is located the parameter for this timeout ?

Trying with a netcat command, run in the VM, connection is not closed so quickly, so I wonder a parameter in node.js or node-red, dealing with this timeout.

I've searched for a few days with no success. :-(

thank you for any help !

best regards,

Nicolas

[steve rickus]

Sounds like node-red on your VM has different settings than your Raspberry -- check the settings.js file in you .node-red install directory:

    // Timeout in milliseconds for HTTP request connections
    //  defaults to 120 seconds
    //httpRequestTimeout: 120000,

Has this default value been modified?

[NicolasC]

Hi,
Thank you for your quick reply.I indeed checked the settings.js file and found no difference.
I wonder if the quoted parameter is not for outbound http request.
Regards,
Nicolas

[Colin Law]

Your original email appeared to say that requests were from the devices to node red running on the VM, so would be inbound to the VM.

Colin

 

[NicolasC]

That's exactly right.

[Colin Law]

In that case it is not for a outbound request. The request was inbound

Colin

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/e60bb72b-7e4e-4aab-a0ec-ee1e2c59667b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[NicolasC]

Sorry if I was not clear.

Yes, request is inbound to Node-red server.

I use basic http request, that is served by http input, http response output.

I can see that the the connection is closed by the server.

Using wireshark, I can see a FIN flag put in a TCP packet, 5 seconds after the http response.

I don't want this FIN flag 5 seodns after... 120 seconds (as it is on the raspberry) would be ok.

Le jeudi 25 janvier 2018 14:28:04 UTC+1, Colin Law a écrit :

In that case it is not for a outbound request. The request was inbound

Colin

On 25 Jan 2018 9:58 a.m., "NicolasC" <nic.cha...@gmail.com> wrote:

That's exactly right.

Le jeudi 25 janvier 2018 09:35:06 UTC+1, Colin Law a écrit :

On 25 January 2018 at 06:45, NicolasC <nic.cha...@gmail.com> wrote:

Hi,
Thank you for your quick reply.I indeed checked the settings.js file and found no difference.
I wonder if the quoted parameter is not for outbound http request.

Your original email appeared to say that requests were from the devices to node red running on the VM, so would be inbound to the VM.

Colin

 

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

[Colin Law]

On 30 January 2018 at 12:00, NicolasC <nic.cha...@gmail.com> wrote:

Sorry if I was not clear.

Yes, request is inbound to Node-red server.

I use basic http request, that is served by http input, http response output.

I can see that the the connection is closed by the server.

Just be absolutely certain are you saying the connection is closed by the node-red server after it sends the http response?

 

Using wireshark, I can see a FIN flag put in a TCP packet, 5 seconds after the http response.

Are you sure it is the node-red server sending that, not the client?

Colin

 

[NicolasC]

Yes, the connection is closed from the node-red server side.

I run debian 9.3 (just installed it, no customization) , nodejs 8.9.4, node-red 0.17.5

[steve rickus]

Perhaps the next step is to start comparing the npm library versions that are installed on both systems. There could be some newer default settings in the http or tcp libraries that are deployed on each host? Hopefully it's not a hardware-specific difference...

[NicolasC]

I just made a netcat command to simulate the http server response, and the connection is not closed. So i guess it is a node.js or node-red configuration ...

Where should i search for http and tcp libraries configuration ?

Thanks a lot for your responses that encourage me !

[steve rickus]

Try using "npm list" and "npm list -g" to get all of the libs installed on the two systems. I don't expect you will find much, but we are running out of ideas.

However, it also occurred to me that the different behaviors could be caused by something along the http connections -- either a proxy server or even different http headers. Can you use wireshark to capture the two different sets of request/response headers, and see any clues there?

[Bart Butenaers]

Hi Nicolas,

I'm not a Node-Red core expert, but here is what I know about it.  Hopefully it can help you in finding a solution.

• When Node-Red starts, an ExpressJs webserver will be started (in the red.js file).  If I'm not mistaken, no general timeout value is specified here:
  

app.use(settings.httpNodeRoot,RED.httpNode);

I see in this thread that a timeout 'could' be set in this location easily.  But since we don't specify anything here, the default ExpressJs timeout will be used (see below).

•  When the flow is started, the HttpIn node will tell ExpressJs to which request URL it will respond, by adding a Layer (layer.js) in the ExpressJs router (router.js).  This happens in the 21-httpin.js file:
  

RED.httpNode.get(this.url,cookieParser(),httpMiddleware,corsHandler,metricsHandler,this.callback,this.errorHandler);

Remark : The RED.httpNode is the in fact the link to ExpressJs that we have setup in red.js.

 

• As soon as a http request arrives in ExpressJs with the specified URL, the ExpressJs Router will pass the http request (via the Layer) to our HttpIn node's callback function:

 

You can see in the Console window (of the Chrome debugger) that the default timeout of the Http Response is 120000.  At line 196 I have added an extra line to set the timeout of the response object, but that is not available normally in Node-Red.  You can see in the official documentation of NodeJs that you can set a timeout both on the response and the request.

So I think it is normal that you have a response of 120", but it is not clear to me why you have in other cases a timeout of 10".  I don't know which flow exists between your httpin node and your httpresponse node???  Could it be that there is a problem?  The httpin node sends the (wrapped) httprequest/response pair in the output message to the httpresponse node.  The latter node will write it's response to the response object.

Good luck with it !

Bart Butenaers

[NicolasC]

Hi,

Thank you very much for your response.

I have located the file, under /usr/lib/node_modules/node-red/nodes/core/io, and have modified the code as you suggested (res.setTimeout(120000)).

But I still have the TCP connection closed 5 seconds after the response ....

I tried to simulate the service with a netcat command, and the connection is not closed...So i seems the problem is in the nodejs node-red side.

Regards,

Nicolas

[erwin....@gmail.com]

Maybe you could post your flow? It could also be that there is something happening in your flow that causes this.

Op woensdag 31 januari 2018 15:50:46 UTC+1 schreef NicolasC:

[NicolasC]

Hi,

the flows are very basic.

I made a test, coding a simple httpServer in node.js directly, and had the problem.

Here is the source of the problem. It appeared in node.js V8.0.0 : https://nodejs.org/api/http.html#http_server_keepalivetimeout

Now, the question is where to specify the keepAliveTimeout property..

[steve rickus]

So, the problem doesn't exist on the Pi because of an older version of node.js? I thought we checked all of those versions previously...

I'm wondering if the client can add a max keepalive to the request header, to override the default server setting? Either way, this seems like a good candidate for adding to the settings.js file...

[NicolasC]

Yes, sorry, Raspberry has 4.8.3 node.js release, vs 8.9.4 for the VM. I didn't expect such change of behavior for such basic operations.

I agree, it should be in node-red settings.js configuration, ....

Till now, I didn't manage to set keepAliveTimeout in node.js.

This is the short term problem to resolve, to be able to patch node-red.

[Dave C-J]

This seems to be acknowledged upstream https://github.com/nodejs/node/issues/13391

[NicolasC]

Hello, I got the thinks working : 

under /usr/lib/node_modules/node-red, edit red.js.

after server.listen(....);

insert :

server.timeout=0;

server.keepAliveTimeout=120000; // or whatever ...

Thank you for all the help and suggestions , that helped me. 

Hope this can help other people !

Regards,

Nicolas