Re: discontinuation JSON object

[Dmitry Utkin]

Which HTTP client do you use? What is on the server side, nginx, apache, ngrest simple server?

среда, 25 июля 2018 г., 12:13:09 UTC+3 пользователь Kumar написал:

Hi,

I am retrieving data as list of struct from an API, first time getting correct json data, and next execution getting discontinuation in JSON data.

Please find following result.

}, {
        "r_index": "6583",
        "address": "0001.085a.085b",
        "bandwidth": "0 Kbit",
        "description": "** Uplink to OSN **",
        "encapsulation": "ARPA",
        "hardware_type": "Aggregated Ethernet interface(s)",
        "input_rate": "0",
        "interface": "Bundle-Ether85",
        "ip_address": "192.168.172.33/30",
        "link_status": "up",
        "mtu": "1514",
        "name": "nl-ley01a-ra1",
        "output_rate": "0",
        "protocol_status": "up",
        "router": "84.116.246.106",
        "timeupdated": "2018-07-10 05:22:04.690350"
    }, {
        "r_index": "6681",
        "address": "0001.085a.085b",
        "bandwidth": ""
        description ":" ** ls05abr42 VOD STREAMING A - Side ** ","
        encapsulation ":"
        ARPA ","
        hardware_type ":"

And from log I can see the proper result

D/25-07-2018 10:55:07.199 phyinfo.cpp:916 getpeinfo: listObj = 6681, 0001.085a.085b, 1000000 Kbit, ** Uplink to OSN **,ARPA,Aggregated Ethernet interface(s), 0, GigabitEthernet1001/0/0/29, 192.168.172.33/30, administratively, 1514,nl-ley01a-ra1, 0, administratively down, 84.116.246.106, 2018-07-10 05:22:04.700110

Suspecting that, this error could be in mapping from struct to JSON object.

I am not able to find solution for this. Could you please help me on identifying and fixing this issue.

Regards,

Santhosh

[Kumar]

Hi,

React js as HTTP client. web server nginx.

Thanks.

[Kumar]

And we observed and noticed this issue could be in recyclerBody() api.

Symptom:
=======
http response body gets truncated when all of the following situations were met:
1. When the response body size exceeds 64k
2. 2nd or further request done from the same client after "service restart nginx"

log snippet from ngrest service:
==============================

==
^[[0mD/25-07-2018 11:04:32.399 ServiceDispatcher.cpp:360 dispatchMessage: Dispatching message /favicon.ico^[[0m
^[[0mD/25-07-2018 11:04:32.399 ServiceDispatcher.cpp:380 dispatchMessage: Invoking service operation ngrest.Favicon/favicon^[[0m
^[[0mD/25-07-2018 11:04:32.399 ClientHandler.cpp:720 writeNextPart: Request 4 handled in 201 microsecond(s)^[[0m
^[[0mD/25-07-2018 11:05:04.820 ClientHandler.cpp:224 disconnected: client #6 disconnected^[[0m
^[[0mD/25-07-2018 11:05:04.972 ClientHandler.cpp:200 connected: Accepted connection on client #6 (host=xxx.xx.xx.xx, port=56672)^[[0m
^[[0mD/25-07-2018 11:05:04.973 ServiceDispatcher.cpp:360 dispatchMessage: Dispatching message /api/cassandraservices/

Whenever client got disconnect and a new connection is made, the response body is truncated.

Workaround:
==========
Commenting the following line in the file has solved the problem for now. But we want to understand, why this happening with PoolRecycler enabled and what the performance impact of commenting these lines?

file:modules/share/ngrest_server.cpp
int ngrest_server_dispatch(ngrest_server_request* request, ngrest_mod_callbacks callbacks)
{
    try {
        ngrest::MessageContext context;

        context.pool = pooler.obtain();
        //PoolRecycler recyclerContext(context.pool);

        context.engine = &engine;
        context.transport = &transport;
        context.request = context.pool->alloc<ngrest::HttpRequest>();
        context.response = context.pool->alloc<ngrest::HttpResponse>();
        context.response->poolBody = pooler.obtain(65536); // 64 KB chunks for output buffer
        //PoolRecycler recyclerBody(context.response->poolBody);
    .
    .
    .
}

Deleted some content due to security and confidential info.

Thanks.

On Thursday, July 26, 2018 at 10:28:09 AM UTC+5:30, Kumar wrote:

Hi,

React js as HTTP client. web server nginx.

Thanks.

On Wednesday, July 25, 2018 at 9:38:50 PM UTC+5:30, Dmitry Utkin wrote:

Which HTTP client do you use? What is on the server side, nginx, apache, ngrest simple server?

среда, 25 июля 2018 г., 12:13:09 UTC+3 пользователь Kumar написал:

Hi,

I am retrieving data as list of struct from an API, first time getting correct json data, and next execution getting discontinuation in JSON data.

Please find following result.

 {
        "r_index": "6681",
        "address": "0001.085a.085b",
        "bandwidth": ""
        description ":" ** ls05abr42 STREAMING A - Side ** ","
        encapsulation ":"
        ARPA ","
        hardware_type ":"

[Kumar]

Deleted few content.

Thanks.

On Wednesday, July 25, 2018 at 2:43:09 PM UTC+5:30, Kumar wrote:

Hi,

I am retrieving data as list of struct from an API, first time getting correct json data, and next execution getting discontinuation in JSON data.

Please find following result.

, {

        "r_index": "6681",
        "address": "0001.085a.085b",
        "bandwidth": ""
        description ":" ** ls05abr42 STREAMING A - Side ** ","
        encapsulation ":"
        ARPA ","
        hardware_type ":"

[Dmitry Utkin]

this may be related to bug #58: https://github.com/loentar/ngrest/issues/58

Please try the merge request and see if it works.

четверг, 26 июля 2018 г., 11:49:36 UTC+3 пользователь Kumar написал:

[Dmitry Utkin]

I believe that commenting out that two lines may lead to memory leaks

четверг, 26 июля 2018 г., 11:46:06 UTC+3 пользователь Kumar написал:

[Kumar]

Hi,

Merged the changes (commit 0623ba3178df01af00d278ed7b6aa93e6a2dda6c) and tested, Working fine. Thanks for quick response.

And uncommented the recycler* and tested as well.

Thanks.

[Dmitry Utkin]

Cool, that merge request is now in ngrest master. So you can revert all changes and update ngrest.

вторник, 31 июля 2018 г., 12:02:13 UTC+3 пользователь Kumar написал: