slow POST reads via mg_read (vs GGI)

[kb1ooo]

I can't figure out why fetching POST data using the embedded API is so slow for me.  I know it's not a mongoose problem because if I receive the data via CGI--using mongoose as my web server--I get expect request times.   E.g.  using curl to send a 77k image

time curl -X POST --data-binary @example.jpg http://localhost:9876/testPost

gives request/response times of 1s to 2s.   Whereas the same request via a C++ CGI is 20ms.   Any help greatly appreciated.  Below is my code using the embedded API.

#include "mongoose.h"
#include <string>
#include <vector>

// g++ -o testPost main.cpp mongoose.c
// ./testPost
//
// time curl -X POST --data-binary @example.jpg http://localhost:9876/testPost

static const char* okReply = "HTTP/1.1 200 OK\r\n"
"Content-Type: text/plain\r\n"
"Connection: close\r\n\r\n";
static const char* badRequestReply = "HTTP/1.1 400 Bad Request\r\n"
"Content-Type: text/plain\r\n"
"Connection: close\r\n\r\n";

using namespace std;

static void* callback(enum mg_event event,
                      struct mg_connection *conn,
                      const struct mg_request_info *request_info)
{
    string uri(request_info->uri);
    string requestMethod(request_info->request_method);
    static char processed[] = "processed";
    if((event == MG_NEW_REQUEST) && (uri == "/testPost") && (requestMethod == "POST"))
    {
        const char* contentLength = mg_get_header(conn, "Content-Length");
        size_t bufSize;

        if(contentLength != 0)
            bufSize = atoi(contentLength);
        else
        {
            mg_printf(conn, "%s", badRequestReply);
            mg_printf(conn, "Error: no content\n");
            return processed;
        }

        vector<char> buf(bufSize);
        mg_read(conn, &buf[0], bufSize);
        mg_printf(conn, "%s", okReply);

        return processed;
    }
    else
        return 0;
}

int main(int argc, char* argv[])
{
    struct mg_context *ctx;
    const char *options[] = {"listening_ports", "9876", NULL};

    ctx = mg_start(callback, 0, options);
    pause();
    return 0;
}

[kb1ooo]

Forgot a couple relevant facts.  I'm using version 3.1 on OSX 10.7.4.

Thanks,
Marc

[Sergey Lyubka]

To debug it, put printfs with timestamps before and after mg_read(). Also show number of bytes supposed to read and actually read.

--
You received this message because you are subscribed to the Google Groups "mongoose-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/mongoose-users/-/Cw01A9sHhfsJ.

To post to this group, send email to mongoos...@googlegroups.com.
To unsubscribe from this group, send email to mongoose-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongoose-users?hl=en.

[kb1ooo]

Sergey,

Here is the time (using OSX highres mach_time.h) and #bytes for the mg_read call which is correct (also checked validity of data).   I've tried both 3.1 and 3.2 versions of mongoose, same result. 

content length = 77214
bytes read = 77214
time is : 1001.25ms

Also, since I noticed proper times when receiving the POST data via CGI, I tried writing an alternate mg_read function that uses forward_body_data (i.e. the function that is used to push request body data to CGI).    See code below.  Replacing the mg_read function in my code with this new mg_read_all function gives me reasonable read times, and I checked validity of read data.

time is: .21ms


int static writefn(void* cookie, const char* buf, int len)
{

    char** bigBufPtr = ((char**) cookie);
   
    memcpy(*bigBufPtr, buf, len);
    *bigBufPtr += len;
    return len;
}

int mg_read_all(struct mg_connection *conn, void* buf)
{
    FILE* fp = fwopen(&buf, writefn);
    return forward_body_data(conn, fp, INVALID_SOCKET, NULL);

[Sergey Lyubka]

I compiled this on my Mac and see the same thing ..

Then I looked at what curl is doing. You can do it too, start netcat and see what curl is pushing to it:

$ nc -l 9876

$ time curl -X POST --data-binary @mongoose.c http://localhost:9876/testPost

Netcat's output is as follows:

$ nc -l 9876

POST /testPost HTTP/1.1

User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3

Host: localhost:9876

Accept: */*

Content-Length: 1062

Content-Type: application/x-www-form-urlencoded

Expect: 100-continue

<HERE IS THE DELAY YOU'RE SEEING, IT IS CURL WAITING BEFORE SENDING DATA>

Copyright (c) 2004-2010 Sergey Lyubka

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN

THE SOFTWARE.

Not sure why CGI is fast. 

I've added a line before the mg_read() call, 

mg_printf(conn, "%s", "HTTP/1.1 100 Continue\r\n\r\n");

This should be done if client sends Expect: 100-continue.

Hope that helps.

To view this discussion on the web visit https://groups.google.com/d/msg/mongoose-users/-/bbxDw5ih9a0J.

[Sergey Lyubka]

Note that forward_body_data() sends 100-continue, so that's why it is fast.

[kb1ooo]

Thanks Sergey,

I added the mg_printf and get proper response times, thank you.

(As an aside to anyone who cares:), my fwopen example is missing an fclose(fp) without which the remaining bytes in the final buffer will not be written)

Marc