Native messaging host gets EOF on stdin prematurely?

[Drew]

As necessitated by the demise of NPAPI, we have replaced our NPAPI component with a native messaging host.

It's seemingly working well for the vast majority of our users.  However, there is a small subset of users that are seemingly seeing the native messaging host receive an EOF on stdin, causing it to terminate prematurely.  Our browser extension is certainly never asking the native messaging host to disconnect, so the only cause I can think of is that Chrome itself is disconnecting the native messaging host for some reason.

The only way I've been able to reproduce this so far is to send a >1MB message from the host to the extension, which is apparently some kind of limit that isn't documented anywhere (the limit doesn't appear to exist in the other direction).  However, I've resolved that by chunking the messages to that they're never >1MB, and the problem seems to remain for some users.

So, I'm wondering if anyone else has seen any similar behavior, or has any ideas about debugging it further.  Thanks.

[Mark Wagner]

The 1MB limitation in host -> browser communication is to protect the browser from runaway host programs, and is documented in the browser source code (but apparently nowhere else).  There is no limit for browser -> host communication (beyond the 4GB limitation imposed by the 32-bit size of the "message length" field).

If you're having trouble, make sure you're counting the bytes correctly: you need to account for the JSON wrapping of chunks, and if you're sending binary data, binary->UTF-8 conversion may be causing the data to expand.  If all else fails, reduce your maximum chunk size so you're not even coming close to the limit (I used a limit of 750,000 bytes before encoding and wrapping).

--
Mark

[Drew]

Thanks for the response.  The 1MB limit should probably be documented on:

http://developer.chrome.com/extensions/messaging.html

I had to figure it out via trial and error.

I don't think the 1MB limit is coming into play for me any more...  I'm chunking 1,000,000 bytes at a time now, and it's already base64-encoded so there should be no issues with it growing due to encoding.  The remaining 48,576 should be way more than enough for JSON wrapping.  I also have log files from users experiencing this issue that show no more than 33,698 bytes being sent back to the extension.

Perhaps you could point me to the Chrome code that handles receiving messages from the host?  I'm wondering if there are any other cases I might dig up that would cause it to disconnect.

[Drew]

I've goitten a little further with this.  After onDisconnect() was called, I checked the value of the user's chrome.extension.lastError, and it was "Error when communicating with the native messaging host.".  I checked the Chrome source for this error message, and it corresponds to the constant kHostInputOuputError.  That constant is used in 3 places:

    LOG(ERROR) << "Error when reading from Native Messaging host: " << result;
    Close(kHostInputOuputError);

      LOG(ERROR) << "Native Messaging host tried sending a message that is "
                 << message_size << " bytes long.";
      Close(kHostInputOuputError);

      LOG(ERROR) << "Error when writing to Native Messaging host: " << result;
      Close(kHostInputOuputError);

Is there any way for me to see these LOG(ERROR) lines, to determine which it is?

[István Lakatos]

I too ran into this issue, Drew, however in my case Chrome seemed to close down my port when I tried to send messages over 8k in size, not 1MB. Luckily the messages I sent were few and far between, so I ended up breaking up my messages in 512 byte chunks to avoid any possible problems regarding message length.

You can find some info here on how to enable logging http://www.chromium.org/for-testers/enable-logging
There is also the https://code.google.com/p/sawbuck/ tool which shows you these logs.

My debugging attempts were unfruitful, though. Maybe you'll be more successful. I have prepared a build of Chrome, though, which I plan to debug to see exactly what is happening in the next couple of days.

[Drew]

Thanks for the logging tip...  I got the log message from the user:

[13592:8128:0221/190514:ERROR:native_message_process_host.cc(325)] Error when writing to Native Messaging host: -101

So, it doesn't seem to be an issue with me writing too large a message.  Something horrible is seemingly happening between Chrome and the native messaging host.  On the host side, my logs indicate that I'm receiving an EOF on stdin, so I'm exiting.

Does anyone know what the -101 error code might mean, or have any ideas about debugging this further?  I don't know the Chrome source code very well.

I'm tempted to simply fire up the native messaging host again if it exits like this.  Not sure if that's a good idea, though.

[Drew]

I've stumbled upon at least one cause of this problem...  Windows opens stdout in text mode by default.  This means any time a \n (ASCII code 10) is outputted, an \r (ASCII code 13) will be outputted directly before it.

I ensure my JSON data contains no newlines, so the only place it was possible for that to happen was in the length field.  Any time a message with a length whose binary representation included a 10 (for example, a 27146-byte message has a binary representation of 10,106,0,0) caused the problem to occur.

My solution is to change stdout's mode to binary:

setmode(fileno(stdout), O_BINARY);

Hopefully this helps someone else out there; I know it flummoxed me for a while.

[Mark Wagner]

On Thursday, March 6, 2014 11:49:43 AM UTC-8, Drew wrote:

I've stumbled upon at least one cause of this problem...  Windows opens stdout in text mode by default.  This means any time a \n (ASCII code 10) is outputted, an \r (ASCII code 13) will be outputted directly before it.

I ensure my JSON data contains no newlines, so the only place it was possible for that to happen was in the length field.  Any time a message with a length whose binary representation included a 10 (for example, a 27146-byte message has a binary representation of 10,106,0,0) caused the problem to occur.

My solution is to change stdout's mode to binary:

setmode(fileno(stdout), O_BINARY);

I think the reason I haven't seen that issue is that my code uses the Windows ReadFile()/WriteFile() APIs, which don't do newline conversion -- they're strictly binary IO functions.

--
Mark

[Drew]

Sounds likely...  I'm using fread()/fwrite(), which do.

I should also mention that I had to set stdin to binary as well:

setmode(fileno(stdout), O_BINARY);

lest byte pairs 13,10 would be converted to 10.

[Drew]

Oops... I meant:

setmode(fileno(stdin), O_BINARY);

of course.

[Vidya Pawar]

Hi all,

I am using the native-host in C++, code below got from this link

http://stackoverflow.com/questions/20220668/communication-between-native-app-and-chrome-extension/20235961#20235961

The issue with this code is it is not sending data above 250 bytes. 

Ideally it should send 1 MB data to and from to native-host.

Can anyone tell me what might be causing this issue?

int main(int argc, char* argv[])

{

    std::cout.setf( std::ios_base::unitbuf ); //instead of "<< eof" and "flushall"

    unsigned int a, c, i, t=0;

    std::string inp;  

    bool bCommunicationEnds = false;

    

    do {

        inp="";

        t=0;

        // Sum the first 4 chars from stdin (the length of the message passed).

        for (i = 0; i <= 3; i++) 

        {

//          t += getchar();

            t += std::cin.get(); 

        }

        // Loop getchar to pull in the message until we reach the total

        //  length provided.

        for (i=0; i < t; i++) 

        {

            //c = getchar();

            c = std::cin.get(); 

            //if(c == EOF)

            if(c == 65)

            {

                bCommunicationEnds = true;

                i = t;

            }

            else

            {

                inp += c;

            }

        }

        if(!bCommunicationEnds)

        {

            //Collect the length of the message

             unsigned int len = inp.length();

            //unsigned int len = strJson.length();

            //// We need to send the 4 btyes of length information

            std::cout << char(((len>>0) & 0xFF))

                << char(((len>>8) & 0xFF))

                << char(((len>>16) & 0xFF))

                << char(((len>>24) & 0xFF));

            //// Now we can output our message

            std::cout << inp;

            

        }

    }while(!bCommunicationEnds);

    return 0;

}

Thanks and Regards,

-Vidya

[Drew]

That code is buggy.  See bkdc's answer at the bottom of your link.

[Don Schmitt]

Hi Drew,

Thanks very much for following up.  I thought I'd share an additional gotcha with text mode:  The CRT will also stop reading in text mode when it encounters a ctrl-Z (0x1A).  This won't occur in your JSON text, but if you send a JSON string that has a 1A as one of the bytes in the length (e.g. a 26-character length JSON message), your NativeHost will not receive any more input.

GetStdHandle / ReadFile took care of this problem.

Thanks!

 - Don

--
You received this message because you are subscribed to the Google Groups "Chromium-extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To post to this group, send email to chromium-...@chromium.org.
Visit this group at http://groups.google.com/a/chromium.org/group/chromium-extensions/.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/1fd58d71-80df-4858-b3d3-47ffd666304a%40chromium.org.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.

[Udit Mehta]

Hi Don,

I am trying to do exactly as you're saying and am unable to do so.

I wish to be able to close the NativeHost through chrome in JavaScript.

What should I send it through port.postMessage(  "??"  )  ?

Thanks,

Udit

On Wednesday, April 2, 2014 at 6:20:08 PM UTC+5:30, donaddon wrote:

Hi Drew,

Thanks very much for following up.  I thought I'd share an additional gotcha with text mode:  The CRT will also stop reading in text mode when it encounters a ctrl-Z (0x1A).  This won't occur in your JSON text, but if you send a JSON string that has a 1A as one of the bytes in the length (e.g. a 26-character length JSON message), your NativeHost will not receive any more input.

GetStdHandle / ReadFile took care of this problem.

Thanks!

 - Don

On Fri, Mar 7, 2014 at 9:27 AM, Drew <zit...@gmail.com> wrote:

Sounds likely...  I'm using fread()/fwrite(), which do.

I should also mention that I had to set stdin to binary as well:

setmode(fileno(stdout), O_BINARY);

lest byte pairs 13,10 would be converted to 10.

--
You received this message because you are subscribed to the Google Groups "Chromium-extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To post to this group, send email to chromium-...@chromium.org.
Visit this group at http://groups.google.com/a/chromium.org/group/chromium-extensions/.