How to write pipe message bigger than buffer size?

[nickdu]

I've created an IPC mechanism using named pipes. I'm using message type and
message read mode. There is support for reading messages larger than the
buffer size since GetOverlappedResult() can return ERROR_MORE_DATA. However
I can't seem to find a way, out of the box, to write a message larger than
the buffer?

For each pipe instance I create a little chunk of memory to hold state
information (like most of the samples). One piece of information is a
pointer to the buffer for reading and writing. The size of this buffer is
equal to the buffer size of the pipe, which is specified via configuration.
If the buffer is 8K is there a way to send a 16K message without:

1. Allocating a new 16K buffer whose lifetime I now have to manage.

or

2. Building a chunking mechanism into my server and client code.

I was hoping there was some way for me to use the 8K buffer and do two
writes and somehow indicate in the first write that it's only part of the
message.
--
Thanks,
Nick

nickno...@community.nospam
remove "nospam" change community. to msn.com

[Gary Chanson]

"nickdu" <nickno...@community.nospam> wrote in message
news:E55D8762-A6AF-4684...@microsoft.com...

Build your messages so that your message handler can recognize that a
message is linked to the next message. Maybe set a flag in a status word or
something like that.

--

- Gary Chanson
- Abolish Public Schools

[nickdu]

I'm still trying to come up with a simple solution for this. I really don't
want to have to inject some header information into the message. What about
assuming that if the bytes read is less than the BufferSize then I'm done
reading? If you happen to have a message to send which is exactly the
BufferSize then you need to do an additional write of zero bytes. Would this
work? Can I rely on the OS to always write at least BufferSize bytes to the
pipe, assuming you're trying to write BufferSize or greater number of bytes,
if the pipe is currently empty? I assume it's dependent on my server reading
the data out of the pipe. My server always reads BufferSize bytes at a time.
Can I count of the OS to always read BufferSize bytes if there are that many
bytes available? I guess what I'm getting at is this:

If there are BufferSize bytes in the pipe and my server requests to read
BufferSize bytes is there any chance of the OS returning less than BufferSize
bytes?

By the way, BufferSize is the size of the buffer when the pipe was created.
--
Thanks,
Nick

nickno...@community.nospam
remove "nospam" change community. to msn.com

[Jeffrey Tan[MSFT]]

Hi Nick,

Thank you for the feedback.

I did not use Named Pipe for a long time. If I remember correct, the size
of the message is specified in the WriteFile() API and received in
ReadFile() by nNumberOfBytesToRead parameter. So, if you always use the
same buffer size in both WriteFile() and ReadFile() API, you should not see
ERROR_MORE_DATA, yes?

If the logic message in your application is larger than the predefined
WriteFile() buffer size, I think it is your responsibility to divide the
logic message into several pieces round at buffer size boundary. In this
situation, using a header to indicate the entire message length is the most
common and easy solution. For example, TCP protocol uses the similar header
to deal with this type of issue.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
msd...@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[nickdu]

I don't want to have to put message sizes in the header. I think I can avoid
that if I can be sure that the OS will *always* read n bytes if there are n
bytes in the buffer and I request to read those n bytes. If there is any
chance of the OS returning some lesser number of bytes then my logic won't
work. Same thing with write, but they are somewhat tied together for named
pipes as a write will be dependent on having room in the pipe which means
previous data written will have to be read.

So for example let's say I create a two way named pipe with a buffer size of
4K. My server then issues a request to read 4K. The client makes a request
to write 4K into that buffer. Is there any chance that OS will write the 4K
in smaller chunks? Could the OS potentially write the 4K with 2 2K writes
and thus potentially cause my server to read 2K initially?

This may be dependent on the atomicity (is that a word?) of named pipes? If
one side is writing into a pipe does it block the reading side until the
write has written bytes = min(bytes requested, free bytes in buffer)? If one
side is reading does it block the writer until the read has read bytes =
min(bytes requested, bytes of data in buffer)?
--
Thanks,
Nick

nickno...@community.nospam
remove "nospam" change community. to msn.com

[m]

look at CreateNamedPipe & PIPE_TYPE_MESSAGE / PIPE_READMODE_MESSAGE in MSDN

"nickdu" <nickno...@community.nospam> wrote in message

news:E2BB0E2F-59B4-4108...@microsoft.com...

[Jeffrey Tan[MSFT]]

Hi Nick,

Thank you for the feedback.

When you are using the message-type pipe and reading mode, the system
treats the bytes written in each write operation to the pipe as a message
unit, which means that the system will ensure the atomic operation. See the
link below:
"Named Pipe Type, Read, and Wait Modes"
http://msdn.microsoft.com/en-us/library/aa365605(VS.85).aspx

[Ben Voigt [C++ MVP]]

"Jeffrey Tan[MSFT]" wrote:
> Hi Nick,
>
> Thank you for the feedback.
>
> When you are using the message-type pipe and reading mode, the system
> treats the bytes written in each write operation to the pipe as a
> message unit, which means that the system will ensure the atomic
> operation. See the link below:
> "Named Pipe Type, Read, and Wait Modes"
> http://msdn.microsoft.com/en-us/library/aa365605(VS.85).aspx

Yes but do message-mode pipes guarantee reliable in-order delivery? I know
that's one of the differences between stream and message sockets, and that
MSDN page didn't seem to say.

[m]

AFAIK yes - reliable delivery is implied by the API

IIRC pipes interact with the network redirector to transmitt their data
(messages or streams) over TCP (assuming that NetBios over TCP is enabled)
and this allows the server side to control security easily (client
impersonation & ACL)

"Ben Voigt [C++ MVP]" <r...@nospam.nospam> wrote in message
news:eGw$HKwMJ...@TK2MSFTNGP06.phx.gbl...