How to set stdin/stdout/stderr to binary mode
KO Myung-Hun
Generally, when porting *nix programs, we should set stdin/stdout/stderr
to binary mode unconditionally. Consequently, we have a CR/LF problem.
So I suggest an other approach.
1. binary mode for stdin/stdout/stderr is meaningful only when they are
piped or redirected.
2. If they are default devices, that is, character devices, we don't
have to set them to binary mode.
3. So we should set them to binary mode, only unless they are character
devices.
4. Therefore, the CR/LF problem can be solved.
The following is the code snippet which I used when to port x264.
-----
#include <io.h> // setmode()
#include <fcntl.h> // O_BINARY
#include <stdio.h> // FILE, fileno()
#include <sys/stat.h> // struct stat, fstat(), S_ISCHR()
static inline void set_bin_mode(FILE *stream)
{
struct stat st;
fstat(fileno(stream), &st);
if(!S_ISCHR(st.st_mode))
setmode(fileno(stream), O_BINARY);
}
-----
Any comments ?
--
KO Myung-Hun
Using Mozilla SeaMonkey 1.1.14
Under OS/2 Warp 4 for Korean with FixPak #15
On AMD ThunderBird 1 GHz with 512 MB RAM
Korean OS/2 User Community : http://www.ecomstation.co.kr
Ilya Zakharevich
Sometimes I use a script which redirects (some of) STDIN/STDOUT/STDERR
of a program to a different VIO window. Then these FH are open to pipes,
but the result should be in text mode.
This might be corrected in a redirector, but then when I start a
redirector I must KNOW in advance that the program in question is morphing
the I/O basing on a handle type....
Yours,
Ilya
KO Myung-Hun
That's no problem. TEXT/BINARY is a attribute of a program level not a
system level. That is, a program always open stdin/stdout/stderr as a
text mode. So if one program set stdin/stdout/stderr to binary mode and
redirected them to other program, then the other program read it as text
mode by default.
As you know, TEXT and BINARY are an method to translate CR/LF and LF. If
you read stdin, CR/LF is translated to LF, and if you write
stdout/stderr, LF is translated to CR/LF.
So one program write LF to stdout on binary mode, and the other program
read it from stdin on text mode, then it will be LF as if it was CR/LF.
Therefore your apprehension is unnecessary.
Ilya Zakharevich
>>> fstat(fileno(stream), &st);
>>>
>>> if(!S_ISCHR(st.st_mode))
>>> setmode(fileno(stream), O_BINARY);
>>> }
> That's no problem. TEXT/BINARY is a attribute of a program level not a
> system level. That is, a program always open stdin/stdout/stderr as a
> text mode.
Sorry, I have no clue what you are talking about. In the code above,
the mode depends on the type of file handle.
>> Sometimes I use a script which redirects (some of) STDIN/STDOUT/STDERR
>> of a program to a different VIO window. Then these FH are open to pipes,
>> but the result should be in text mode.
>> This might be corrected in a redirector, but then when I start a
>> redirector I must KNOW in advance that the program in question is morphing
>> the I/O basing on a handle type....
In fact, no fancy redirecting is needed to cause havoc. I (almost
always) redirect output of non-interactive text-mode programs via
foo bar |& tee 00log-foo-bar
for future reference. Switching stdout of foo to binmode if it is a
pipe would make the output almost unreadable...
> So one program write LF to stdout on binary mode, and the other program
> read it from stdin on text mode, then it will be LF as if it was CR/LF.
Not applicable. Redirectors (like tee) must work in binary mode.
(E.g., otherwise you won't be able to notice bin-mode-related bugs in
the logs.)
Yours,
Ilya
KO Myung-Hun
Ilya Zakharevich wrote:
> On 2009-02-13, KO Myung-Hun <ko...@chollian.net> wrote:
>
>>>> fstat(fileno(stream), &st);
>>>>
>>>> if(!S_ISCHR(st.st_mode))
>>>> setmode(fileno(stream), O_BINARY);
>>>> }
>>>>
>
>
>> That's no problem. TEXT/BINARY is a attribute of a program level not a
>> system level. That is, a program always open stdin/stdout/stderr as a
>> text mode.
>>
>
> Sorry, I have no clue what you are talking about. In the code above,
> the mode depends on the type of file handle.
>
>
Hmm... We don't understand each other.
>>> Sometimes I use a script which redirects (some of) STDIN/STDOUT/STDERR
>>> of a program to a different VIO window. Then these FH are open to pipes,
>>> but the result should be in text mode.
>>>
>
>
Please explain you are saying what the matter is here, in detail. It
would be good if you provide examples.
>>> This might be corrected in a redirector, but then when I start a
>>> redirector I must KNOW in advance that the program in question is morphing
>>> the I/O basing on a handle type....
>>>
>
> In fact, no fancy redirecting is needed to cause havoc. I (almost
> always) redirect output of non-interactive text-mode programs via
>
> foo bar |& tee 00log-foo-bar
>
> for future reference. Switching stdout of foo to binmode if it is a
> pipe would make the output almost unreadable...
>
Why do you think so ? Did you test it and get the result what you expect ?
>
>> So one program write LF to stdout on binary mode, and the other program
>> read it from stdin on text mode, then it will be LF as if it was CR/LF.
>>
>
> Not applicable. Redirectors (like tee) must work in binary mode.
> (E.g., otherwise you won't be able to notice bin-mode-related bugs in
> the logs.)
>
I don't understand what you are saying.
What's the 'bin-mode-related bugs in the logs' ?
KO Myung-Hun
We can use 'isatty()' instead of fstat().
-----
#include <io.h> // isatty(), setmode()
#include <fcntl.h> // O_BINARY
#include <stdio.h> // FILE, fileno()
static inline void set_bin_mode(FILE *stream)
{
if(!isatty(fileno(stream)))
setmode(fileno(stream), O_BINARY);
}
-----
Ilya Zakharevich
>> Generally, when porting *nix programs, we should set stdin/stdout/stderr
>> to binary mode unconditionally. Consequently, we have a CR/LF problem.
>>
>> So I suggest an other approach.
>>
>> 1. binary mode for stdin/stdout/stderr is meaningful only when they are
>> piped or redirected.
>>
>> 2. If they are default devices, that is, character devices, we don't
>> have to set them to binary mode.
>>
>> 3. So we should set them to binary mode, only unless they are character
>> devices.
>>
>> 4. Therefore, the CR/LF problem can be solved.
> We can use 'isatty()' instead of fstat().
Dear Myung-Hun,
I appreciate very much your contribution to OS/2 landscape; however, I
can't avoid mentioning that oftentimes it's damn hard to understand
what you want to express in your communication.
Let me try to express what *I* know about binmode() problems:
1) To get smooth operation on DOSISH systems, some files must be
opened in text mode, some in binary mode.
2) Some ALREADY OPENED files (usually stdin/stdout) must be changed
to a correct mode (binary, or text).
3) To add insult to injury, some APIs open files behind the curtain
(e.g., mkstemp()). Such files must also be changed to a correct
mode.
Solutions:
A) To resolve "1", one should pepper the source code of the program
by suitable "t" and "b" (for fopen()), and O_BINARY/O_TEXT for
open().
B) To resolve "2", one should use setmode(). Sometimes setmode()
must be made conditional on command-line arguments (e.g., given
'-', a program may send a binary stream to STDOUT; otherwise it
would emit there some messages for the reader).
C) To resolve "3" one should hunt for usage of the mentioned API, and
setmode() appropriately.
[Of course, "B" is compounded by the fact that
B1) messages-for-user should be emited in text mode; otherwise one
gets "a staircase", as in
message 1
message 2
which would eventially start to wrap around, making output hard
to read (or completely unreadable if "\r" is used).
]
=======================================================
The programmers, being virtuous, thus lazy, want a bulk solution, not
a labor-intesive one.
"C" might be, in principle, solved by compiler switches which would,
e.g., change the mode of opened temporary files [*].
"A" may also be automated (to some extend): if all the opened files must
be in the SAME mode, one can use, e.g., -Zbin-files (sp?) instead of
adding "b"/O_BINARY.
=======================================================
What you propose is a kind of lazy-man "solution" to "B": you note
that OFTENTIMES when one needs a text mode, then stdin/stdout/stderr
is opened to a console. And when one needs a binary mode, the
filehandle is NEVER opened to a console.
So automatically switching the mode basing on the console/non-console
LOOKS LIKE being beneficial. In many usage scenarios, it IS beneficial.
----
However, note the important difference with the solution for "A":
-Zbin-files is beneficial only for SOME PROGRAMS. And your solution
is beneficial only for SOME USAGE SCENARIOS.
E.g., as I said, I `tee' the output of most programs which output a lot:
foo 2>&1 | tee 00foo-output
Now `foo' would see that it outputs to a pipe, and would switch to
binary mode; I would see the staircased output (as mentioned above).
Effectively, foo becomes not-tee-able (or at least I need to use a
special flavor of tee which would mutiliate the output of `foo').
---
In short: your solution is better than doing nothing, but much worse
than "proper" setmode()ing of stdout etc basing on investigation of
the program logic.
Hope this helps,
Ilya
=======================================================
[*] BTW, is there any need for text-mode "unnamed" temporary files?
The only way I see where text-mode may be beneficial is when
a) I open a temporary file; then I start a child process;
b) child communicates with me via the inherited file
handle, and reads/writes in text mode.
Does it ever happens?
Lars Erdmann
to define the default opening mode: text or binary ? That would cover 2) and
3)
Lars
"Ilya Zakharevich" <nospam...@ilyaz.org> schrieb im Newsbeitrag
news:slrngvl6eu.9r...@chorin.math.berkeley.edu...
Ilya Zakharevich
> Wasn't there some define in some RTL header file that you can set/override
> to define the default opening mode: text or binary ? That would cover 2) and
> 3)
Sigh...
>> 1) To get smooth operation on DOSISH systems, some files must be
>> opened in text mode, some in binary mode.
>>
>> 2) Some ALREADY OPENED files (usually stdin/stdout) must be changed
>> to a correct mode (binary, or text).
>>
>> 3) To add insult to injury, some APIs open files behind the curtain
>> (e.g., mkstemp()). Such files must also be changed to a correct
>> mode.
>> "A" may also be automated (to some extend): if all the opened files must
>> be in the SAME mode, one can use, e.g., -Zbin-files (sp?) instead of
>> adding "b"/O_BINARY.
["A" was about fixing "1"]
So, as I said, -Zbin-file (which, I expect, is NOT a define - probably
links with a differently initialized variable) would solve "1" (for
some applications), but not "2" and "3".
So AFAIK, the situation is exactly opposite to what you wrote. Do I
miss something?
[Here I trust the docs that mkstemp() opens in text mode - as
opposed to the mode regulated by -Zbin-files. (sp?)]
Yours,
Ilya
Dave Yeo
> Wasn't there some define in some RTL header file that you can set/override
> to define the default opening mode: text or binary ? That would cover 2) and
> 3)
From emx/src/doc/emxlib.src
By default, text mode is used for files. See â– hpt{fread()} and
â– hpt{fwrite()} for details. The default can be globally changed to
binary mode by linking with binmode.o or binmode.obj
(â– hpt{-Zbin-files} option of GCC).
This from the klibc sources so I'd guess it would work with either
Dave
Dave Yeo
> On 2009-05-01, Lars Erdmann<lars.e...@arcor.de> wrote:
>> Wasn't there some define in some RTL header file that you can set/override
>> to define the default opening mode: text or binary ? That would cover 2) and
>> 3)
>
> Sigh...
>
>>> 1) To get smooth operation on DOSISH systems, some files must be
>>> opened in text mode, some in binary mode.
>>>
>>> 2) Some ALREADY OPENED files (usually stdin/stdout) must be changed
>>> to a correct mode (binary, or text).
>>>
>>> 3) To add insult to injury, some APIs open files behind the curtain
>>> (e.g., mkstemp()). Such files must also be changed to a correct
>>> mode.
>
>>> "A" may also be automated (to some extend): if all the opened files must
>>> be in the SAME mode, one can use, e.g., -Zbin-files (sp?) instead of
>>> adding "b"/O_BINARY.
>
> ["A" was about fixing "1"]
>
> So, as I said, -Zbin-file (which, I expect, is NOT a define - probably
> links with a differently initialized variable) would solve "1" (for
> some applications), but not "2" and "3".
I believe that -Zbin-files is equivalent to setmode(foo,O_BINARY) and
should work for 1 and 3. Knut also tried to make it work for streams
like stdout and stderr but it is buggy.
For EMX you need to use _fsetmode for streams whereas for klibc setmode
should be good enough.
>
> So AFAIK, the situation is exactly opposite to what you wrote. Do I
> miss something?
>
> [Here I trust the docs that mkstemp() opens in text mode - as
> opposed to the mode regulated by -Zbin-files. (sp?)]
-Zbin-files should force binary mode with mkstemp(), I haven't tested
though.
>
> Yours,
> Ilya
Usually when I try to compile something I grep for setmode and WIN32 as
most things have already been ported to Win32. Then it is usually just a
matter of expanding a define or adding a simpler one which usually
works. Occasionally I find a well written program where configure
actually tests for setmode() in io.h and things just work. (BSD also has
a setmode() that is totally different, only DOSSISH systems have an io.h)
Note that Windows and the toolkit also use _setmode()and setmode() is
listed as depreciated in the mingw headers.
Dave
Dave Yeo
FYI this is binmode.s
/ binmode.s (emx+gcc) -- Copyright (c) 1992-1994 by Eberhard Mattes
.text
L_setbinmode:
movl $1, __fmode_bin
ret
.stabs "___CTOR_LIST__", 23, 0, 0, L_setbinmode
Dave
KO Myung-Hun
A and C are irrelevant to this thread, so I don't care.
[snip]
> What you propose is a kind of lazy-man "solution" to "B": you note
> that OFTENTIMES when one needs a text mode, then stdin/stdout/stderr
> is opened to a console. And when one needs a binary mode, the
> filehandle is NEVER opened to a console.
>
> So automatically switching the mode basing on the console/non-console
> LOOKS LIKE being beneficial. In many usage scenarios, it IS beneficial.
>
> ----
>
> However, note the important difference with the solution for "A":
> -Zbin-files is beneficial only for SOME PROGRAMS. And your solution
> is beneficial only for SOME USAGE SCENARIOS.
>
> E.g., as I said, I `tee' the output of most programs which output a lot:
>
> foo 2>&1 | tee 00foo-output
>
> Now `foo' would see that it outputs to a pipe, and would switch to
> binary mode; I would see the staircased output (as mentioned above).
>
If you see the staircased output, it's due to 'tee' operation. I.E.
'tee' opened stdout and/or stdout as binary mode. Because even though
'foo' open its stdout and/or stderr as binarymode, 'tee' is not affected
by it. I.E. initial mode of stdin/stdout/stderr of 'tee' is text mode no
matter 'foo' opened them as whatever. If you see the log of 'tee' using
binary editor, you can see whether the line terminator is CR/LF or LF.
If you just 'LF', 'tee' store the output in binary mode.
For reference, text mode means that CR/LF is converted to LF when
reading from stdin, and LF is converted to CR/LF when writing to stdout
and/or stderr. But in binary mode, anything are not converted.
Anyway if you provide me with screen shot, it's better. And tell me
about 'tee' you are using.
Becuase 4OS2 has a internal command of 'tee', and there is GNU version
of 'tee'.
> Effectively, foo becomes not-tee-able (or at least I need to use a
> special flavor of tee which would mutiliate the output of `foo').
>
> ---
> In short: your solution is better than doing nothing, but much worse
> than "proper" setmode()ing of stdout etc basing on investigation of
> the program logic.
>
>
Of course. The customized thing should be better than the generalized
one. If not so, we don't need to know 'customization'.
[snip]
> [*] BTW, is there any need for text-mode "unnamed" temporary files?
> The only way I see where text-mode may be beneficial is when
>
> a) I open a temporary file; then I start a child process;
>
> b) child communicates with me via the inherited file
> handle, and reads/writes in text mode.
>
> Does it ever happens?
>
As I said before, 'text mode' and 'binary mode' is specific to the
current process. As well as, they are not inherited.
In other words, 'text mode' and 'binary mode' is implemented logically
in libc level for a process. If you use DosRead()/DosWrite(), they also
use so-called 'binary mode'.
--
KO Myung-Hun
Using Mozilla SeaMonkey 1.1.16
Ilya Zakharevich
>> What you propose is a kind of lazy-man "solution" to "B": you note
>> that OFTENTIMES when one needs a text mode, then stdin/stdout/stderr
>> is opened to a console. And when one needs a binary mode, the
>> filehandle is NEVER opened to a console.
>>
>> So automatically switching the mode basing on the console/non-console
>> LOOKS LIKE being beneficial. In many usage scenarios, it IS beneficial.
>>
>> ----
>>
>> However, note the important difference with the solution for "A":
>> -Zbin-files is beneficial only for SOME PROGRAMS. And your solution
>> is beneficial only for SOME USAGE SCENARIOS.
>>
>> E.g., as I said, I `tee' the output of most programs which output a lot:
>>
>> foo 2>&1 | tee 00foo-output
>>
>> Now `foo' would see that it outputs to a pipe, and would switch to
>> binary mode; I would see the staircased output (as mentioned above).
>>
>
> If you see the staircased output, it's due to 'tee' operation.
No. `tee' does nothing to the output. It is due to the program
outputing in binary mode, when the intent is to output the text.
> I.E. 'tee' opened stdout and/or stdout as binary mode.
As far as BOTH stdout/stdin are binary, `tee' CANNOT be at fault. It
just passes stuff around.
> Because even though
> 'foo' open its stdout and/or stderr as binarymode, 'tee' is not affected
> by it. I.E. initial mode of stdin/stdout/stderr of 'tee' is text mode no
> matter 'foo' opened them as whatever.
Wrong. The initial mode of tee is binary, both on stdin, and stdout.
> Anyway if you provide me with screen shot, it's better.
I have no idea which screenshot you are talking about here...
> And tell me about 'tee' you are using.
GNU tee, sh-utils 1.12
> Becuase 4OS2 has a internal command of 'tee', and there is GNU version
> of 'tee'.
I was suffering 4os2's version for several years, before I found that
disabling it removes all the shortcomings (what I remember now is its
confusion what slash means; I also vaguely recollect that it might
have been causing some unnecessary staircasing - or hiding
staircasing, which is as bad).
> Of course. The customized thing should be better than the generalized
> one. If not so, we don't need to know 'customization'.
To the contrary. A general solution would be much better than a
customized one (as far as it is a SOLUTION). Your recipe is just
CRUTCHES: a smart workaround which works in many situations.
Unfortunately, this does not make it into a "solution"...
>> [*] BTW, is there any need for text-mode "unnamed" temporary files?
>> The only way I see where text-mode may be beneficial is when
>>
>> a) I open a temporary file; then I start a child process;
>>
>> b) child communicates with me via the inherited file
>> handle, and reads/writes in text mode.
>>
>> Does it ever happens?
> As I said before, 'text mode' and 'binary mode' is specific to the
> current process. As well as, they are not inherited.
Correct; note also that this is not only irrelevant, but `the opposite
of being relevant' (is there a term for it? ;-).
IF `binary' state were inherited, there would be no problem with the
above scenario: the child would have the same mode as parent, so the
IPC would not distinguish whether temporary file was opened binary or text.
But as stated, if libc changes the mode of temporary file of mkstemp()
to binary, the communication through the temporary file WOULD be
garbled, since the modes in parent and child would not match anymore.
[When discussing [*], I was thinking about whether it is viable to
change the default mode of the handle returned by mkstemp(). I
would not even try to count hours it took me and Paul to track the
corresponding bug in ImageMagick port...]
Yours,
Ilya
KO Myung-Hun
Ilya Zakharevich wrote:
> On 2009-05-10, KO Myung-Hun <ko...@chollian.net> wrote:
>
>>> What you propose is a kind of lazy-man "solution" to "B": you note
>>> that OFTENTIMES when one needs a text mode, then stdin/stdout/stderr
>>> is opened to a console. And when one needs a binary mode, the
>>> filehandle is NEVER opened to a console.
>>>
>>> So automatically switching the mode basing on the console/non-console
>>> LOOKS LIKE being beneficial. In many usage scenarios, it IS beneficial.
>>>
>>> ----
>>>
>>> However, note the important difference with the solution for "A":
>>> -Zbin-files is beneficial only for SOME PROGRAMS. And your solution
>>> is beneficial only for SOME USAGE SCENARIOS.
>>>
>>> E.g., as I said, I `tee' the output of most programs which output a lot:
>>>
>>> foo 2>&1 | tee 00foo-output
>>>
>>> Now `foo' would see that it outputs to a pipe, and would switch to
>>> binary mode; I would see the staircased output (as mentioned above).
>>>
>>>
>> If you see the staircased output, it's due to 'tee' operation.
>>
>
> No. `tee' does nothing to the output. It is due to the program
> outputing in binary mode, when the intent is to output the text.
>
>
Sure ? You said 'tee' opened stdout as a binary mode. This is the problem.
>> I.E. 'tee' opened stdout and/or stdout as binary mode.
>>
>
> As far as BOTH stdout/stdin are binary, `tee' CANNOT be at fault. It
> just passes stuff around.
>
>
Same above.
>> Because even though
>> 'foo' open its stdout and/or stderr as binarymode, 'tee' is not affected
>> by it. I.E. initial mode of stdin/stdout/stderr of 'tee' is text mode no
>> matter 'foo' opened them as whatever.
>>
>
> Wrong. The initial mode of tee is binary, both on stdin, and stdout.
>
>
Wrong. You should say that 'tee' changes the mode of stdin and stdout
from text to binary UNCONDITIONALLY at startup. And it's the point,
because opening stdin/stdout/stderr as binary mode is incorrect when it
is connected to console. My idea started from here. 'tee' should open
stdout as text mode in this case.
>> Anyway if you provide me with screen shot, it's better.
>>
>
> I have no idea which screenshot you are talking about here...
>
>
For a staircase, and the executable and the source of 'foo'.
>> And tell me about 'tee' you are using.
>>
>
> GNU tee, sh-utils 1.12
>
>
>> Becuase 4OS2 has a internal command of 'tee', and there is GNU version
>> of 'tee'.
>>
>
> I was suffering 4os2's version for several years, before I found that
> disabling it removes all the shortcomings (what I remember now is its
> confusion what slash means; I also vaguely recollect that it might
> have been causing some unnecessary staircasing - or hiding
> staircasing, which is as bad).
>
>
I think, 4OS2's way is right but slash meaning.
>> Of course. The customized thing should be better than the generalized
>> one. If not so, we don't need to know 'customization'.
>>
>
> To the contrary. A general solution would be much better than a
> customized one (as far as it is a SOLUTION). Your recipe is just
> CRUTCHES: a smart workaround which works in many situations.
> Unfortunately, this does not make it into a "solution"...
>
>
I think, it's just a guide-line. So it's not important whether it is a
solution or not. It's enough if it provided a idea to people.
>>> <snip>
>> As I said before, 'text mode' and 'binary mode' is specific to the
>> current process. As well as, they are not inherited.
>>
>
> Correct; note also that this is not only irrelevant, but `the opposite
> of being relevant' (is there a term for it? ;-).
>
>
Hmm...
> IF `binary' state were inherited, there would be no problem with the
> above scenario: the child would have the same mode as parent, so the
> IPC would not distinguish whether temporary file was opened binary or text.
>
> But as stated, if libc changes the mode of temporary file of mkstemp()
> to binary, the communication through the temporary file WOULD be
> garbled, since the modes in parent and child would not match anymore.
>
> [When discussing [*], I was thinking about whether it is viable to
> change the default mode of the handle returned by mkstemp(). I
> would not even try to count hours it took me and Paul to track the
> corresponding bug in ImageMagick port...]
>
>
I think -Zbin-files can be helpful.
Ilya Zakharevich
>> No. `tee' does nothing to the output. It is due to the program
>> outputing in binary mode, when the intent is to output the text.
> Sure ?
Yes.
> You said 'tee' opened stdout as a binary mode.
Yes, AND the input in binary mode. So it does not mangle the data
passing through it.
> This is the problem.
No:
>> As far as BOTH stdout/stdin are binary, `tee' CANNOT be at fault. It
>> just passes stuff around.
>>> Because even though
>>> 'foo' open its stdout and/or stderr as binarymode, 'tee' is not affected
>>> by it. I.E. initial mode of stdin/stdout/stderr of 'tee' is text mode no
>>> matter 'foo' opened them as whatever.
>> Wrong. The initial mode of tee is binary, both on stdin, and stdout.
> Wrong. You should say that 'tee' changes the mode of stdin and stdout
> from text to binary
I should not say this, because it is not correct.
File descriptors on OS/2 are always binary. Think of tee as
essentially doing
while (DosRead(...))
DosWrite()
> UNCONDITIONALLY at startup. And it's the point, because opening
> stdin/stdout/stderr as binary mode is incorrect when it is connected
> to console.
Absolutely wrong. The only way to not mangle the data is to have
I/O done in binary mode.
> My idea started from here. 'tee' should open stdout as text mode in
> this case.
This would mangle the data. I do not want any mangle.
>> I have no idea which screenshot you are talking about here...
> For a staircase, and the executable and the source of 'foo'.
??? Which executable? I'm completely lost...
You proposed a "fix". I did show when the "fix" would not work...
What "executable" do you mean now?
> I think, it's just a guide-line. So it's not important whether it is a
> solution or not. It's enough if it provided a idea to people.
I agree that as a 0th approximation, it is a reasonable idea. But
only as a 0TH approximation...
Yours,
Ilya
KO Myung-Hun
Ilya Zakharevich wrote:
> On 2009-05-24, KO Myung-Hun <ko...@chollian.net> wrote:
>
>>> No. `tee' does nothing to the output. It is due to the program
>>> outputing in binary mode, when the intent is to output the text.
>>>
>
>
>> Sure ?
>>
>
> Yes.
>
>
I think this is true for tee. Is the purpose of tee is to output binary
data ?
> File descriptors on OS/2 are always binary.
This is what I said before. But only for OS/2 APIs.
> Think of tee as
> essentially doing
>
> while (DosRead(...))
> DosWrite()
>
>
Do you have sources of tee ? Actually, tee does like this, I.E. use
DosRead()/DosWrite() pair directly ? Or is it just your imagination ?
>> UNCONDITIONALLY at startup. And it's the point, because opening
>> stdin/stdout/stderr as binary mode is incorrect when it is connected
>> to console.
>>
>
> Absolutely wrong. The only way to not mangle the data is to have
> I/O done in binary mode.
>
Huh ? Absolutely wrong ? You're sure of yourself too much. Are you
saying that outputting to console in binary mode is right and it is
meaningful ? If so, why should not 'foo' output the data to pipe in
binary mode not to mangle the data ?
>> My idea started from here. 'tee' should open stdout as text mode in
>> this case.
>>
>
> This would mangle the data. I do not want any mangle.
>
>
If you write the data to file or to pipe, I agree with you. My
suggestion also do that. But I don't understand why you are sticky to
binary mode for console.
>> For a staircase, and the executable and the source of 'foo'.
>>
>
> ??? Which executable? I'm completely lost...
>
> You proposed a "fix". I did show when the "fix" would not work...
> What "executable" do you mean now?
>
>
Of 'foo'. I wanted to reproduce the problem here because seeing is
believing.
And tee is also what should be fixed.
>> I think, it's just a guide-line. So it's not important whether it is a
>> solution or not. It's enough if it provided a idea to people.
>>
>
> I agree that as a 0th approximation, it is a reasonable idea. But
> only as a 0TH approximation...
>
>
Wow. Thanks for your appreciation.
Ilya Zakharevich
> I think this is true for tee. Is the purpose of tee is to output binary
> data ?
There is no "binary" or "text" data, as far as tee'ing is concerned.
The purpose of tee is to make A COPY OF input data, and do nothing
else. In particular, it should not in any way mangle the data; it
should output exactly the same it did input.
> Do you have sources of tee ? Actually, tee does like this, I.E. use
> DosRead()/DosWrite() pair directly ? Or is it just your imagination ?
`tee' is defined by its semantic, not its implementation.
> Are you saying that outputting to console in binary mode is right
Yes; at least sometimes. What confuses you is that there ARE many
cases where the application does not know what it does. For such
confused applications, one may need some mangling of data to get a
meaningful output to the console.
If an application KNOWS that it is talking to console, no mangling is needed.
> If you write the data to file or to pipe, I agree with you. My
> suggestion also do that. But I don't understand why you are sticky to
> binary mode for console.
Is it so hard to understand? I DO NOT WANT ANY MANGLE; I need `tee'
to be transparent; I want the output with `tee' to be THE SAME as
without `tee'.
I have seen too many situations when using 4OS2's `tee' would HIDE
bugs in my applications. Now, when I switched it off, I do not have
these (very embarassing) problems with the "real" `tee'.
> Of 'foo'. I wanted to reproduce the problem here because seeing is
> believing.
>
> And tee is also what should be fixed.
4os2 is IMO irrepably broken. Good luck to wish some of its problems
fixed. GNU tee is fine:
>perl -wle "binmode STDOUT; print for 1..3"
1
2
3
>perl -wle "binmode STDOUT; print for 1..3" | tee
1
2
3
Yours,
Ilya
KO Myung-Hun
Ilya Zakharevich wrote:
> On 2009-05-30, KO Myung-Hun <ko...@chollian.net> wrote:
>
>
>> Do you have sources of tee ? Actually, tee does like this, I.E. use
>> DosRead()/DosWrite() pair directly ? Or is it just your imagination ?
>>
>
> `tee' is defined by its semantic, not its implementation.
>
>
You don't understand why I asked you. The reason is that the initial
mode of stdin/stdout/stderr is text mode not binary mode if
read()/write() pair is used, as I said before.
> If an application KNOWS that it is talking to console, no mangling is needed.
>
>
If so, when you use printf() in C, '\n' need not to be mangled to '\r\n'
to console ? It is sure that those applications KNOW that they are
talking to console.
On the contrary, if they don't know what they are talking to, the data
should not be mangled.
>> If you write the data to file or to pipe, I agree with you. My
>> suggestion also do that. But I don't understand why you are sticky to
>> binary mode for console.
>>
>
> Is it so hard to understand?
Yes, so hard to understand your not-adaptability
>> Of 'foo'. I wanted to reproduce the problem here because seeing is
>> believing.
>>
>> And tee is also what should be fixed.
>>
>
> 4os2 is IMO irrepably broken.
Yes, it's just your opinion.
> Good luck to wish some of its problems
> fixed. GNU tee is fine:
>
IMO, the irreparably broken is GNU tee, exactly GNU tee what you are
using. The example is your 'foo'. But, 4OS2 and tee 5.94 works fine.
Especially, tee 5.94 works as what I expected. i.e., it output the input
data to console in text mode, to file in binary mode.
Anyway, this is the point that we cannot approach to each other.