std::ios::binary?
Steven T. Hatton
Says this about the std::ios::binary openmode flag:
*binary*: perform input and output in binary mode (as opposed to text mode)
And that is basically _all_ it says about it. What the heck does the binary
flag mean?
--
If our hypothesis is about anything and not about some one or more
particular things, then our deductions constitute mathematics. Thus
mathematics may be defined as the subject in which we never know what we
are talking about, nor whether what we are saying is true.-Bertrand Russell
Alf P. Steinbach
> §27.4.2.1.4 Type ios_base::openmode
> Says this about the std::ios::binary openmode flag:
> *binary*: perform input and output in binary mode (as opposed to text mode)
>
> And that is basically _all_ it says about it. What the heck does the binary
> flag mean?
Basically it means that the i/o functions should not do any translation
to/from external representation of the data: they should behave as they
really should have behaved by default, but unfortunately do not.
For example, a Windows text file has each line terminated by carriage return
+ linefeed, as opposed to just linefeed in Unix, and in text mode '\n' is
translated accordingly for output, and these sequences are translated to
'\n' on input -- in practice '\r' is carriage return and '\n' is linefeed.
Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
That's useful for including a short descriptive text snippet at the start of
a binary file, but I suspect it was originally a misunderstanding of the
Unix shell command to send the current line immediately (which for an empty
line means zero bytes, which in Unix indicates end-of-file). So in text
mode in Windows, a Ctrl Z might be translated to end-of-file on input.
Interestingly the C++ iostream utilities are so extremely badly designed
that you can't make a simple copy-standard-input-to-standard-output-exactly
program using only the standard C++ library, on systems where this is
meaningful but text translation occurs in text mode.
Of course, the religious C++'ers maintain that that shouldn't be possible
anyway because you can't do it on, say, a mobile phone, where C++ could be
used for something, but then they forget that i/o is there for a reason.
--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Gianni Mariani
> §27.4.2.1.4 Type ios_base::openmode
> Says this about the std::ios::binary openmode flag:
> *binary*: perform input and output in binary mode (as opposed to text mode)
>
> And that is basically _all_ it says about it. What the heck does the binary
> flag mean?
On some platforms "\n" translates to "\r\n" when written (and the
reverse when read) to a file while on others it does not. binary mode
will make no such translation.
Steven T. Hatton
Is that per the Standard, or "implementation imposed"? I know that what you
describe is what I typically think of when I think of binary I/O. For
example, in ancient times we used to have to explicitly tell ftp to use
binary mode transfer.
Steven T. Hatton
> * Steven T. Hatton:
>> §27.4.2.1.4 Type ios_base::openmode
>> Says this about the std::ios::binary openmode flag:
>> *binary*: perform input and output in binary mode (as opposed to text
>> mode)
>>
>> And that is basically _all_ it says about it. What the heck does the
>> binary flag mean?
>
> Basically it means that the i/o functions should not do any translation
> to/from external representation of the data: they should behave as they
> really should have behaved by default, but unfortunately do not.
>
> For example, a Windows text file has each line terminated by carriage
> return + linefeed, as opposed to just linefeed in Unix, and in text mode
> '\n' is translated accordingly for output, and these sequences are
> translated to
> '\n' on input -- in practice '\r' is carriage return and '\n' is
> linefeed.
Yes. I am aware of DOS spiders. ^M is what it looks like in Emacs, and it
is never a nice thing to have in a tarball. dos2unix is a great tool.
> Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
> That's useful for including a short descriptive text snippet at the start
> of a binary file, but I suspect it was originally a misunderstanding of
> the Unix shell command to send the current line immediately (which for an
> empty
> line means zero bytes, which in Unix indicates end-of-file). So in text
> mode in Windows, a Ctrl Z might be translated to end-of-file on input.
>
> Interestingly the C++ iostream utilities are so extremely badly designed
> that you can't make a simple
> copy-standard-input-to-standard-output-exactly program using only the
> standard C++ library, on systems where this is meaningful but text
> translation occurs in text mode.
I'm now wondering if I really understood. If I read "characters" from a
std::istream, it goes into an error state when it hits an EOF. That's why
stuff like this works (when it works)
std::vector<float_pair> positions
(istream_iterator<float_pair> (file),
(istream_iterator<float_pair> ()));
I don't believe binary files are terminated by a special character, but I
could be wrong (again).
Take this example:
####################################
Thu Jul 28 06:03:39:> cat main.cpp
#include <fstream>
#include <vector>
#include <iterator>
#include <iostream>
#include <sstream>
using namespace std;
main(int argc, char* argv[]){
if(argc<2) { cerr << "give me a file name" << endl; return -1; }
ifstream file (argv[1],ios::binary);
if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
-1; }
std::vector<unsigned char> data;
copy(istream_iterator<unsigned char>(file)
, istream_iterator<unsigned char>()
, back_inserter(data));
cout<<"read "<<data.size()<<"bytes of data"<< endl;
file.clear();
file.seekg(0,ios::beg);
ostringstream oss;
oss << file.rdbuf();
cout<<"read "<<oss.str().size()<<"bytes of data"<< endl;
}
Thu Jul 28 06:10:21:> g++ -obinio main.cpp
Thu Jul 28 06:13:04:> ./binio binio
read 22075bytes of data
read 22470bytes of data
##########################
Notice the second output is larger than the first.
> Of course, the religious C++'ers maintain that that shouldn't be possible
> anyway because you can't do it on, say, a mobile phone, where C++ could be
> used for something, but then they forget that i/o is there for a reason.
I suspect there are "political" reasons things turned out the way they did.
I really don't know how much of a performance hit it would be if certain
platforms had to do some extra endian shuffling. I do believe the lack of
real binary I/O in the Standard Library is an unexpected inconvenience.
Stroustrup bluntly states that binary I/O is beyond the scope of C++
Standard, and beyond the scope of TC++PL(SE). §21.2.1
Here's an interesting observation:
compiled with gcc 3.3.5
-rwxr-xr-x 1 hattons users 40830 2005-07-28 06:22 binio-3.3.5
compiled with gcc 4.0.1
-rwxr-xr-x 1 hattons users 22470 2005-07-28 06:23 binio-4.0.1
And 4.0.1 produces (much) faster code as well.
Alf P. Steinbach
> * Alf P. Steinbach:
> >
[snip]
> > return + linefeed, as opposed to just linefeed in Unix, and in text mode
> > '\n' is translated accordingly for output, and these sequences are
> > translated to
> > '\n' on input -- in practice '\r' is carriage return and '\n' is
> > linefeed.
>
> Yes. I am aware of DOS spiders. ^M is what it looks like in Emacs,
That's because a carriage return is ASCII 13, Ctrl M.
It would be different using EBCDIC, I imagine.
;-)
[snip]
> > Interestingly the C++ iostream utilities are so extremely badly designed
> > that you can't make a simple
> > copy-standard-input-to-standard-output-exactly program using only the
> > standard C++ library, on systems where this is meaningful but text
> > translation occurs in text mode.
>
> I'm now wondering if I really understood. If I read "characters" from a
> std::istream, it goes into an error state when it hits an EOF. That's why
> stuff like this works (when it works)
>
> std::vector<float_pair> positions
> (istream_iterator<float_pair> (file),
> (istream_iterator<float_pair> ()));
>
> I don't believe binary files are terminated by a special character, but I
> could be wrong (again).
Not in Unix, and not in Windows. However a binary file can contain any byte
values, and those that look like end-of-line markers will be translated in
text mode, and the first that looks like an end-of-file marker may be
translated. All depending on the implementation.
[snip]
> ifstream file (argv[1],ios::binary);
>
> if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
> -1; }
>
> std::vector<unsigned char> data;
>
> copy(istream_iterator<unsigned char>(file)
> , istream_iterator<unsigned char>()
> , back_inserter(data));
>
> cout<<"read "<<data.size()<<"bytes of data"<< endl;
>
> file.clear();
> file.seekg(0,ios::beg);
> ostringstream oss;
[snip]
> ##########################
>
> Notice the second output is larger than the first.
That's probably because the ostringstream does some text mode shenanigans;
although I haven't checked.
> > Of course, the religious C++'ers maintain that that shouldn't be possible
> > anyway because you can't do it on, say, a mobile phone, where C++ could be
> > used for something, but then they forget that i/o is there for a reason.
>
> I suspect there are "political" reasons things turned out the way they did.
> I really don't know how much of a performance hit it would be if certain
> platforms had to do some extra endian shuffling.
? The _default_ is translation. That is, the default is the overhead &
performance hit (+ other much more evil effects) you're mentioning.
> I do believe the lack of
> real binary I/O in the Standard Library is an unexpected inconvenience.
> Stroustrup bluntly states that binary I/O is beyond the scope of C++
> Standard, and beyond the scope of TC++PL(SE). §21.2.1
There is no §21.2.1.
Steven T. Hatton
> * Steven T. Hatton:
>> I don't believe binary files are terminated by a special character, but I
>> could be wrong (again).
>
> Not in Unix, and not in Windows. However a binary file can contain any
> byte values, and those that look like end-of-line markers will be
> translated in text mode, and the first that looks like an end-of-file
> marker may be
> translated. All depending on the implementation.
And they call it a "standard"? :/
> [snip]
>> ifstream file (argv[1],ios::binary);
>>
>> if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
>> -1; }
>>
>> std::vector<unsigned char> data;
>>
>> copy(istream_iterator<unsigned char>(file)
>> , istream_iterator<unsigned char>()
>> , back_inserter(data));
>>
>> cout<<"read "<<data.size()<<"bytes of data"<< endl;
>>
>> file.clear();
>> file.seekg(0,ios::beg);
>> ostringstream oss;
> [snip]
>
>> ##########################
>>
>> Notice the second output is larger than the first.
>
> That's probably because the ostringstream does some text mode shenanigans;
> although I haven't checked.
Sorry, I forgot one important point.
$ls -l binio-3.3.5
-rwxr-xr-x 1 hattons users 40830 2005-07-28 06:22 binio-3.3.5
###### note the file size above, and the second output value below:
$./binio-3.3.5 binio-3.3.5
read 40034bytes of data
read 40830bytes of data
As I understand things, when I do `file.rdbuf() >> oss' I am getting a raw
stream. That, too, may be implementation defined for all I know.
>> I suspect there are "political" reasons things turned out the way they
>> did. I really don't know how much of a performance hit it would be if
>> certain platforms had to do some extra endian shuffling.
>
> ? The _default_ is translation. That is, the default is the overhead &
> performance hit (+ other much more evil effects) you're mentioning.
So is it the case that ios::binary may still not produce real binary
streams? That is, the stream could still act in some ways like a text
stream, e.g., eof?
>> I do believe the lack of
>> real binary I/O in the Standard Library is an unexpected inconvenience.
>> Stroustrup bluntly states that binary I/O is beyond the scope of C++
>> Standard, and beyond the scope of TC++PL(SE). §21.2.1
>
> There is no §21.2.1.
TC++PL(SE).
P.J. Plauger
news:BIidnfI0nK9...@speakeasy.net...
> §27.4.2.1.4 Type ios_base::openmode
> Says this about the std::ios::binary openmode flag:
> *binary*: perform input and output in binary mode (as opposed to text
> mode)
>
> And that is basically _all_ it says about it. What the heck does the
> binary
> flag mean?
"Binary" and "text" are both terms of art from the C Standard, which
is included by reference in the C++ Standard. Binary I/O is byte
transparent, with the possible exception of padding NUL bytes.
Text I/O endeavors to translate between internal newline-delimited
text lines and however the system commonly represents text outside
the program.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
P.J. Plauger
news:42e885e7....@news.individual.net...
>* Steven T. Hatton:
>> §27.4.2.1.4 Type ios_base::openmode
>> Says this about the std::ios::binary openmode flag:
>> *binary*: perform input and output in binary mode (as opposed to text
>> mode)
>>
>> And that is basically _all_ it says about it. What the heck does the
>> binary
>> flag mean?
>
> Basically it means that the i/o functions should not do any translation
> to/from external representation of the data: they should behave as they
> really should have behaved by default, but unfortunately do not.
>
> For example, a Windows text file has each line terminated by carriage
> return
> + linefeed, as opposed to just linefeed in Unix, and in text mode '\n' is
> translated accordingly for output, and these sequences are translated to
> '\n' on input -- in practice '\r' is carriage return and '\n' is
> linefeed.
>
> Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
> That's useful for including a short descriptive text snippet at the start
> of
> a binary file, but I suspect it was originally a misunderstanding of the
> Unix shell command to send the current line immediately (which for an
> empty
> line means zero bytes, which in Unix indicates end-of-file).
No, the usage originated in early systems that couldn't describe the
length of a file to the nearest byte. A CTL-Z delimited the logical
end of text, so your program didn't read trailing garbage.
> So in text
> mode in Windows, a Ctrl Z might be translated to end-of-file on input.
> Interestingly the C++ iostream utilities are so extremely badly designed
> that you can't make a simple
> copy-standard-input-to-standard-output-exactly
> program using only the standard C++ library, on systems where this is
> meaningful but text translation occurs in text mode.
Well, yes you can. Open files in binary mode and use read/write.
> Of course, the religious C++'ers maintain that that shouldn't be possible
> anyway because you can't do it on, say, a mobile phone, where C++ could be
> used for something, but then they forget that i/o is there for a reason.
Nonsense.
P.J. Plauger
> Gianni Mariani wrote:
>
>> Steven T. Hatton wrote:
>>> §27.4.2.1.4 Type ios_base::openmode
>>> Says this about the std::ios::binary openmode flag:
>>> *binary*: perform input and output in binary mode (as opposed to text
>>> mode)
>>>
>>> And that is basically _all_ it says about it. What the heck does the
>>> binary flag mean?
>>
>> On some platforms "\n" translates to "\r\n" when written (and the
>> reverse when read) to a file while on others it does not. binary mode
>> will make no such translation.
>
> Is that per the Standard, or "implementation imposed"?
The C and C++ Standards both require that text mode I/O do whatever
is necessary to convert between the universal internal form of text
streams and whatever the execution environment requires instead.
> I know that what you
> describe is what I typically think of when I think of binary I/O. For
> example, in ancient times we used to have to explicitly tell ftp to use
> binary mode transfer.
And you still do, sometimes, if your FTP utility can't make an
intelligent guess.
P.J. Plauger
> I'm now wondering if I really understood. If I read "characters" from a
> std::istream, it goes into an error state when it hits an EOF. That's why
> stuff like this works (when it works)
>
> std::vector<float_pair> positions
> (istream_iterator<float_pair> (file),
> (istream_iterator<float_pair> ()));
>
> I don't believe binary files are terminated by a special character, but I
> could be wrong (again).
Correct. But the I/O subsystem on every OS has *some* way to tell
you when you run out of input characters.
> ...
>> Of course, the religious C++'ers maintain that that shouldn't be possible
>> anyway because you can't do it on, say, a mobile phone, where C++ could
>> be
>> used for something, but then they forget that i/o is there for a reason.
>
> I suspect there are "political" reasons things turned out the way they
> did.
Nonsense. In the early 1970s, Unix pioneered the notion of a universal
format for text streams, by pushing any needed mappings out to the
device drivers. That text stream model became an integral part of C,
which first evolved under Unix. In the late 1970s and early 1980s,
Whitesmiths, Ltd. ported C to several dozen operating systems. We
elaborated the text/binary I/O model as a way of preserving both the
universal text stream format and the transparent text stream, as
needed. All that technology was captured in the C Standard in the
mid 1980s. It was then incorporated by reference in the C++ Standard
in the mid 1990s. It's there because it works.
> I really don't know how much of a performance hit it would be if certain
> platforms had to do some extra endian shuffling. I do believe the lack of
> real binary I/O in the Standard Library is an unexpected inconvenience.
Might be, if there were such a lack.
> Stroustrup bluntly states that binary I/O is beyond the scope of C++
> Standard, and beyond the scope of TC++PL(SE). §21.2.1
Stroustrup is not nearly as familiar with the Standard C++ library
as he is with the language he invented.
P.J. Plauger
> Alf P. Steinbach wrote:
>
>> * Steven T. Hatton:
>
>>> I don't believe binary files are terminated by a special character, but
>>> I
>>> could be wrong (again).
>>
>> Not in Unix, and not in Windows. However a binary file can contain any
>> byte values, and those that look like end-of-line markers will be
>> translated in text mode, and the first that looks like an end-of-file
>> marker may be
>> translated. All depending on the implementation.
>
> And they call it a "standard"? :/
Yes they do. The C Standard describes how to impose order over a
diverse range of operating systems. You can describe practically
every car made today as having a steering wheel, an accelerator,
and a brake -- if you want to emphasize what's standard about
them. Or you can discuss at great length the different kinds of
linkages and braking systems -- if you want to emphasize how
they differ. Depends on your "political" goal, I suppose.
Alf P. Steinbach
>
> > Interestingly the C++ iostream utilities are so extremely badly designed
> > that you can't make a simple
> > copy-standard-input-to-standard-output-exactly
> > program using only the standard C++ library, on systems where this is
> > meaningful but text translation occurs in text mode.
>
> Well, yes you can. Open files in binary mode and use read/write.
That should be only a few lines; could you please present the code?
> > Of course, the religious C++'ers maintain that that shouldn't be possible
> > anyway because you can't do it on, say, a mobile phone, where C++ could be
> > used for something, but then they forget that i/o is there for a reason.
>
> Nonsense.
See above. ;-)
P.J. Plauger
>* P.J. Plauger:
>>
>> > Interestingly the C++ iostream utilities are so extremely badly
>> > designed
>> > that you can't make a simple
>> > copy-standard-input-to-standard-output-exactly
>> > program using only the standard C++ library, on systems where this is
>> > meaningful but text translation occurs in text mode.
>>
>> Well, yes you can. Open files in binary mode and use read/write.
>
> That should be only a few lines; could you please present the code?
#include <fstream>
int main(int argc, char **argv)
{ // copy a file
if (2 < argc)
{ // copy argv[1] to argv[2] transparently
std::ifstream ifs(argv[1],
std::ios_base::in | std::ios_base::binary);
std::ofstream ofs(argv[2],
std::ios_base::out | std::ios_base::binary);
ofs << ifs.rdbuf();
}
return (0);
}
(I was wrong about needing read and write.)
Alf P. Steinbach
> * Alf P. Steinbach
> > * P.J. Plauger:
> > >
> > > > Interestingly the C++ iostream utilities are so extremely badly
> > > > designed that you can't make a simple copy-standard-input-to-
> > > > library, on systems where this is meaningful but text translation
> > > > occurs in text mode.
> > >
> > > Well, yes you can. Open files in binary mode and use read/write.
> >
> > That should be only a few lines; could you please present the code?
>
> #include <fstream>
>
> int main(int argc, char **argv)
> { // copy a file
> if (2 < argc)
> { // copy argv[1] to argv[2] transparently
> std::ifstream ifs(argv[1],
> std::ios_base::in | std::ios_base::binary);
> std::ofstream ofs(argv[2],
> std::ios_base::out | std::ios_base::binary);
>
> ofs << ifs.rdbuf();
> }
> return (0);
> }
>
> (I was wrong about needing read and write.)
Thanks for the code.
Since I (naturally) don't use iostreams much -- and in fact it's been some
time since I did C++ development -- I learned a new idiom.
And you weren't wrong about read and write: it can be done that way.
What you were wrong about:
The above doesn't copy standard input to standard output. ;-)
Cheers,
- Alf
Steven T. Hatton
Josuttis provides a similar example.
What I would like to know is how to get in iterator over a buffer such as
std::basic_filebuf, so that I can do something like:
copy(istream_iterator<unsigned char>(file_buf.eback())
, istream_iterator<unsigned char>(file_buf.egptr())
, back_inserter(data));
Which I can't do without inheriting from the buffer. That makes binary I/O
seem inconsistent with the rest of the library.
Steven T. Hatton
>
> Thanks for the code.
>
> Since I (naturally) don't use iostreams much -- and in fact it's been
> some
> time since I did C++ development -- I learned a new idiom.
>
> And you weren't wrong about read and write: it can be done that way.
>
> What you were wrong about:
>
> The above doesn't copy standard input to standard output. ;-)
>
>
> Cheers,
>
> - Alf
Dose this potentially modify the stream data? If so, where?
/* The following code example is taken from the book
* "The C++ Standard Library - A Tutorial and Reference"
* by Nicolai M. Josuttis, Addison-Wesley, 1999
*
* (C) Copyright Nicolai M. Josuttis 1999.
* Permission to copy, use, modify, sell and distribute this software
* is granted provided this copyright notice appears in all copies.
* This software is provided "as is" without express or implied
* warranty, and with no claim as to its suitability for any purpose.
*/
#include <iostream>
int main ()
{
// copy all standard input to standard output
std::cout << std::cin.rdbuf();
Steven T. Hatton
Actually I believe that should be more like:
copy(file_buf.eback(), file_buf.egptr(), back_inserter(data));
Alf P. Steinbach
>
> Dose this potentially modify the stream data?
Yes.
> If so, where?
Well, I don't really care exactly where -- when you know the car has
square wheels it doesn't really matter exactly what prevents the motor from
starting and the door from opening, so I've never been interested in that.
> /* The following code example is taken from the book
> * "The C++ Standard Library - A Tutorial and Reference"
> * by Nicolai M. Josuttis, Addison-Wesley, 1999
> *
> * (C) Copyright Nicolai M. Josuttis 1999.
> * Permission to copy, use, modify, sell and distribute this software
> * is granted provided this copyright notice appears in all copies.
> * This software is provided "as is" without express or implied
> * warranty, and with no claim as to its suitability for any purpose.
> */
> #include <iostream>
>
> int main ()
> {
> // copy all standard input to standard output
> std::cout << std::cin.rdbuf();
> }
With MSVC 7.1 under Windows XP Professional:
P:\> dir | find "exe"
28.07.2005 15:05 233 472 vc_project.exe
P:\> vc_project <vc_project.exe >x
P:\> dir | find "x"
28.07.2005 15:05 233 472 vc_project.exe
28.07.2005 15:09 4 484 x
P:\> _
Steven T. Hatton
> * Steven T. Hatton:
>>
>> Dose this potentially modify the stream data?
>
> Yes.
>
>
>> If so, where?
>
> Well, I don't really care exactly where -- when you know the car has
> square wheels it doesn't really matter exactly what prevents the motor
> from starting and the door from opening, so I've never been interested in
> that.
>
>
>> #include <iostream>
>>
>> int main ()
>> {
>> // copy all standard input to standard output
>> std::cout << std::cin.rdbuf();
>> }
>
> With MSVC 7.1 under Windows XP Professional:
>
> P:\> dir | find "exe"
> 28.07.2005 15:05 233 472 vc_project.exe
>
> P:\> vc_project <vc_project.exe >x
>
> P:\> dir | find "x"
> 28.07.2005 15:05 233 472 vc_project.exe
> 28.07.2005 15:09 4 484 x
>
> P:\> _
>
Well, I _am_ interested because I am (was) under the impression that
std::cout.rdbuf() would give me raw data. but now, it seems as if it might
kill spiders, or something like that. The standard streams may actually be
bad examples since they are very OS specific.
Steven T. Hatton
> Well, I _am_ interested because I am (was) under the impression that
> std::cout.rdbuf() would give me raw data. but now, it seems as if it
> might
> kill spiders, or something like that. The standard streams may actually
> be bad examples since they are very OS specific.
How 'bout this?
#include <fstream>
#include <iostream>
using namespace std;
int main(int argc, char* argv[]) {
if(argc < 3) { cerr<<"in_file_name out_file_name"<< endl; return -1; }
ifstream ifs(argv[1],ios::binary);
if(!ifs) { cerr<<" failed to open input file: "<<argv[1]<<endl; return
-1; }
ofstream ofs(argv[2],ios::binary);
if(!ofs) { cerr<<" failed to open output file: "<<argv[2]<<endl; return
-1; }
ofs << ifs.rdbuf();
Alf P. Steinbach
>
> Well, I _am_ interested because I am (was) under the impression that
> std::cout.rdbuf() would give me raw data. but now, it seems as if it might
It does.
> The standard streams may actually be bad examples since they are very
> OS specific.
The OS specificity is common to all streams; the standard streams are not
special in this regard and do no special processing, have special features
etc., except there's no standard C++ way, AFAIK, to turn off their standard
C++ stream objects' trashing of the data.
Julián Albo
> Also, for example, in a Windows text file Ctrl Z denotes end-of-file.
> That's useful for including a short descriptive text snippet at the start
> of a binary file, but I suspect it was originally a misunderstanding of
> the Unix shell command to send the current line immediately (which for an
> empty line means zero bytes, which in Unix indicates end-of-file).
> So in text mode in Windows, a Ctrl Z might be translated to end-of-file
> on input.
The ctrl-z convention in ms-dos is inherited from cp/m operating systems,
they used such convention because they not stored the exact length of the
file, only the number of sectors used. Mark the end with an special
character was the easier solution.
But the ctl-z was not required. If the text file size was a multiple of the
sector size, inserting the eof mark was not required to avoid wasting 1
sector (dozens of GiB disks were not very popular on small systemes those
days ;-) ).
--
Salu2
P.J. Plauger
I was wrong that read and write are necessary, that's all.
> What you were wrong about:
>
> The above doesn't copy standard input to standard output. ;-)
Sorry, I missed that bit. It is true in C90/C95 that you can't
freopen a standard stream to change its mode; it *might* work
properly in C99 but it's not guaranteed. So it has indeed been
a longstanding limitation of Standard C that you can't idly
switch between text and binary I/O, any more than you can idly
switch between byte and wide-character I/O. But that's generally
a minor nuisance. The existence of portable software tools,
such as the MKS Toolkit, shows that you can still implement much
of the Unix idiom on arbitrary operating systems.
P.J. Plauger
You're presuming that you have to use unsigned char to transmit
binary date. While in principle you can make a perverse implementation
of C that corrupts char data and still conforms, in the real world
that doesn't happen. Just do the obvious with an ifstream opened in
binary mode and it'll work fine.
P.J. Plauger
It can, on all but Unix systems. As frequently described earlier
in this thread, reading a text stream under Windows converts CR/LF
to LF and stops reading at CTL-Z.
P.J. Plauger
>* Steven T. Hatton:
>>
>> Well, I _am_ interested because I am (was) under the impression that
>> std::cout.rdbuf() would give me raw data. but now, it seems as if it
>> might
>> kill spiders [control characters], or something like that.
>
> It does.
>
>
>> The standard streams may actually be bad examples since they are very
>> OS specific.
>
> The OS specificity is common to all streams; the standard streams are not
> special in this regard and do no special processing, have special features
> etc., except there's no standard C++ way, AFAIK, to turn off their
> standard
> C++ stream objects' trashing of the data.
What's special about the standard streams is that they're opened
in text mode prior to program startup. At least that's how command
interpreters (shells) almost always work. Nothing prevents you from
starting a program with its standard streams opened in binary
mode, however.
Steven T. Hatton
> Sorry, I missed that bit. It is true in C90/C95 that you can't
> freopen a standard stream to change its mode; it *might* work
> properly in C99 but it's not guaranteed. So it has indeed been
> a longstanding limitation of Standard C that you can't idly
> switch between text and binary I/O, any more than you can idly
> switch between byte and wide-character I/O. But that's generally
> a minor nuisance. The existence of portable software tools,
> such as the MKS Toolkit, shows that you can still implement much
> of the Unix idiom on arbitrary operating systems.
I have no doubt whatsoever that C++ apps _can_ be ported to Windows and act
like the run on GNU/Linux an other Unix oriented systems. You can run the
KDE on Windows! But I am fairly well convinced that C#, C#++ (aka C++/CLI)
and Java will be more portable for the average developer than C++ is. Get
yourself a copy of Java I/O, and read it. Consider what that is like for
the average 18 to 24-year-old trying cs major. Forget that you've been
doing this stuff for so ling that you can writhe hello worl in assebler
without looking anything up.
http://www.cafeaulait.org/books/javaio/
How can C++ I/O be more like that without breaking it? I want a clear and
easy way to create a std::vector<unsigned char> v(begin, end);
where /begin/ and /end/ are the start and end+1 of a file opened in binary
mode.
Any, yes, the last I looked, Java is impemented in C and C++ with a whole
bunch of asm stuff mixed in.
Steven T. Hatton
> You're presuming that you have to use unsigned char to transmit
> binary date. While in principle you can make a perverse implementation
> of C that corrupts char data and still conforms, in the real world
> that doesn't happen. Just do the obvious with an ifstream opened in
> binary mode and it'll work fine.
The type conversion isn't the issue. The issue is that I want it to act
like an STL container. I don't believe I can use iterators over a non-text
file reliably because they will detect content as EOF.
This won't work on a std::ifstream opened in binary mode:
copy(istream_iterator<unsigned char>(file)
, istream_iterator<unsigned char>()
, back_inserter(data));
--
P.J. Plauger
> P.J. Plauger wrote:
>
>> Sorry, I missed that bit. It is true in C90/C95 that you can't
>> freopen a standard stream to change its mode; it *might* work
>> properly in C99 but it's not guaranteed. So it has indeed been
>> a longstanding limitation of Standard C that you can't idly
>> switch between text and binary I/O, any more than you can idly
>> switch between byte and wide-character I/O. But that's generally
>> a minor nuisance. The existence of portable software tools,
>> such as the MKS Toolkit, shows that you can still implement much
>> of the Unix idiom on arbitrary operating systems.
>
> I have no doubt whatsoever that C++ apps _can_ be ported to Windows and
> act
> like the run on GNU/Linux an other Unix oriented systems. You can run the
> KDE on Windows! But I am fairly well convinced that C#, C#++ (aka
> C++/CLI)
> and Java will be more portable for the average developer than C++ is.
C# and Java benefit from running on just one (virtual) system.
They avoid the multiplicity of text file formats in the real
world by, well, avoiding the multiplicity of text file formats
in the real world. C and C++, by contrast, have faced the
problem head on for many years and have dealt with it pretty
successfully.
> Get
> yourself a copy of Java I/O, and read it.
Did so years ago. We've even implemented it and licensed it to
others. But don't get Pete Becker started on all the things wrong
with Java I/O, however, because he bore the brunt of the effort.
> Consider what that is like for
> the average 18 to 24-year-old trying cs major. Forget that you've been
> doing this stuff for so ling that you can writhe hello worl in assebler
> without looking anything up.
Thanks for the advice, but I've been writing tutorial books (over
a dozen) and articles (about 350) for several decades now. I spend
a *lot* of time considering such matters.
> http://www.cafeaulait.org/books/javaio/
>
> How can C++ I/O be more like that without breaking it? I want a clear and
> easy way to create a std::vector<unsigned char> v(begin, end);
> where /begin/ and /end/ are the start and end+1 of a file opened in binary
> mode.
I can think of several ways of doing this, none of which are
particularly hard. Or hard to teach, at least to a student
with an open mind.
> Any, yes, the last I looked, Java is impemented in C and C++ with a whole
> bunch of asm stuff mixed in.
What a coincidence. C is implemented in C, with just a little bit
of assembly language mixed in. And C++ is implemented in C and C++,
with just a little bit of assembly language mixed in. Coincidence?
I don't think so.
P.J. Plauger
> P.J. Plauger wrote:
>
>> You're presuming that you have to use unsigned char to transmit
>> binary date. While in principle you can make a perverse implementation
>> of C that corrupts char data and still conforms, in the real world
>> that doesn't happen. Just do the obvious with an ifstream opened in
>> binary mode and it'll work fine.
>
> The type conversion isn't the issue. The issue is that I want it to act
> like an STL container. I don't believe I can use iterators over a
> non-text
> file reliably because they will detect content as EOF.
>
> This won't work on a std::ifstream opened in binary mode:
>
> copy(istream_iterator<unsigned char>(file)
> , istream_iterator<unsigned char>()
> , back_inserter(data));
You're simply wrong.
Richard Herring
<chatt...@germania.sup> writes
>P.J. Plauger wrote:
>
>> You're presuming that you have to use unsigned char to transmit
>> binary date. While in principle you can make a perverse implementation
>> of C that corrupts char data and still conforms, in the real world
>> that doesn't happen. Just do the obvious with an ifstream opened in
>> binary mode and it'll work fine.
>
>The type conversion isn't the issue. The issue is that I want it to act
>like an STL container. I don't believe I can use iterators over a non-text
>file reliably because they will detect content as EOF.
I suspect you don't understand how istream_iterators work. They know
nothing of content, they just call on operator>>() to handle the
formatting. How would an istream_iterator<MyType> know what values were
supposed to correspond to end-of-file? More prosaically, there's no such
character as EOF, so even an istream_iterator<char> would find it
difficult to interpret content as end-of-file. What really happens is
that they go into the end-of-file state when the underlying stream's
eof() becomes true, not when they see any particular content.
>
>This won't work on a std::ifstream opened in binary mode:
(Well, technically it shouldn't, but the reason is that ifstream is a
basic_ifstream<char>, not basic_ifstream<unsigned char>, so the
iterators have the wrong type ;-)
>
>copy(istream_iterator<unsigned char>(file)
> , istream_iterator<unsigned char>()
> , back_inserter(data));
>
Did you try it?
--
Richard Herring
Steven T. Hatton
> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
> news:ALmdnXb0C8d...@speakeasy.net...
>
>> P.J. Plauger wrote:
> C# and Java benefit from running on just one (virtual) system.
> They avoid the multiplicity of text file formats in the real
> world by, well, avoiding the multiplicity of text file formats
> in the real world. C and C++, by contrast, have faced the
> problem head on for many years and have dealt with it pretty
> successfully.
My response is this: Java and C# provide their portability at runtime. They
run on an abstraction layer which maps their data and I/O representations
to platform specific representations. Mathematica also does this kind of
thing. Bear in mind that Gosling was the first to port Emacs to C. That's
byte-compiled, garbage-collected, exception-throwing, pointerless (from the
user's perspective) library-loading, ELisp virtual machine, Emacs.
Two other noteworthy programmers got their start in XEmacs, which is a fork
of Emacs that tries to remain compatable. Marc Andreesen and Jamie
Zawinski. They were two of the most significant actors in the creation of
Netscape. Netscap uses a somewhat different approach to platform
abstraction than to CLI and the JVM. It's called the Netscape Portable
Runtime. It's kind of like Netcape/Mozilla has the virtual machine built
in to it. Look at the TOC and you should be able to understand it as an
outline of a collection of interfaces:
http://www.mozilla.org/projects/nspr/reference/html/index.html
Mozilla has many features similar to Emacs, such as the built-in
interpreter, the event loop, the multiple buffers, etc.
The C++ approach should be to provide the abstraction in the form of source
code that can be optionally compiled into the user's program to the minimal
extent it is useful.
>> Get
>> yourself a copy of Java I/O, and read it.
>
> Did so years ago. We've even implemented it and licensed it to
> others. But don't get Pete Becker started on all the things wrong
> with Java I/O, however, because he bore the brunt of the effort.
I never said it was perfect. And I am not speaking from the implementor's
perspective. I am talking about the ease of use, and relative uniformity
of interfaces.
>> Consider what that is like for
>> the average 18 to 24-year-old trying cs major. Forget that you've been
>> doing this stuff for so ling that you can writhe hello worl in assebler
>> without looking anything up.
>
> Thanks for the advice, but I've been writing tutorial books (over
> a dozen) and articles (about 350) for several decades now. I spend
> a *lot* of time considering such matters.
I'm not sure if you can appreciate how inelegant C++'s I/O library looks
from my perspective. Don't get me wrong, there are nice parts of it too.
Nonetheless, it's difficult trying to remember the meaning of a couple
dozen cryptically named state flags, the counter intuitive functions to
read and modify their values, the half a dozen pointers into the input and
output buffers (eback is the front, mind you), which function such as
snextc() calls another function such as sbumpc(), and under what
conditions, what the difference between uflow() and underflow() is, etc.
That doesn't even address the couple dozen <cstdio> functions. It's hard
to know what function and flags are relevant to the current state. Which
function are redundant, which modifications to the stream state only apply
for the duration of one function call to the stream extractor or inserter,
how to save the state of the stream before you modify it, so that you can
recover it, etc.
All that lowlevel stuff is useful in some circumstances, but there is no
coherent interface for a person who wants to work with a file (in the Unix
sense). There should be file descriptor objects that provides information
about the file such as is typically found on any operating system. The
abstraction doesn't have to be part of a runtime virtual machine. It can
be a simple abstract base class or class template that is implemented for
each environment. I want to open the file and read it into whatever buffer
is best suited to my application. I shouldn't need to remember a whole
bunch of information, or pull out some esoteric function just because I
want raw data.
What's a binary stream? Well, it's kind of like a stream of text, except it
doesn't modify the data to suit the platform. Can I use it to read binary
data? Well, sort of, but there are a few caveats which you will probably
have to figure out through experimentation. What is the underlying data
representation? It depends.
So far, in the past 48 hours I've seen three examples of veteran programmers
not fully understanding basic notions associated with C++ I/O.
>> http://www.cafeaulait.org/books/javaio/
>>
>> How can C++ I/O be more like that without breaking it? I want a clear
>> and easy way to create a std::vector<unsigned char> v(begin, end);
>> where /begin/ and /end/ are the start and end+1 of a file opened in
>> binary mode.
>
> I can think of several ways of doing this, none of which are
> particularly hard. Or hard to teach, at least to a student
> with an open mind.
Where's my_binary_file.begin() and my_binary_file.end()?
>> Any, yes, the last I looked, Java is impemented in C and C++ with a whole
>> bunch of asm stuff mixed in.
>
> What a coincidence. C is implemented in C, with just a little bit
> of assembly language mixed in. And C++ is implemented in C and C++,
> with just a little bit of assembly language mixed in. Coincidence?
> I don't think so.
That's great, but I see no reason that C++ should fail to provide a higher
level of abstraction which presents the low level services it can
manipulate, in a coherent, minimal, yet complete interface.
Julián Albo
> That's great, but I see no reason that C++ should fail to provide a higher
> level of abstraction which presents the low level services it can
> manipulate, in a coherent, minimal, yet complete interface.
The reasons probably are: some people must write and standarize it. Compiler
and library writers must implement it. And probably the people that can
benefit from it will keep prefering other languages instead of C++ after
all.
But you can write, or initiate a project to write, such libraries and
propose it for inclusion in the standard.
--
Salu2
P.J. Plauger
> P.J. Plauger wrote:
>
>> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
>> news:ALmdnXb0C8d...@speakeasy.net...
>>
>>> P.J. Plauger wrote:
>
>> C# and Java benefit from running on just one (virtual) system.
>> They avoid the multiplicity of text file formats in the real
>> world by, well, avoiding the multiplicity of text file formats
>> in the real world. C and C++, by contrast, have faced the
>> problem head on for many years and have dealt with it pretty
>> successfully.
>
> My response is this: Java and C# provide their portability at runtime.
> They
> run on an abstraction layer which maps their data and I/O representations
> to platform specific representations. Mathematica also does this kind of
> thing.
So does C (and C++). You just happen not to like that model.
> The C++ approach should be to provide the abstraction in the form of
> source
> code that can be optionally compiled into the user's program to the
> minimal
> extent it is useful.
Agreed, and it does, IMO.
>>> Get
>>> yourself a copy of Java I/O, and read it.
>>
>> Did so years ago. We've even implemented it and licensed it to
>> others. But don't get Pete Becker started on all the things wrong
>> with Java I/O, however, because he bore the brunt of the effort.
>
> I never said it was perfect. And I am not speaking from the implementor's
> perspective. I am talking about the ease of use, and relative uniformity
> of interfaces.
So was I. There's nothing like implementing something to see all the
traps and pitfalls waiting for the unwary. And there's nothing like
providing commercial support for learning how the unwary programmers
fall into those traps.
>>> Consider what that is like for
>>> the average 18 to 24-year-old trying cs major. Forget that you've been
>>> doing this stuff for so ling that you can writhe hello worl in assebler
>>> without looking anything up.
>>
>> Thanks for the advice, but I've been writing tutorial books (over
>> a dozen) and articles (about 350) for several decades now. I spend
>> a *lot* of time considering such matters.
>
> I'm not sure if you can appreciate how inelegant C++'s I/O library looks
> from my perspective.
Oh, I can. I just don't much care, since you seem to think your
opinion weighs more than the experiences of several million
programmers over several decades.
> Don't get me wrong, there are nice parts of it too.
> Nonetheless, it's difficult trying to remember the meaning of a couple
> dozen cryptically named state flags, the counter intuitive functions to
> read and modify their values, the half a dozen pointers into the input and
> output buffers (eback is the front, mind you), which function such as
> snextc() calls another function such as sbumpc(), and under what
> conditions, what the difference between uflow() and underflow() is, etc.
>
> That doesn't even address the couple dozen <cstdio> functions. It's hard
> to know what function and flags are relevant to the current state. Which
> function are redundant, which modifications to the stream state only apply
> for the duration of one function call to the stream extractor or inserter,
> how to save the state of the stream before you modify it, so that you can
> recover it, etc.
All that may be true, but only a tiny fraction of programmers have
to work at those levels. And Java has comparable mysteries at the
same subterranean levels.
> All that lowlevel stuff is useful in some circumstances, but there is no
> coherent interface for a person who wants to work with a file (in the Unix
> sense). There should be file descriptor objects that provides information
> about the file such as is typically found on any operating system. The
> abstraction doesn't have to be part of a runtime virtual machine. It can
> be a simple abstract base class or class template that is implemented for
> each environment. I want to open the file and read it into whatever
> buffer
> is best suited to my application. I shouldn't need to remember a whole
> bunch of information, or pull out some esoteric function just because I
> want raw data.
>
> What's a binary stream? Well, it's kind of like a stream of text, except
> it
> doesn't modify the data to suit the platform. Can I use it to read binary
> data? Well, sort of, but there are a few caveats which you will probably
> have to figure out through experimentation. What is the underlying data
> representation? It depends.
>
> So far, in the past 48 hours I've seen three examples of veteran
> programmers
> not fully understanding basic notions associated with C++ I/O.
Okay, I got that you don't understand C++ iostreams, or even C
file I/O. And, following your usual modus operandi, you're
making the rest of the world wrong for your frustration. You
can always do it over right. Or go back to Java.
>>> http://www.cafeaulait.org/books/javaio/
>>>
>>> How can C++ I/O be more like that without breaking it? I want a clear
>>> and easy way to create a std::vector<unsigned char> v(begin, end);
>>> where /begin/ and /end/ are the start and end+1 of a file opened in
>>> binary mode.
>>
>> I can think of several ways of doing this, none of which are
>> particularly hard. Or hard to teach, at least to a student
>> with an open mind.
>
> Where's my_binary_file.begin() and my_binary_file.end()?
See istreambuf_iterator and stop whingeing.
>>> Any, yes, the last I looked, Java is impemented in C and C++ with a
>>> whole
>>> bunch of asm stuff mixed in.
>>
>> What a coincidence. C is implemented in C, with just a little bit
>> of assembly language mixed in. And C++ is implemented in C and C++,
>> with just a little bit of assembly language mixed in. Coincidence?
>> I don't think so.
>
> That's great, but I see no reason that C++ should fail to provide a higher
> level of abstraction which presents the low level services it can
> manipulate, in a coherent, minimal, yet complete interface.
I think it does, at least well enough. YM(obviously)V.
Steven T. Hatton
Are you sure about that?
hattons@ljosalfr:~/code/c++/scratch/binary/
Thu Jul 28 15:59:00:> cat main.cpp
#include <fstream>
#include <vector>
#include <iterator>
#include <iostream>
#include <sstream>
using namespace std;
main(int argc, char* argv[]){
if(argc<2) { cerr << "give me a file name" << endl; return -1; }
ifstream file (argv[1],ios::binary|ios::in);
if(!file) { cerr << "couldn't open the file:"<< argv[1] << endl; return
-1; }
cout<<"File "<<argv[1]<<" openemode = ios::binary|ios::in "<< endl;
std::vector<unsigned char> data;
copy(istream_iterator<unsigned char>(file)
, istream_iterator<unsigned char>()
, back_inserter(data));
cout<<"Read "<<data.size()<<"bytes of data"<< endl;
file.clear();
file.seekg(0,ios::beg);
ostringstream oss;
oss << file.rdbuf();
cout<<"Read "<<oss.str().size()<<"bytes of data"<< endl;
}
hattons@ljosalfr:~/code/c++/scratch/binary/
Thu Jul 28 15:59:06:> g++ -obinio main.cpp
hattons@ljosalfr:~/code/c++/scratch/binary/
Thu Jul 28 15:59:21:> ./binio binio
File binio openemode = ios::binary|ios::in
Read 40157bytes of data
Read 40974bytes of data
hattons@ljosalfr:~/code/c++/scratch/binary/
Thu Jul 28 15:59:30:> ls -l binio
Julián Albo
>>> This won't work on a std::ifstream opened in binary mode:
>>>
>>> copy(istream_iterator<unsigned char>(file)
>>> , istream_iterator<unsigned char>()
>>> , back_inserter(data));
>>
>> You're simply wrong.
>
> Are you sure about that?
(sample snipped for brevity)
The code works. It does not work as you expected, because your expectations
are wrong. You are doing default formatted stream input and expcting a
result according to different rules.
Try to unset skipws flag, for example.
--
Salu2
Steven T. Hatton
> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
> news:ffidnZR5EYj...@speakeasy.net...
>
>> P.J. Plauger wrote:
>>
>>> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
>>> news:ALmdnXb0C8d...@speakeasy.net...
>>>
>>>> P.J. Plauger wrote:
>>
>>> C# and Java benefit from running on just one (virtual) system.
>>> They avoid the multiplicity of text file formats in the real
>>> world by, well, avoiding the multiplicity of text file formats
>>> in the real world. C and C++, by contrast, have faced the
>>> problem head on for many years and have dealt with it pretty
>>> successfully.
>>
>> My response is this: Java and C# provide their portability at runtime.
>> They
>> run on an abstraction layer which maps their data and I/O representations
>> to platform specific representations. Mathematica also does this kind of
>> thing.
>
> So does C (and C++). You just happen not to like that model.
You're playing semantics, and you know damn good and well what I meant.
>> The C++ approach should be to provide the abstraction in the form of
>> source
>> code that can be optionally compiled into the user's program to the
>> minimal
>> extent it is useful.
>
> Agreed, and it does, IMO.
It could do far better.
>> I never said it was perfect. And I am not speaking from the
>> implementor's
>> perspective. I am talking about the ease of use, and relative uniformity
>> of interfaces.
>
> So was I. There's nothing like implementing something to see all the
> traps and pitfalls waiting for the unwary. And there's nothing like
> providing commercial support for learning how the unwary programmers
> fall into those traps.
Hmmm... I know what my experience was. It was relatively easy to pick up,
and it does a whole lot, out of the box.
<quote>
streambuf: The Stream Buffer Classes
excerpted from pages 84-109 of Standard C++ IOStreams and Locales,
by Angelika Langer and Klaus Kreft
© 2000 by Addison Wesley Longman Inc.
Reproduced by permission of Addison Wesley Longman. All rights reserved.
You might wonder why on earth we devote 10+ pages of our book to the guts of
stream buffers, which seem to be nothing more than an implementation detail
of the IOStreams library. For an answer, let us quote Jerry Schwarz, the
"inventor" of IOStreams (quote taken from the foreword):
"A major goal in my original design was that it be extensible in
interesting ways. In particular, in the stream library the streambuf class
was an implementation detail, but in the iostream library I intended it to
be a usable class in its own right. I was hoping for the promulgation of
many streambufs with varied functionality. I wrote a few myself, but almost
no one else did. I answered many more questions of the form "how do I make
my numbers look like this" than "how do I write a streambuf". And textbook
authors also tended to ignore streambufs. Apparently they did not share my
view that the architecture of the input/output library was an interesting
case study."
</quote>
>>>> Consider what that is like for
>>>> the average 18 to 24-year-old trying cs major. Forget that you've been
>>>> doing this stuff for so ling that you can writhe hello worl in assebler
>>>> without looking anything up.
>>>
>>> Thanks for the advice, but I've been writing tutorial books (over
>>> a dozen) and articles (about 350) for several decades now. I spend
>>> a *lot* of time considering such matters.
>>
>> I'm not sure if you can appreciate how inelegant C++'s I/O library looks
>> from my perspective.
>
> Oh, I can. I just don't much care, since you seem to think your
> opinion weighs more than the experiences of several million
> programmers over several decades.
A lot of code I've seen uses C-style I/O, or some third-party C++ I/O
library. My opinion is not unique.
> All that may be true, but only a tiny fraction of programmers have
> to work at those levels. And Java has comparable mysteries at the
> same subterranean levels.
Yes, but Java does a better job of abstracting them. Perhaps I need to buy
and read the book quoted above. It looks as if it provides an insightful
treatment.
> Okay, I got that you don't understand C++ iostreams, or even C
> file I/O.
Not well, and therein lies a problem with a lot of experience C++
programmers. They simply assume the person already know a good deal about
C programming.
>>> I can think of several ways of doing this, none of which are
>>> particularly hard. Or hard to teach, at least to a student
>>> with an open mind.
>>
>> Where's my_binary_file.begin() and my_binary_file.end()?
>
> See istreambuf_iterator and stop whingeing.
I was asking about the stream buffer not the stream. I don't want formatted
I/O. Also, I was pointing out the divergince in interfaces betwee I/O and
the STL, not to mention std::string.
>> That's great, but I see no reason that C++ should fail to provide a
>> higher level of abstraction which presents the low level services it can
>> manipulate, in a coherent, minimal, yet complete interface.
>
> I think it does, at least well enough. YM(obviously)V.
I know there are more comprehensive C++ I/O libraries. I prefer to seek
standards because they mean my code and coding practices should be
portable.
One thing that seem like it would be useful is extending the idea used in
numeric limits to apply ot the I/O classes, and any other classe for which
one might want information, and in clude code to print the status and
description of the class being examined.
P.J. Plauger
>>> My response is this: Java and C# provide their portability at runtime.
>>> They
>>> run on an abstraction layer which maps their data and I/O
>>> representations
>>> to platform specific representations. Mathematica also does this kind
>>> of
>>> thing.
>>
>> So does C (and C++). You just happen not to like that model.
>
> You're playing semantics, and you know damn good and well what I meant.
Yes, I know what you mean and no, I'm not "playing semantics." I
meant exactly what I said.
>>> The C++ approach should be to provide the abstraction in the form of
>>> source
>>> code that can be optionally compiled into the user's program to the
>>> minimal
>>> extent it is useful.
>>
>> Agreed, and it does, IMO.
>
> It could do far better.
So can many things. If you think you know how to do something
better, fer Crissake go do it and prove it to the rest of us.
Meanwhile, we'll muddle along with the stuff that works. And
I bet you'd be more productive if you spent more time learning
the available tools and less time making a case that they're
designed badly.
>>> I never said it was perfect. And I am not speaking from the
>>> implementor's
>>> perspective. I am talking about the ease of use, and relative
>>> uniformity
>>> of interfaces.
>>
>> So was I. There's nothing like implementing something to see all the
>> traps and pitfalls waiting for the unwary. And there's nothing like
>> providing commercial support for learning how the unwary programmers
>> fall into those traps.
>
> Hmmm... I know what my experience was. It was relatively easy to pick up,
> and it does a whole lot, out of the box.
So does printf, and iostreams, if you confine yourself to
the obious usages. You keep comparing different levels of
abstraction.
>>> I'm not sure if you can appreciate how inelegant C++'s I/O library looks
>>> from my perspective.
>>
>> Oh, I can. I just don't much care, since you seem to think your
>> opinion weighs more than the experiences of several million
>> programmers over several decades.
>
> A lot of code I've seen uses C-style I/O, or some third-party C++ I/O
> library. My opinion is not unique.
That doesn't make it necessarily right, or important, however.
>> All that may be true, but only a tiny fraction of programmers have
>> to work at those levels. And Java has comparable mysteries at the
>> same subterranean levels.
>
> Yes, but Java does a better job of abstracting them.
Once again, a matter of opinion. Java is no longer sufficiently
important to me to critique, however.
> Perhaps I need to buy
> and read the book quoted above. It looks as if it provides an insightful
> treatment.
Not a bad idea. Langer and Kreft put a lot of work into their
book, as did Josuttis.
>> Okay, I got that you don't understand C++ iostreams, or even C
>> file I/O.
>
> Not well, and therein lies a problem with a lot of experience C++
> programmers. They simply assume the person already know a good deal about
> C programming.
That's an endemic problem in our trade, not unique to C and/or C++
programmers. But I don't see what that has to do with your
willingness to learn something before you start criticizing it.
>>>> I can think of several ways of doing this, none of which are
>>>> particularly hard. Or hard to teach, at least to a student
>>>> with an open mind.
>>>
>>> Where's my_binary_file.begin() and my_binary_file.end()?
>>
>> See istreambuf_iterator and stop whingeing.
>
> I was asking about the stream buffer not the stream.
And I told you the proper answer to your question, if you're
capable of hearing it.
> I don't want formatted
> I/O.
I wasn't offering it.
> Also, I was pointing out the divergince in interfaces betwee I/O and
> the STL, not to mention std::string.
And I was pointing out the convergence, if you'll only stop
being right and listen.
>>> That's great, but I see no reason that C++ should fail to provide a
>>> higher level of abstraction which presents the low level services it can
>>> manipulate, in a coherent, minimal, yet complete interface.
>>
>> I think it does, at least well enough. YM(obviously)V.
>
> I know there are more comprehensive C++ I/O libraries. I prefer to seek
> standards because they mean my code and coding practices should be
> portable.
Standards can help in that area, to be sure. Doesn't seem to
bother Java programmers, however.
> One thing that seem like it would be useful is extending the idea used in
> numeric limits to apply ot the I/O classes, and any other classe for which
> one might want information, and in clude code to print the status and
> description of the class being examined.
I suppose, even though I don't see the utility of such a blend.
But isn't that akin to mixing iostreams and STL?
Alf P. Steinbach
> Nothing prevents you from
> starting a program with its standard streams opened in binary
> mode, however.
That's news to me. Exactly how does one do that in C++? Or, if it is an
operating system matter, in Windows (where it matters most, although Mac is
also problematic in this regard)?
Stephen Howe
> did.
> I really don't know how much of a performance hit it would be if certain
> platforms had to do some extra endian shuffling. I do believe the lack of
> real binary I/O in the Standard Library is an unexpected inconvenience.
Your beliefs are wrong. There is real binary I/O.
That is what the read() & write() member functions if istream and ostream
are for.
> Stroustrup bluntly states that binary I/O is beyond the scope of C++
> Standard, and beyond the scope of TC++PL(SE). §21.2.1
NO HE DOES NOT !!!!
You are misquoting . He says
"It is possible to define streams for which the physical I/O is not done in
terms of characters. However, such streams are beyond the scope of the C++
standard and beyond the scope of this book (21.10[15]) "
He is not talking abour binary I/O.
Stephen Howe
Stephen Howe
> yourself a copy of Java I/O, and read it. Consider what that is like for
> the average 18 to 24-year-old trying cs major. Forget that you've been
> doing this stuff for so ling that you can writhe hello worl in assebler
> without looking anything up.
Frankly, you look like a complete moron to this newsgroup.
That may sound unkind but it is the honest truth.
To me, you seem just like Peter Olcott.
He was indulged in comp.lang.c++.moderated and comp.theory.
At one point he claimed to have overthrown the Halting Problem and Alan
Turing was wrong.
Check out thread
http://groups-beta.google.com/group/comp.theory/browse_frm/thread/2a388333436e13be/513de627b7a574f3?lnk=st&q=peter+Olcott+Halting+Problem&rnum=7&hl=en#513de627b7a574f3
where he admits his crack-pottery
But here in comp.lang.c++, ..., who do you think you are advising?
No doubt you will be advising P. J. Plauger that he should attend some ISO C
and C++ committee meetings or advising Stroustrup or Andrew Koenig that they
need to improve their knowledge on C when they have been on standardisation
committees measured in decades. You just look plain foolish.
Stephen Howe
Stephen Howe
>> byte values, and those that look like end-of-line markers will be
>> translated in text mode, and the first that looks like an end-of-file
>> marker may be
>> translated. All depending on the implementation.
>
> And they call it a "standard"? :/
Yup. Implementions can vary a lot from embedded devices to PCs to minis to
mainframes.
Both the C and C++ standards give a lot of implementation freedom for
compilers
And that is why both languages have been very successful and will continue
to be so.
> Sorry, I forgot one important point.
> $ls -l binio-3.3.5
> -rwxr-xr-x 1 hattons users 40830 2005-07-28 06:22 binio-3.3.5
>
> ###### note the file size above, and the second output value below:
>
> $./binio-3.3.5 binio-3.3.5
> read 40034bytes of data
> read 40830bytes of data
>
> As I understand things, when I do `file.rdbuf() >> oss' I am getting a raw
> stream.
Yeah, so?
istream_iterator<unsigned char> is using >> to read from the file and I
think it might discard any initial whitespace characters.
> So is it the case that ios::binary may still not produce real binary
> streams?
It will.
> That is, the stream could still act in some ways like a text
> stream, e.g., eof?
No. It is text mode you have to watch out for that is OS-specific.
Binary mode is simple.
Stephen Howe
Alf P. Steinbach
That was uncalled for, Stephen. All this name-calling and
appeal-to-authority (which I don't think the authorities in question would
agree with!) etc. indicates that somewhere in this thread, and I don't care
to check, you have run out of arguments. An apology of about the same size
and intensity would be the right thing to do.
P.J. Plauger
>* P.J. Plauger:
>> Nothing prevents you from
>> starting a program with its standard streams opened in binary
>> mode, however.
>
> That's news to me. Exactly how does one do that in C++? Or, if it is an
> operating system matter, in Windows (where it matters most, although Mac
> is
> also problematic in this regard)?
Take your favorite command interpreter and have it open files
in binary mode. How you present the standard streams to a
process you spawn (via system or exec) is system specific,
but it's straightforward. Just don't ask me for a good command
line notation for binary files -- I got grossed out somewhere
around 2>&1.
Steven T. Hatton
> That was uncalled for, Stephen. All this name-calling and
> appeal-to-authority (which I don't think the authorities in question would
> agree with!) etc. indicates that somewhere in this thread, and I don't
> care
> to check, you have run out of arguments. An apology of about the same
> size and intensity would be the right thing to do.
What I want to know is whether the Jerry Schwartz who designed the C++ I/O
is the same one who has recently been working on the Java memory model.
Alf P. Steinbach
I'm sorry, that won't work, and is not very meaningful.
First, because the text-mode mangling & destruction of the data stream is
internal to the C++ libraries.
I.e. it can be there no matter what.
Second, because in e.g. Windows and Unix there's no such thing as having the
command interpreter opening the standard streams in binary mode or text mode
when running a program, and there's correspondingly no information about
that passed to the program, so it's not that I won't ask you for the
details: there are no such details. I think perhaps you're remembering
something about FTP command interpreters. FTP file transfer doesn't apply.
Third, because even if a C++ implementation did support an extension to set
the openmode of the standard streams (as I recall MSVC does support this,
but not any convention for passing openmode information into the program),
that would not be standard C++, and the point was about standard C++, not
what one can achieve using language/library extensions or assembly language.
Cheers,
- Alf
Dietmar Kuehl
> I don't believe I can use iterators over a
> non-text file reliably because they will detect content as EOF.
Nope, they don't. The problem is that you are doing text formatted
I/O with your choice of iterators. You want to use different
iterators:
> copy(istream_iterator<unsigned char>(file)
> , istream_iterator<unsigned char>()
> , back_inserter(data));
This should read
std::copy(std::istreambuf_iterator<char>(file),
std::istreambuf_iterator<char>(),
std::back_inserter(data));
with a suitable definition of 'data'. Dump the ill-advised use of
'unsigned char' in this context and use 'istream*buf*_iterator'.
Actually, I guess that P.J.Plauger missed that you used the wrong
iterator class in this reply.
--
<mailto:dietma...@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence
Dietmar Kuehl
>> What I would like to know is how to get in iterator over a buffer such as
>> std::basic_filebuf, so that I can do something like:
You can't and you shouldn't. Actually, you would be well advised
to read the documentation of classes before guessing about their
semantics: if at all, you want to use 'gptr()' as your starting
point because the sequence [eback(), gptr()) consists of already
processed characters (... and your wrong use clearly shows that
you had no inkling about what you are doing).
>> copy(istream_iterator<unsigned char>(file_buf.eback())
>> , istream_iterator<unsigned char>(file_buf.egptr())
>> , back_inserter(data));
>>
>> Which I can't do without inheriting from the buffer. That makes binary
>> I/O seem inconsistent with the rest of the library.
No, it doesn't. You should operate on top of the stream buffer
abstraction, not within. Maybe you should at least try to
understand what the classes and functions are about and for.
Dietmar Kuehl
> What I want to know is whether the Jerry Schwartz who designed the C++ I/O
> is the same one who has recently been working on the Java memory model.
I don't know for sure but I consider it likely.
Dietmar Kuehl
> http://www.cafeaulait.org/books/javaio/
>
> How can C++ I/O be more like that without breaking it?
You mean, how can C++ abandon its easy to use I/O library in favor
of crap? Well, I prefer to stick with the current C++ I/O...
> I want a clear and
> easy way to create a std::vector<unsigned char> v(begin, end);
std::ifstream in("file", std::ios_base::binary);
std::vector<unsigned char> v((std::istreambuf_iterator<char>(in)),
std::istreambuf_iterator<char>());
Big deal.
Steven T. Hatton
> Steven T. Hatton wrote:
>> http://www.cafeaulait.org/books/javaio/
>>
>> How can C++ I/O be more like that without breaking it?
>
> You mean, how can C++ abandon its easy to use I/O library in favor
> of crap? Well, I prefer to stick with the current C++ I/O...
>
>> I want a clear and
>> easy way to create a std::vector<unsigned char> v(begin, end);
>
> std::ifstream in("file", std::ios_base::binary);
> std::vector<unsigned char> v((std::istreambuf_iterator<char>(in)),
> std::istreambuf_iterator<char>());
>
> Big deal.
Won't that eat whitespaces?
Steven T. Hatton
>
> You're presuming that you have to use unsigned char to transmit
> binary date. While in principle you can make a perverse implementation
> of C that corrupts char data and still conforms, in the real world
> that doesn't happen. Just do the obvious with an ifstream opened in
> binary mode and it'll work fine.
Actually that wasn't _my_ assumption. It's what someone else sugested I do.
I'm fairly well convinced that my original approach is completely viable.
That is to use std::stringstream and std::string. Especially since I plan
of parseing the object file. I just took this tangent because it bothered
me that I didn't know how to work with this stuff as well as I should.
Steven T. Hatton
>>The type conversion isn't the issue. The issue is that I want it to act
>>like an STL container. I don't believe I can use iterators over a
>>non-text file reliably because they will detect content as EOF.
>
> I suspect you don't understand how istream_iterators work. They know
> nothing of content, they just call on operator>>() to handle the
> formatting.
Yes. That is corret. That part I understood.
> How would an istream_iterator<MyType> know what values were
> supposed to correspond to end-of-file? More prosaically, there's no such
> character as EOF, so even an istream_iterator<char> would find it
> difficult to interpret content as end-of-file. What really happens is
> that they go into the end-of-file state when the underlying stream's
> eof() becomes true, not when they see any particular content.
Again. That wasn't in question. It was simply easier to state it as I did.
Since not of the objects in question have eyes, none of them can actually
"see".
>>This won't work on a std::ifstream opened in binary mode:
>
> (Well, technically it shouldn't, but the reason is that ifstream is a
> basic_ifstream<char>, not basic_ifstream<unsigned char>, so the
> iterators have the wrong type ;-)
>>
>>copy(istream_iterator<unsigned char>(file)
>> , istream_iterator<unsigned char>()
>> , back_inserter(data));
>>
> Did you try it?
>
I certainly did.
Alf P. Steinbach
> Dietmar Kuehl wrote:
>
> > Steven T. Hatton wrote:
> >> http://www.cafeaulait.org/books/javaio/
> >>
> >> How can C++ I/O be more like that without breaking it?
> >
> > You mean, how can C++ abandon its easy to use I/O library in favor
> > of crap? Well, I prefer to stick with the current C++ I/O...
> >
> >> I want a clear and
> >> easy way to create a std::vector<unsigned char> v(begin, end);
> >
> > std::ifstream in("file", std::ios_base::binary);
> > std::vector<unsigned char> v((std::istreambuf_iterator<char>(in)),
> > std::istreambuf_iterator<char>());
> >
> > Big deal.
> Won't that eat whitespaces?
Read Dietmar's previous reply.
Anyway, the things that can be done easily, like the above, are of the
variety "error handling omitted", and it's not like what seems like an easy
way when presented to you is easy to come by when you don't already know it.
The popularity in certain circles seems to stem from the _complexity_, how
hard it is to use, and how it allows one to employ all sorts of "clever"
tricks to combat the inefficiency and unsafety, and how there is a whole
package of terminology mostly only understood by the converts, allowing easy
recognition of fellow believers and -- of course -- of unbelievers.
So, when Dietmar characterizes alternatives to the standard library's i/o
facilities as "crap", that should IMO be considered as an emotional
utterance from a believer to an agnostic who just possibly may be converted
to the one true faith, and whose rash, random questioning should be stopped
before it lands on some less easily handled or diverted topic.
Like, say, exceptions, or like, say, undefined behavior of input
conversions, or like, say, the inability to copy standard input to standard
output exactly, which is rather basic. Not to mention two-phase
initialization and other such ugliness in the internal design. Happily,
many practicing C++ programmers can choose whatever works or make it from
scratch, but students seldom can -- and that means we're putting them off.
Steven T. Hatton
That's what I always want with my sound files, and image files! Who needs a
bunch of white space anyway? ;) They call _that_ binary? A big ol'
stream.set_unformatted_binary(), would have saved me a lot of time.
Thanks.
Ian
> <quote>
> streambuf: The Stream Buffer Classes
> excerpted from pages 84-109 of Standard C++ IOStreams and Locales,
> by Angelika Langer and Klaus Kreft
>
> © 2000 by Addison Wesley Longman Inc.
> Reproduced by permission of Addison Wesley Longman. All rights reserved.
>
> You might wonder why on earth we devote 10+ pages of our book to the guts of
> stream buffers, which seem to be nothing more than an implementation detail
> of the IOStreams library. For an answer, let us quote Jerry Schwarz, the
> "inventor" of IOStreams (quote taken from the foreword):
>
> "A major goal in my original design was that it be extensible in
> interesting ways. In particular, in the stream library the streambuf class
> was an implementation detail, but in the iostream library I intended it to
> be a usable class in its own right. I was hoping for the promulgation of
> many streambufs with varied functionality. I wrote a few myself, but almost
> no one else did. I answered many more questions of the form "how do I make
> my numbers look like this" than "how do I write a streambuf". And textbook
> authors also tended to ignore streambufs. Apparently they did not share my
> view that the architecture of the input/output library was an interesting
> case study."
> </quote>
>
implement one or two.
They are fun and very useful when understood.
Ian
Steven T. Hatton
> Steven T. Hatton wrote:
>
>> That's great, but I see no reason that C++ should fail to provide a
>> higher level of abstraction which presents the low level services it can
>> manipulate, in a coherent, minimal, yet complete interface.
>
> The reasons probably are: some people must write and standarize it.
> Compiler and library writers must implement it. And probably the people
> that can benefit from it will keep prefering other languages instead of
> C++ after all.
Technically speaking, it is not the perview of compiler writers, but rather
the library implementors. Yes, they are often one in the same. After you
posted, I began thinking about why there hasn't been as much contribution
as one might expect to something as important as the C++ Standard Library.
There /are/ several freely available I/O libraries for various kinds. Some
of them look very nice, and from experience with software that uses them,
some of the are very reliable, and powerful. The problem, I suspect, is
the licensing conflicts with the conditions of the C++ Standard.
> But you can write, or initiate a project to write, such libraries and
> propose it for inclusion in the standard.
Actually, what probably should be done is a much more difficult task. That
is to identify the abstract interfaces which effectively capture the common
aspects of the vast majority of existing, and probable implementation of
any given design domain such as local harddrive I/O. Also worth
considering is the further abstraction of the different I/O interfaces such
as harddrive, IPC, network, in order to create a baseclass for all I/O
instances. I'm really not talking about the existing std::ios_base, or
basic_ios. They may prove to provide some element of that interface, but I
am considering starting from tabular rasa.
These should be interfaces, sans data, or implemented functions.
Julián Albo
>> Try to unset skipws flag, for example.
>>
> That's what I always want with my sound files, and image files! Who needs
> a bunch of white space anyway? ;) They call _that_ binary?
So do you think that the codes of whitespace characters are not binary?
Binary, as ther people say in this thread several times, is related only to
new line conventions and end of file detection. You have mixed the concept
of text/binary mode with the formatted/unformatted input, and you blame the
language for that?
> A big ol' stream.set_unformatted_binary(), would have saved me a lot of
> time.
Learning the ways the language already has to do things instead of expecting
that ways more likely to you are written will save you a lot more.
--
Salu2
Richard Herring
<chatt...@germania.sup> writes
>Richard Herring wrote:
>
>>>The type conversion isn't the issue. The issue is that I want it to act
>>>like an STL container. I don't believe I can use iterators over a
>>>non-text file reliably because they will detect content as EOF.
>>
>> I suspect you don't understand how istream_iterators work. They know
>> nothing of content, they just call on operator>>() to handle the
>> formatting.
>
>Yes. That is corret. That part I understood.
It certainly didn't read that way. EOF is a macro, eof() is a member
function, end-of-file is a state, so your "EOF" is possibly ambiguous.
>
>> How would an istream_iterator<MyType> know what values were
>> supposed to correspond to end-of-file? More prosaically, there's no such
>> character as EOF, so even an istream_iterator<char> would find it
>> difficult to interpret content as end-of-file. What really happens is
>> that they go into the end-of-file state when the underlying stream's
>> eof() becomes true, not when they see any particular content.
>
>Again. That wasn't in question.
"iterators [...] will detect content" certainly suggested that you
thought istream_iterators detect content.
>It was simply easier to state it as I did.
Easier for you... but since it's wrong, it requires more work on the
part of other people to clear up the misconceptions lest other readers
of this thread get the wrong message.
>Since not of the objects in question have eyes, none of them can actually
>"see".
It's a metaphor. If you have trouble with that, feel free to substitute
some periphrastic verbiage that expresses it more technically.
Metaphor aside, neither streams in binary mode, nor operator >>, nor
istream_iterators, use content to determine end of file.
--
Richard Herring
Dietmar Kuehl
> * Steven T. Hatton:
>> Dietmar Kuehl wrote:
>> > std::ifstream in("file", std::ios_base::binary);
>> > std::vector<unsigned char> v((std::istreambuf_iterator<char>(in)),
>> > std::istreambuf_iterator<char>());
>> Won't that eat whitespaces?
It won't. Stream buffer iterators operate on the character level and
don't use any form of formatting, neither when reading nor when
writing. They are quite different from stream iterators which operate
on object level and are for [text] formatted reading and writing. The
only mutations done when using stream buffer iterators are those done
by the underlying stream buffer and/or the system like possible
operations performed in text mode. Since the file is opened in binary,
these are also suppressed
> Anyway, the things that can be done easily, like the above, are of the
> variety "error handling omitted",
Is it, now? Well, I didn't check for whether 'in' could be opened, nor
did I check the result of closing it. However, I concentrated on the
problem at hand. Can I take your comment towards error handling that
you are always posting commercial strength code which can be used
unchanged in production code (from a code quality point of view; there
are, of course, copyright restrictions)? ... or are you seeing other
needs for error handling which are not covered? If so, which?
> and it's not like what seems like an easy way when presented to you is
> easy to come by when you don't already know it.
Yes, indeed, it is hard to find this solution if you got stuck with
the first class which sounds as if it could solve the problem (the
stream iterator classes). Given the specification which already asked
for an approach using iterators I consider it indeed obvious how the
problem is to be addressed - assuming you browse the complete list of
possible classes (which in this case consists of a set with a total of
two elements).
> The popularity in certain circles seems to stem from the _complexity_, how
> hard it is to use, and how it allows one to employ all sorts of "clever"
> tricks to combat the inefficiency and unsafety, and how there is a whole
> package of terminology mostly only understood by the converts, allowing
> easy
> recognition of fellow believers and -- of course -- of unbelievers.
Wow. The solution I posted above qualifies as "complex" if I understood
your correct. Can you please show me *any* alternative which achieves
the same objective simpler. Use Java or .Net classes, whatever library
you wish (as long as I can view documentation of the used components
online to verify that it is not made up to fit this problem).
You really think the above is "hard to use"? ... and, honestly, I don't
see an employment of clever tricks there. ... and concerning the
terminology: you always need to understand the basics of a library
component to use it effectively. This applies to the standard C++ library
as well as to any alternatives. Of course, you will struggle hard if
you refuse to accept these basics and consider it a clever idea to use
the basics of some other library.
> So, when Dietmar characterizes alternatives to the standard library's i/o
> facilities as "crap", that should IMO be considered as an emotional
> utterance from a believer to an agnostic who just possibly may be
> converted to the one true faith, and whose rash, random questioning should
> be stopped before it lands on some less easily handled or diverted topic.
Ah, yes, I'm one of those highly religious fanatics who never ever would
possibly admit a flaw in his belief. This attitude is, of course,
clearly reflected in all articles I'm writing and absolute devotion and
faith is shown: the True Belief is not to be questioned. Every letter is
Holy Law. Hark infidel: do not dare to befoul thy STL!
As an alternative interpretation of my characterization I would offer
that I had experience with the Java I/O library which I didn't like at
all. Of course, to some something like
BufferedReader in = new BufferedReader(new FileReader(filename));
is far easier to read then
std::ifstream in(filename);
not to mention that the latter is more complex to everybody except to
those converted to the True Faith. Of course, non-intuitive code like
std::ifstream in("input.txt");
std::string name;
int age;
double weight;
while (in >> name >> age >> weight)
/* process the data */;
if (!in.eof())
std::cout << "format error during input\n";
should likewise abandoned in favor of the much clearer Java approach:
BufferedReader in = new BufferedReader(new FileReader("input.txt"));
String name;
int age;
double weight;
NumberFormat nf = NumberFormat.getInstance();
try {
while (true) // I haven't figured out how to easily detect end of
{ // file in a rush; sorry, I haven't embraced the Java I/O
StringTokenizer tok = new StringTokenizer(in.readLine(), "\t");
name = nextToken();
Number num = nf.parse(nextToken);
age = num.intValue();
num = nf.parse(nextToken);
weight = num.doubleValue();
// process the data
}
}
catch (ParseException e)
{
System.out.println("format error during input\n");
}
Now, *THAT* looks clear to me! It is *much* more obvious than the C++
stuff. ...or, just maybe, things like this are the reason for my
characterization of Java's I/O library. Have you possibly considered
this? Of course, there is a fair chance that I have missed the correct
Java approach to parsing text files. In this case I would appreciate
being pointed into the right direction and I guarantee that I'm willing
to reevaluate Java I/O if it is just me.
> Like, say, exceptions, or like, say, undefined behavior of input
> conversions, or like, say, the inability to copy standard input to
> standard
> output exactly, which is rather basic. Not to mention two-phase
> initialization and other such ugliness in the internal design. Happily,
> many practicing C++ programmers can choose whatever works or make it from
> scratch, but students seldom can -- and that means we're putting them
> off.
You mean, we should embrace Java I/O as an alternative in C++ and the
Java code above is *not* putting students off while the C++ code is?
Well, I might have to readjust my measure of complexity then...
P.J. Plauger
> Julián Albo wrote:
>
>> Steven T. Hatton wrote:
>>
>>> That's great, but I see no reason that C++ should fail to provide a
>>> higher level of abstraction which presents the low level services it can
>>> manipulate, in a coherent, minimal, yet complete interface.
>>
>> The reasons probably are: some people must write and standarize it.
>> Compiler and library writers must implement it. And probably the people
>> that can benefit from it will keep prefering other languages instead of
>> C++ after all.
>
> Technically speaking, it is not the perview of compiler writers, but
> rather
> the library implementors. Yes, they are often one in the same.
That was generally true in C. In the C++ world, however, compiler
writers and library writers form sets that are almost completely
disjoint.
> After you
> posted, I began thinking about why there hasn't been as much contribution
> as one might expect to something as important as the C++ Standard Library.
I stand in awe of your persistent arrogance.
> There /are/ several freely available I/O libraries for various kinds.
> Some
> of them look very nice, and from experience with software that uses them,
> some of the are very reliable, and powerful. The problem, I suspect, is
> the licensing conflicts with the conditions of the C++ Standard.
Oh, for heaven's sake. There are *no* licensing conflicts imposed
by any "conditions" in the C++ Standard. Please learn *something*
about what you're talking about before opening your mouth.
>> But you can write, or initiate a project to write, such libraries and
>> propose it for inclusion in the standard.
>
> Actually, what probably should be done is a much more difficult task.
> That
> is to identify the abstract interfaces which effectively capture the
> common
> aspects of the vast majority of existing, and probable implementation of
> any given design domain such as local harddrive I/O. Also worth
> considering is the further abstraction of the different I/O interfaces
> such
> as harddrive, IPC, network, in order to create a baseclass for all I/O
> instances. I'm really not talking about the existing std::ios_base, or
> basic_ios. They may prove to provide some element of that interface, but
> I
> am considering starting from tabular rasa.
>
> These should be interfaces, sans data, or implemented functions.
That pronouncement was so abstract and mealy mouthed that it could
be applied to damn near anything. It certainly described what went
into the design of every programming language I/O subsystem since
Fortran II. It could almost be applied to a project for making
universal mounting hardware for disk drives. Pure middle-management
speak.
Alf P. Steinbach
> Alf P. Steinbach wrote:
> > * Steven T. Hatton:
> >> Dietmar Kuehl wrote:
> >> > std::ifstream in("file", std::ios_base::binary);
> >> > std::vector<unsigned char> v((std::istreambuf_iterator<char>(in)),
> >> > std::istreambuf_iterator<char>());
>
> >> Won't that eat whitespaces?
>
> It won't.
Right, that's already covered.
Btw., the quoting is a bit off; this is the second time in this thread
something seems to be attributed to me that someone else wrote.
[lengthy diatribe snipped]
> You mean, we should embrace Java I/O as an alternative in C++ and the
> Java code above is *not* putting students off while the C++ code is?
> Well, I might have to readjust my measure of complexity then...
Dietmar, you make me laugh. :-)
Admit it, you _are_ a bit religious about these matters, aren't you?
Anyway, no, I'm not into Java.
Julián Albo
>>> The reasons probably are: some people must write and standarize it.
>>> Compiler and library writers must implement it. And probably the people
>>> that can benefit from it will keep prefering other languages instead of
>>> C++ after all.
>> Technically speaking, it is not the perview of compiler writers, but
>> ratherthe library implementors. Yes, they are often one in the same.
> That was generally true in C. In the C++ world, however, compiler
> writers and library writers form sets that are almost completely
> disjoint.
But event in that case, compiler writers may have more work because of the
desire to optimize better the code generated by some uses of the library.
--
Salu2
Dietmar Kuehl
> * Dietmar Kuehl:
>> Alf P. Steinbach wrote:
>> > * Steven T. Hatton:
>>
>> It won't.
>
> Right, that's already covered.
>
> Btw., the quoting is a bit off; this is the second time in this thread
> something seems to be attributed to me that someone else wrote.
Actually I don't think so: the question is attributed to Steven,
not to you. Stuff attributed to you used '>' as a prefix while
stuff attributed to Steven used '>>'. Maybe there is a space
missing for the interpretation for some newsreaders but then I'm
not aware of any strict quoting rules specifying prefixes...
>> You mean, we should embrace Java I/O as an alternative in C++ and the
>> Java code above is *not* putting students off while the C++ code is?
>> Well, I might have to readjust my measure of complexity then...
>
> Dietmar, you make me laugh. :-)
OK.
> Admit it, you _are_ a bit religious about these matters, aren't you?
I don't think I'm religious about these matters because, at least
to me, being religious would include that I cannot accept changing
my believe. I don't think this is the case: if there are convincing
arguments to change my perspective, I'm willing to do so. However,
I have worked with I/O libraries in other languages than C++ and
without other I/O libraries than IOStreams in C++ and I have no
reason to think that there are better alternatives except Perl's
I/O processing with its integration with regular expression. Of
course, this view is influenced by the fact that I'm mostly either
accessing databases or text files. For example, I don't normally
process binary formatted files or Java serialized data where the
latter is, of course, a pretty useless format anyway which
arrogantly assumes that everybody is using Java, not to mention
specific sets of classes.
> Anyway, no, I'm not into Java.
I'm not too much into Java either but I have a certain view of
the language and did some work with it. This is very my evaluation
comes from. It neither comes from ignorance nor from religious
believes - and I stay with what I have said: its I/O library is
crap.
Steven T. Hatton
> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
> news:oIOdnQdTz_K...@speakeasy.net...
>
>> After you posted, I began thinking about why there hasn't been as much
>> contribution as one might expect to something as important as the C++
>> Standard Library.
[...]
I am not the subject of this newsgroup.
>> There /are/ several freely available I/O libraries for various kinds.
>> Some
>> of them look very nice, and from experience with software that uses them,
>> some of the are very reliable, and powerful. The problem, I suspect, is
>> the licensing conflicts with the conditions of the C++ Standard.
>
> Oh, for heaven's sake. There are *no* licensing conflicts imposed
> by any "conditions" in the C++ Standard. Please learn *something*
> about what you're talking about before opening your mouth.
Uh, if GPL code were used in the C++ Standard Library, it could seriously
incumber the use of the library for people who wanted to write proprietary
software. That is probably why the Boost licensing is worded quite
differently.
>>> But you can write, or initiate a project to write, such libraries and
>>> propose it for inclusion in the standard.
>>
>> Actually, what probably should be done is a much more difficult task.
>> That
>> is to identify the abstract interfaces which effectively capture the
>> common
>> aspects of the vast majority of existing, and probable implementation of
>> any given design domain such as local harddrive I/O. Also worth
>> considering is the further abstraction of the different I/O interfaces
>> such
>> as harddrive, IPC, network, in order to create a baseclass for all I/O
>> instances. I'm really not talking about the existing std::ios_base, or
>> basic_ios. They may prove to provide some element of that interface, but
>> I
>> am considering starting from tabular rasa.
>>
>> These should be interfaces, sans data, or implemented functions.
>
> That pronouncement was so abstract and mealy mouthed that it could
> be applied to damn near anything.
Rather than insulting me, why don't you either provide convincing evidence
to demonstrate my error, or ask for clarification? I actually did come up
with some ideas of what these things might mean. In the above statement,
it was very much my intent to be abstract, and as abstract as possible.
That is exactly where the realization of the design of something as
foundational as the C++ I/O library should begin.
Certainly, the first stage of design is identifying some sense of design
domain. When that is identified, then you try to identify the common
feature that all instances of the abstract type will share. That will lead
to identifying explicitly belongs in the interface. It may also lead to
identifying what else belongs in the abstraction class. It's likely that
will come to realize that some of the original candidates for inclusion are
not appropriately grouped in your abstraction class.
When such candidates are removed from the abstraction class, any unique
contribution they made to the interface should likewise be removed. This
is one place where the work gets difficult. It means reexamining all the
other members of the abstraction class to determine what aspects of the
excluded candidate they might have in common. It also means the
possibility of opening the door to candidates which did not meet the
inclusion requirements imposed by the excluded candidate.
As for "it could be applied to damn near anything."; in Unix, what is a
file?
> It certainly described what went
> into the design of every programming language I/O subsystem since
> Fortran II. It could almost be applied to a project for making
> universal mounting hardware for disk drives.
IOW, I described the tried and true approach.
P.J. Plauger
> P.J. Plauger wrote:
>
>> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
>> news:oIOdnQdTz_K...@speakeasy.net...
>>
>
>>> After you posted, I began thinking about why there hasn't been as much
>>> contribution as one might expect to something as important as the C++
>>> Standard Library.
> [...]
>
> I am not the subject of this newsgroup.
No, but you became the subject of this posting when you opined
(rather arrogantly) in the first person.
>>> There /are/ several freely available I/O libraries for various kinds.
>>> Some
>>> of them look very nice, and from experience with software that uses
>>> them,
>>> some of the are very reliable, and powerful. The problem, I suspect, is
>>> the licensing conflicts with the conditions of the C++ Standard.
>>
>> Oh, for heaven's sake. There are *no* licensing conflicts imposed
>> by any "conditions" in the C++ Standard. Please learn *something*
>> about what you're talking about before opening your mouth.
>
> Uh, if GPL code were used in the C++ Standard Library, it could seriously
> incumber the use of the library for people who wanted to write proprietary
> software. That is probably why the Boost licensing is worded quite
> differently.
You were talking about an international *standard*, not a
specific implementation. There is a difference, you know.
>>>> But you can write, or initiate a project to write, such libraries and
>>>> propose it for inclusion in the standard.
>>>
>>> Actually, what probably should be done is a much more difficult task.
>>> That
>>> is to identify the abstract interfaces which effectively capture the
>>> common
>>> aspects of the vast majority of existing, and probable implementation of
>>> any given design domain such as local harddrive I/O. Also worth
>>> considering is the further abstraction of the different I/O interfaces
>>> such
>>> as harddrive, IPC, network, in order to create a baseclass for all I/O
>>> instances. I'm really not talking about the existing std::ios_base, or
>>> basic_ios. They may prove to provide some element of that interface,
>>> but
>>> I
>>> am considering starting from tabular rasa.
>>>
>>> These should be interfaces, sans data, or implemented functions.
>>
>> That pronouncement was so abstract and mealy mouthed that it could
>> be applied to damn near anything.
>
> Rather than insulting me, why don't you either provide convincing evidence
> to demonstrate my error, or ask for clarification?
I didn't intend to be insulting, merely descriptive. I gave a
specific example -- that the words could apply to any programming
language since Fortran. I don't expect ever to convince you of
anything, however. My goal was merely to put your pronunciamento
in context for other readers.
In this particular case, the words are way too vague to demonstrate
any error, or offer any hope of clarification. As Enrico Fermi
once famously said of a weak theory, "it isn't even wrong."
> I actually did come up
> with some ideas of what these things might mean. In the above statement,
> it was very much my intent to be abstract, and as abstract as possible.
> That is exactly where the realization of the design of something as
> foundational as the C++ I/O library should begin.
>
> Certainly, the first stage of design is identifying some sense of design
> domain. When that is identified, then you try to identify the common
> feature that all instances of the abstract type will share. That will
> lead
> to identifying explicitly belongs in the interface. It may also lead to
> identifying what else belongs in the abstraction class. It's likely that
> will come to realize that some of the original candidates for inclusion
> are
> not appropriately grouped in your abstraction class.
>
> When such candidates are removed from the abstraction class, any unique
> contribution they made to the interface should likewise be removed. This
> is one place where the work gets difficult. It means reexamining all the
> other members of the abstraction class to determine what aspects of the
> excluded candidate they might have in common. It also means the
> possibility of opening the door to candidates which did not meet the
> inclusion requirements imposed by the excluded candidate.
Do you even realize how vacuous this sort of drivel appears to
others? You could be giving a Powerpoint presentation to investment
bankers on software for dry cleaning establishments, and not have
to change a single blessed word in the last two paragraphs.
> As for "it could be applied to damn near anything."; in Unix, what is a
> file?
I use one to knock the burrs off metal castings, myself.
>> It certainly described what went
>> into the design of every programming language I/O subsystem since
>> Fortran II. It could almost be applied to a project for making
>> universal mounting hardware for disk drives.
>
> IOW, I described the tried and true approach.
Well, yes, in such broad terms that you offer absolutely no
guidance. For another apt quote, I'm reminded of a Brazilian
singer who put down another band's music by saying, "It don't
smell good; it don't smell bad; it just don't smell."
Steven T. Hatton
> Alf P. Steinbach wrote:
>> * Steven T. Hatton:
>>> Dietmar Kuehl wrote:
>>> > std::ifstream in("file", std::ios_base::binary);
>>> > std::vector<unsigned char> v((std::istreambuf_iterator<char>(in)),
>>> > std::istreambuf_iterator<char>());
>
>>> Won't that eat whitespaces?
> It won't. Stream buffer iterators operate on the character level and
> don't use any form of formatting, neither when reading nor when
> writing.
Sorry. I had missed the fact that you were using a istrembuf_iterator
instead of a istream_iterator.
Steven T. Hatton
> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
> news:wKCdndVzyOj...@speakeasy.net...
>
>> Uh, if GPL code were used in the C++ Standard Library, it could seriously
>> incumber the use of the library for people who wanted to write
>> proprietary software. That is probably why the Boost licensing is worded
>> quite differently.
>
> You were talking about an international *standard*, not a
> specific implementation. There is a difference, you know.
Yes, but the Standard does contain code. If that code were taken directly
from a GPL product such as Socket++, there could be problems.
>> Rather than insulting me, why don't you either provide convincing
>> evidence to demonstrate my error, or ask for clarification?
>
> I didn't intend to be insulting, merely descriptive.
hua.
> I gave a
> specific example -- that the words could apply to any programming
> language since Fortran. I don't expect ever to convince you of
> anything, however. My goal was merely to put your pronunciamento
> in context for other readers.
>
> In this particular case, the words are way too vague to demonstrate
> any error, or offer any hope of clarification. As Enrico Fermi
> once famously said of a weak theory, "it isn't even wrong."
And then you turned around and said that my approach was the same as has
been taken for 3 decades.
> Do you even realize how vacuous this sort of drivel appears to
> others? You could be giving a Powerpoint presentation to investment
> bankers on software for dry cleaning establishments, and not have
> to change a single blessed word in the last two paragraphs.
If you don't understand, simply ask for clarification, or an example. I can
easily provide both. For an example, see the discussion in TC++PL(SE) in
chapter 12. (IIRC)
[...]
Ráðumk þér, Loddfáfnir,
en þú ráð nemir, -
njóta mundu, ef þú nemr,
þér munu góð, ef þú getr -:
orðum skipta
þú skalt aldregi
við ósvinna apa,
Steven T. Hatton
>> Anyway, no, I'm not into Java.
>
> I'm not too much into Java either but I have a certain view of
> the language and did some work with it. This is very my evaluation
> comes from. It neither comes from ignorance nor from religious
> believes - and I stay with what I have said: its I/O library is
> crap.
The adjective used indicates to me that you are not thinking objectively.
And your assessment of Java's I/O is not helpful. It neither informs us as
to what you don't like about the Java I/O library, nor does it tell us what
feature of the C++ I/O library you find superior. I suspect a person with
your level of expertise could provide valuable insight. I'd be interested
to know the technical details of your assessment.
Steven T. Hatton
> So do you think that the codes of whitespace characters are not binary?
> Binary, as ther people say in this thread several times, is related only
> to new line conventions and end of file detection.
Yes, that is, more or less what it means, in C terminology which is implied
in the C++ Standard Library. It is by no means a common use of the meaning
of "binary I/O" in other contexts.
> You have mixed the
> concept of text/binary mode with the formatted/unformatted input, and you
> blame the language for that?
Since whitespace is only meaningfull to me in terms of text, that was a
reasonable assumption, even though it does not reflect the design of the
I/O library. The documentation I was reading said this, which I believe is
misleading:
<quote>
If mode & ios_base::binary is nonzero, the function appends b to strmode to
open a binary stream instead of a text stream."
...
"A binary stream consists of one or more bytes of arbitrary information. You
can write the value stored in an arbitrary object to a (byte-oriented)
binary stream and read exactly what was stored in the object when you wrote
it. The library functions do not alter the bytes you transmit between the
program and a binary stream. They can, however, append an arbitrary number
of null bytes to the file that you write with a binary stream. The program
must deal with these additional null bytes at the end of any binary
stream."
</quote>
It then goes on to describe how to control streams using the C functions:
<quote>
Controlling Streams: fopen returns the address of an object of type FILE.
You use this address as the stream argument to several library functions to
perform various operations on an open file. For a byte stream, all input
takes place as if each character is read by calling fgetc, and all output
takes place as if each character is written by calling fputc. For a wide
stream (with Amendment 1), all input takes place as if each character is
read by calling fgetwc, and all output takes place as if each character is
written by calling fputwc.
...
Functions in the first three groups are declared in <stdio.h>:
* the byte read functions -- fgetc, fgets, fread, fscanf, getc, getchar,
gets, scanf, and ungetc
* the byte write functions -- fprintf, fputc, fputs, fwrite, printf,
putc, putchar, puts, vfprintf, and vprintf
* the position functions -- fflush, fseek, fsetpos, and rewind
...
</quote>
The use of <stdio.h> shows us that the author is really thinking in C terms,
not in C++ terms. This is, however, documentation intended for C++.
>> A big ol' stream.set_unformatted_binary(), would have saved me a lot of
>> time.
>
> Learning the ways the language already has to do things instead of
> expecting that ways more likely to you are written will save you a lot
> more.
Well, it doesn't help that Stroustrup sidestepped the issue of binary I/O in
TC++PL(SE). As you can see above, there are problems with another commonly
recommended source of documentation for the C++ Standard Library.
I'm now beginning to realize much of my confusion comes from distinguishing
between the I/O streams and the the I/O stream buffers.
Julián Albo
>> Binary, as ther people say in this thread several times, is related only
>> to new line conventions and end of file detection.
> Yes, that is, more or less what it means, in C terminology which is
> implied in the C++ Standard Library. It is by no means a common use
> of the meaning of "binary I/O" in other contexts.
Maybe, but when reading C++ documentation you must suppose that the C++
sense is used by default.
> As you can see above, there are problems with another commonly recommended
> source of documentation for the C++ Standard Library.
Probably they are written under the assumption that people that reads it
already knows and have some experience with the fundamental concepts. If
you jump to it under knowledge and assumptions based in experience with
other languages, skipping the basic steps, you are in trouble.
--
Salu2
Dietmar Kuehl
> Dietmar Kuehl wrote:
[talking about Java I/O]
>> - and I stay with what I have said: its I/O library is crap.
>
> The adjective used indicates to me that you are not thinking objectively.
You mean, I would have used a stronger term if I were objective? Yes,
you are probably right...
> And your assessment of Java's I/O is not helpful. It neither informs us
> as to what you don't like about the Java I/O library, nor does it tell us
> what feature of the C++ I/O library you find superior.
Well, I think I gave a fairly good basis to decide even for yourself
in the article where I posted C++ and Java code side by side: just
have a look at the code and decide for yourself. As I said in this
article, I think I did the best possible in Java but I would like to
be illuminated in case I missed Java functionality which does text
parsing more easily. Since I'm very fluent using C++ streams, the
comparison may not be fair because I may have missed something in
Java which I didn't do in C++. Of course, you will need to remove
a fair amount of sarcasm if you want to understand my thinking.
> I suspect a person with your level of expertise could provide
> valuable insight. I'd be interested to know the technical details
> of your assessment.
Since I already posted the core issue I have with Java I/O your
interest cannot be that big: you apparently missed it altough you
wrote an answer to the corresponding artice, albeit to a different
part of it.
Dietmar Kuehl
> P.J. Plauger wrote:
>> Oh, for heaven's sake. There are *no* licensing conflicts imposed
>> by any "conditions" in the C++ Standard. Please learn *something*
>> about what you're talking about before opening your mouth.
>
> Uh, if GPL code were used in the C++ Standard Library, it could seriously
> incumber the use of the library for people who wanted to write proprietary
> software. That is probably why the Boost licensing is worded quite
> differently.
I second P.J.Plauger's advice! The standard C++ library is implemented
multiple times, including several commercial versions (at least three), at
least one GPL version, and at least one "free" version (although none is
public domain as far as I can tell). There is no licensing issue at all
with the standard C++ library because the standard is not an
implementation but a specification. The specification itself has
licensing restriction but there are no restrictions on implementing this
specification.
Steven T. Hatton
> Steven T. Hatton wrote:
>>> What I would like to know is how to get in iterator over a buffer such
>>> as std::basic_filebuf, so that I can do something like:
>
> You can't and you shouldn't. Actually, you would be well advised
> to read the documentation of classes before guessing about their
> semantics: if at all, you want to use 'gptr()' as your starting
> point because the sequence [eback(), gptr()) consists of already
> processed characters (... and your wrong use clearly shows that
> you had no inkling about what you are doing).
This is what I wrote:
copy(file_buf.eback(), file_buf.egptr(), back_inserter(data));
I'm not sure why you say that is fundamentally wrong. I'm assuming
eback()==gptr() when the iteration begins. I'm looking at page 676 of
TC++SL.
>>> copy(istream_iterator<unsigned char>(file_buf.eback())
>>> , istream_iterator<unsigned char>(file_buf.egptr())
>>> , back_inserter(data));
>>>
>>> Which I can't do without inheriting from the buffer. That makes binary
>>> I/O seem inconsistent with the rest of the library.
>
> No, it doesn't.
How it /seems/, and how it actually /is/, may not be consistent. But you
are not in a position to determine how things seem to me.
> You should operate on top of the stream buffer
> abstraction, not within.
That is what I had wanted to do, but I didn't see a way of getting an
iterator representing egptr(). I understand that, in the general case of
I/O streams, egptr() and the end of the input stream are not necessarily
the same thing, and therefore the analog to container.end() is not as easy
to express.
The one mistake I made was not using the best resources I have available.
§13.13.2
> Maybe you should at least try to
> understand what the classes and functions are about and for.
This is a difficult thing to understand. Partly because it preserves some
very ancient, and less than elegant aspects, and partly because there are
multiple dimensions to the conceptual structure of the I/O system. There
is the inheritance hierarchy of the basic_ios class templates shown here:
http://baldur.globalsymmetry.com/~hattons/Standard/doc/html/inherit__graph__35.png
(I still have a lot of work to do on those files, and the ones on the server
are not the latest.)
There is a dimension of stream buffers which has its own inheritance
hierarchy.
There is a dimension of template arguments for the character type, and
character traits, which can impact both the streams and stream buffers.
One thing I find messy about the C++ I/O library is the use of format flags.
The following is on idea I have come up with which may have some
advantages:
class ios_base {
public:
static struct {
static const fmtflags boolalpha;
static const fmtflags dec;
static const fmtflags fixed;
static const fmtflags hex;
static const fmtflags internal;
static const fmtflags left;
static const fmtflags oct;
static const fmtflags right;
static const fmtflags scientific;
static const fmtflags showbase;
static const fmtflags showpoint;
static const fmtflags showpos;
static const fmtflags skipws;
static const fmtflags unitbuf;
static const fmtflags uppercase;
static const fmtflags adjustfield;
static const fmtflags basefield;
static const fmtflags floatfield;
} format_flags;
static struct {
static const openmode app;
static const openmode ate;
static const openmode binary;
static const openmode in;
static const openmode out;
static const openmode trunc;
} openmode_flags;
static namespace {
static const iostate badbit;
static const iostate eofbit;
static const iostate failbit;
static const iostate goodbit;
} state_flags;
static struct {
static const seekdir beg;
static const seekdir cur;
static const seekdir end;
} seekdir_flags;
};
Steven T. Hatton
> Steven T. Hatton wrote:
>> P.J. Plauger wrote:
>>> Oh, for heaven's sake. There are *no* licensing conflicts imposed
>>> by any "conditions" in the C++ Standard. Please learn *something*
>>> about what you're talking about before opening your mouth.
>>
>> Uh, if GPL code were used in the C++ Standard Library, it could seriously
>> incumber the use of the library for people who wanted to write
>> proprietary
>> software. That is probably why the Boost licensing is worded quite
>> differently.
>
> I second P.J.Plauger's advice! The standard C++ library is implemented
> multiple times, including several commercial versions (at least three), at
> least one GPL version, and at least one "free" version (although none is
> public domain as far as I can tell). There is no licensing issue at all
> with the standard C++ library because the standard is not an
> implementation but a specification. The specification itself has
> licensing restriction but there are no restrictions on implementing this
> specification.
What I'm talking about is the other way around. If you simply took
something such as http://www.alhem.net/Sockets/index.html and, say, for
example, copied its header files and used them as a basis for the
specification, there could be legal problems. What you are talking about
are products based on the Standard, not a standard based on the products.
Dietmar Kuehl
> This is what I wrote:
I know what you wrote.
> copy(file_buf.eback(), file_buf.egptr(), back_inserter(data));
>
> I'm not sure why you say that is fundamentally wrong.
Have you considered the possibility that it is fundamtentally wrong?
First of all, you are dealing with stream buffer internals which you
are only supposed to access when implementing stream buffers, not
when using them. Second, as I said before, the sequence
[eback(), gptr()) consists of characters which have already been read.
Finally, the sequence [gptr(), egptr()) is actually an internal of the
stream buffer consisting essentially of a portion of the file. This is
a level of abstraction you don't want to operate on when using a stream.
It is the right level of abstraction, however, for reading from an
external source.
> I'm assuming eback()==gptr() when the iteration begins.
Say what you mean, mean what you say! Even if the assumption may be true
at the beginning of the file, it is source obfuscation at the best to
use the wrong pointer which may happen by accident to be identical to
the correct pointer at some point.
> I'm looking at page 676 of TC++SL.
I guess you are talking of Nico's book in which case I would have written
or translated the IOStream portion. I don't have the book at hand and
thus I can't check what is said on page 676. However, I doubt that it
states that eback() is any good for a stream buffer user or that it is
the right thing to get the start of a buffer.
In addition, it the wrong tag to the problem anyway: to get an iterator
for a stream buffer you use 'std::istreambuf_iterator<...>'.
> How it /seems/, and how it actually /is/, may not be consistent. But you
> are not in a position to determine how things seem to me.
Right, I forgot that you keep phrasing your statements vague up to the
point of being meaningless. Have you considered becoming a politician?
These peoples job is to talk nonsense the whole day to entertain the
public with the search of any concrete statement the politicians cannot
worm their way out.
>> You should operate on top of the stream buffer
>> abstraction, not within.
>
> That is what I had wanted to do, but I didn't see a way of getting an
> iterator representing egptr(). I understand that, in the general case of
> I/O streams, egptr() and the end of the input stream are not necessarily
> the same thing, and therefore the analog to container.end() is not as easy
> to express.
I think uttering 'std::istreambuf_iterator<char>()' does not take that
much effort or intellectual capacity...
> The one mistake I made was not using the best resources I have available.
> §13.13.2
I'd say that
Expression Effect
streambuf _iterator<char>() Creates an end-of-stream iterator
in the mentioned paragraph is a pretty clear statement of how to get
the end iterator for a stream buffer... (I have an electronic version
of the book and thus can check for paragraphs but not for page numbers).
> This is a difficult thing to understand. Partly because it preserves some
> very ancient, and less than elegant aspects, and partly because there are
> multiple dimensions to the conceptual structure of the I/O system. There
> is the inheritance hierarchy of the basic_ios class templates shown here:
>
http://baldur.globalsymmetry.com/~hattons/Standard/doc/html/inherit__graph__35.png
This is a broken link. Anyway, you don't want to educate me about the
IOStreams or stream buffer class hierarchies, do you? I know pretty
well how the C++ I/O mechanisms work, you know. ... and I think they
are pretty simple to use once you have accepted the basic ideas of
using extractors and inserters and/or the use of sequences. For most
of the normal work you don't have to understand these hierarchies at
all! You just operate on an 'std::istream' or an 'std::ostream' - you
can even get a 'std::istreambuf_iterator' with an 'std::istream'.
> One thing I find messy about the C++ I/O library is the use of format
> flags. The following is on idea I have come up with which may have some
> advantages:
You pasted some code there, I didn't see any idea not to mention anything
having advantages (well, you didn't state what it would have advantages
over...).
Dietmar Kuehl
> What I'm talking about is the other way around. If you simply took
> something such as http://www.alhem.net/Sockets/index.html and, say, for
> example, copied its header files and used them as a basis for the
> specification, there could be legal problems. What you are talking about
> are products based on the Standard, not a standard based on the products.
My understanding is that interfaces cannot be copyrighted. The actual file
can, the semantics cannot. That is, if someone specified a library having
the same interface and semantics as some other library, using his own
words for the description, this would not be a copyright infringement.
However, "taking" something is not how the C++ committee operates, anyway.
The details of how the C++ committee operates were discussed publically in
the past. That is, you could have had all necessary information but you
decided to make insulting statements instead.
Steven T. Hatton
> Steven T. Hatton wrote:
>> This is what I wrote:
>
> I know what you wrote.
>
>> copy(file_buf.eback(), file_buf.egptr(), back_inserter(data));
>>
>> I'm not sure why you say that is fundamentally wrong.
>
> Have you considered the possibility that it is fundamtentally wrong?
> First of all, you are dealing with stream buffer internals which you
> are only supposed to access when implementing stream buffers, not
> when using them. Second, as I said before, the sequence
> [eback(), gptr()) consists of characters which have already been read.
> Finally, the sequence [gptr(), egptr()) is actually an internal of the
> stream buffer consisting essentially of a portion of the file. This is
> a level of abstraction you don't want to operate on when using a stream.
> It is the right level of abstraction, however, for reading from an
> external source.
Recall that when I described the objective that I said the fact that I
needed to inherit from the stream buffer seemed wrong. I was only asking
about the range of data I specified. Nothing more.
>> I'm assuming eback()==gptr() when the iteration begins.
>
> Say what you mean, mean what you say! Even if the assumption may be true
> at the beginning of the file, it is source obfuscation at the best to
> use the wrong pointer which may happen by accident to be identical to
> the correct pointer at some point.
All I had intended by my example was to indicate the range of data I wanted
access to. One of the more difficult things about using the STL model is
that the association between iterators, and the objects they are designed
to work in conjunction with is not appearant from looking at the object.
This is one of the advantages to object oriented programming. The
functions that have access to member data are present in the class
interface. In the case of std::streambuf<> I cannot simply look at the
interface and see that std::istreambuf_iterator<> is available in the
library. The problem represents the downside of loose coupling. In Java,
iterators are typically provided by a factory method of the container.
This is somewhat similar to the STL containers that provide iterator
typedefs as well as begin() and end() as part of their definition.
In the US we have a traditional children's story called _Hansel and Gretel_,
about two children who went for walk in the woods. They were afraid they
would get lost so the left a trail of breadcrumbs. We often use that as a
metaphor for something done intentionally so that a person can follow an
otherwise obscure path. That's what having the iterator in the interface
of the collection provides, a trail of breadcrumbs that tells you there is
an iterator available for that class.
Now consider a typical STL algorithm. You cannot look at an STL container
and see which algorithms are associated with it. Of course, once you
become familiar with the library that isn't that much of a problem. The
problem is that it takes longer to learn the library because it's harder to
find the information and how the different parts fit together. That is
what was missing from std::streambuf<>.
>> I'm looking at page 676 of TC++SL.
>
> I guess you are talking of Nico's book in which case I would have written
> or translated the IOStream portion. I don't have the book at hand and
> thus I can't check what is said on page 676. However, I doubt that it
> states that eback() is any good for a stream buffer user or that it is
> the right thing to get the start of a buffer.
No, it merely shows where the beginning of the available data is. Figure
13.5 and associated text.
> In addition, it the wrong tag to the problem anyway: to get an iterator
> for a stream buffer you use 'std::istreambuf_iterator<...>'.
Yes. I now know that. Thank you for posting that example yesterday.
>> How it /seems/, and how it actually /is/, may not be consistent. But you
>> are not in a position to determine how things seem to me.
>
> Right, I forgot that you keep phrasing your statements vague up to the
> point of being meaningless. Have you considered becoming a politician?
> These peoples job is to talk nonsense the whole day to entertain the
> public with the search of any concrete statement the politicians cannot
> worm their way out.
Perhaps you are not finding meaning in what I am saying because you believe
my objective are different than they are. I'm trying to explain to you
something that you cannot see. That is, what the current C++ looks like to
a person who is learning it. I believe that is a valuable perspective for
a person involved in design to understand.
>>> You should operate on top of the stream buffer
>>> abstraction, not within.
>>
>> That is what I had wanted to do, but I didn't see a way of getting an
>> iterator representing egptr(). I understand that, in the general case of
>> I/O streams, egptr() and the end of the input stream are not necessarily
>> the same thing, and therefore the analog to container.end() is not as
>> easy to express.
>
> I think uttering 'std::istreambuf_iterator<char>()' does not take that
> much effort or intellectual capacity...
See above. Also consider that it is an easy pitfall to try and use
std::istream_iterator<> without changing the format state of the stream to
skip whitespace.
>> The one mistake I made was not using the best resources I have available.
>> §13.13.2
>
> I'd say that
>
> Expression Effect
> streambuf _iterator<char>() Creates an end-of-stream iterator
>
> in the mentioned paragraph is a pretty clear statement of how to get
> the end iterator for a stream buffer... (I have an electronic version
> of the book and thus can check for paragraphs but not for page numbers).
>
>> This is a difficult thing to understand. Partly because it preserves
>> some very ancient, and less than elegant aspects, and partly because
>> there are
>> multiple dimensions to the conceptual structure of the I/O system. There
>> is the inheritance hierarchy of the basic_ios class templates shown here:
>>
>
http://baldur.globalsymmetry.com/~hattons/Standard/doc/html/inherit__graph__35.png
I trust you were able to go to the parent directory and find the doxygen
output. Sorry about that, I decided to update the files and forgot about
the consequence it might have on the file names.
This is basically the same diagram:
http://gcc.gnu.org/onlinedocs/libstdc++/latest-doxygen/inherit__graph__98.png
>
> This is a broken link. Anyway, you don't want to educate me about the
> IOStreams or stream buffer class hierarchies, do you? I know pretty
> well how the C++ I/O mechanisms work, you know. ... and I think they
> are pretty simple to use once you have accepted the basic ideas of
> using extractors and inserters and/or the use of sequences.
And knowing which iterators are available, as well as what flags to use to
modify the behavior of the stream. Also relevant is the distinction
between formatted and unformatted operations. It is well advised also to
understand that unformatted streams may still alter the underlying data,
and that formatted operations may alter the data in a so-called "binary"
stream. It is easy to put the stream into "stupid" format states where the
wrong combination of flags is set causing the numeric format to behave as
if the default were set. strm.clear(flag) unsets everything _but_ flag.
strm.setf(flags) doesn't mean `set flags' even though that's what it does.
Then there is 'basic_ios' verses 'ios_base'. std::string::clear() does
something completely different than std::stringstream::clear(). Reading a
stream completely and correctly puts it into a fail state.
Then there are the necessary complexities of the design to consider.
> For most
> of the normal work you don't have to understand these hierarchies at
> all! You just operate on an 'std::istream' or an 'std::ostream' - you
> can even get a 'std::istreambuf_iterator' with an 'std::istream'.
So what happens when I have a C-style file pointer returned from a library?
Can I use that with C++ I/O?
>> One thing I find messy about the C++ I/O library is the use of format
>> flags. The following is on idea I have come up with which may have some
>> advantages:
>
> You pasted some code there, I didn't see any idea not to mention anything
> having advantages (well, you didn't state what it would have advantages
> over...).
The advantage to that code is simple. It provides support for code
completion. I now realize it can be improved by subgroupping flags
according to their related fields. It's a very similar concept to scoped
enumerators. The biggest drawback to adding anything like that is that
there are already too many ways to do the same thing. It would add clutter
to an already cluttered interface. There are several things that could be
done with the basic concept that would provide a more usable interface than
the existing one. Unfortunately, the existing interface will remain, no
matter what.
Julián Albo
> access to. One of the more difficult things about using the STL model is
> that the association between iterators, and the objects they are designed
> to work in conjunction with is not appearant from looking at the object.
> This is one of the advantages to object oriented programming. The
> functions that have access to member data are present in the class
> interface. In the case of std::streambuf<> I cannot simply look at the
> interface and see that std::istreambuf_iterator<> is available in the
> library. The problem represents the downside of loose coupling. In Java,
> iterators are typically provided by a factory method of the container.
> This is somewhat similar to the STL containers that provide iterator
> typedefs as well as begin() and end() as part of their definition.
And what is the contaniner in that case? You seem to think that the iterator
is in some way part of the iterated thing, and that is not always
neccessary true. You want that when any wants to write a new type of
iteration over some thing, that thing must be modified? For the only
benefit that you can know all iterations avaliable by looking at his
header? Sorry, no. I will not use a language with design guidelines like
that.
I want a language written under the assumption that programmers can read
documentation. Perhpas this is that Stroustrup refers to when he says
"serious programmer"? ;)
--
Salu2
Steven T. Hatton
> Steven T. Hatton wrote:
>> What I'm talking about is the other way around. If you simply took
>> something such as http://www.alhem.net/Sockets/index.html and, say, for
>> example, copied its header files and used them as a basis for the
>> specification, there could be legal problems. What you are talking about
>> are products based on the Standard, not a standard based on the products.
>
> My understanding is that interfaces cannot be copyrighted. The actual file
> can, the semantics cannot. That is, if someone specified a library having
> the same interface and semantics as some other library, using his own
> words for the description, this would not be a copyright infringement.
>
> However, "taking" something is not how the C++ committee operates, anyway.
> The details of how the C++ committee operates were discussed publically in
> the past. That is, you could have had all necessary information but you
> decided to make insulting statements instead.
It was not at all intended to be insulting. I am baffled about how you could
find that insulting. I was simply thinking about this statement by the
inventor of the I/O streams:
"A major goal in my original design was that it be extensible in interesting
ways. In particular, in the stream library the streambuf class was an
implementation detail, but in the iostream library I intended it to be a
usable class in its own right. I was hoping for the promulgation of many
streambufs with varied functionality. I wrote a few myself, but almost no
one else did. I answered many more questions of the form "how do I make my
numbers look like this" than "how do I write a streambuf". And textbook
authors also tended to ignore streambufs. Apparently they did not share my
view that the architecture of the input/output library was an interesting
case study."
I realized that there are a lot of things available that people have open
sourced. I tried to think of a reason why there wasn't that kind of
contribution to the C++ Standard Library. What I was really suggesting is
that some of the people who had created these open source libraries would
be happy to have the used, but there might be legal issue that precluded
that.
You are looking for insults where there are none. When I insult you, you
will know it. Trust me.
Steven T. Hatton
> Steven T. Hatton wrote:
>
>> access to. One of the more difficult things about using the STL model is
>> that the association between iterators, and the objects they are designed
>> to work in conjunction with is not appearant from looking at the object.
>> This is one of the advantages to object oriented programming. The
>> functions that have access to member data are present in the class
>> interface. In the case of std::streambuf<> I cannot simply look at the
>> interface and see that std::istreambuf_iterator<> is available in the
>> library. The problem represents the downside of loose coupling. In
>> Java, iterators are typically provided by a factory method of the
>> container. This is somewhat similar to the STL containers that provide
>> iterator typedefs as well as begin() and end() as part of their
>> definition.
>
> And what is the contaniner in that case? You seem to think that the
> iterator is in some way part of the iterated thing, and that is not always
> neccessary true. You want that when any wants to write a new type of
> iteration over some thing, that thing must be modified? For the only
> benefit that you can know all iterations avaliable by looking at his
> header? Sorry, no. I will not use a language with design guidelines like
> that.
You seem to be confused between my making an observation with the objective
of identifying problem areas so that improvements might be made, and the
false idea that I expect absolute adherence to some design principle. I
was stating facts as I see them.
> I want a language written under the assumption that programmers can read
> documentation. Perhpas this is that Stroustrup refers to when he says
> "serious programmer"? ;)
"Let me outline the program development environment I'd like for C++. First
of all, I want incremental compilation. When I make a minor change, I want
the ``system'' to note that it was minor, and have the new version compiled
and ready to run in a second. Similarly, I want simple requests, such as
``Show me the declaration of this f?'' ``what fs are in scope here?''
``what is the resolution of this use of +?'' ``Which classes are derived
from class Shapen?'' and ``what destructors are called at the end of this
block?'' answered in a second.
"A C++ program contains a wealth of information that in a typlical
environment is available only to a compiler. I want that information at
the programmer's fingertips." - D&E § 9.4.4 /Beyond Files and Syntax/
Dietmar Kuehl
> In the US we have a traditional children's story called _Hansel and
> Gretel_,
I'd guess that it actually originates from Germany, at least with this
name (which actually is "Hänsl und Gretel").
> Now consider a typical STL algorithm. You cannot look at an STL container
> and see which algorithms are associated with it.
That is easy to figure out because the answer is simple and generic:
if there are no algorithms in form of member functions, there is no
algorithm associated with a container. That is because the algorithms
are generic and independent of the container. Actually, they don't
even require a container in the first place. Essentially, you can use
all algorithms you get appropriate iterators for, i.e. you would
figure out which iterators exists or which you want to create. This
provides the scope of algorithms. Of course, the set of iterators
related to a container is also open: beyond a simple iteration over
all elements you can have e.g. filtering iterators which skip certain
elements.
> That is what was missing from std::streambuf<>.
Well, the stream buffer iterators are entirely independent from
'std::streambuf'! They don't use any private or protected members but
the normal public interface. Thus, it makes no sense to reference
them from their members.
>>> I'm looking at page 676 of TC++SL.
>>
>> I guess you are talking of Nico's book in which case I would have written
>> or translated the IOStream portion. I don't have the book at hand and
>> thus I can't check what is said on page 676. However, I doubt that it
>> states that eback() is any good for a stream buffer user or that it is
>> the right thing to get the start of a buffer.
>
> No, it merely shows where the beginning of the available data is. Figure
> 13.5 and associated text.
Neither the picture nor the text implies, however, that 'eback()' is
in some form the beginning of the input. It the beginning of the input
buffer but this is quite different from the input. For typical stream
buffers the input comes to rest at least temporarily in the input
buffer but that's all.
> I'm trying to explain to you
> something that you cannot see. That is, what the current C++ looks like
> to a person who is learning it. I believe that is a valuable perspective
> for a person involved in design to understand.
I think I have a pretty good view of how C++ looks to people learning
it, especially when it comes to the library part. ...and, indeed, it
is non-trivial and it has its own idioms which are different from
idioms used in other most other language. What is kind of news to me
is that somebody learning a language insists in twisting it to become
a different language...
> And knowing which iterators are available, as well as what flags to use to
> modify the behavior of the stream. Also relevant is the distinction
> between formatted and unformatted operations. It is well advised also to
> understand that unformatted streams may still alter the underlying data,
> and that formatted operations may alter the data in a so-called "binary"
> stream. It is easy to put the stream into "stupid" format states where
> the wrong combination of flags is set causing the numeric format to behave
> as if the default were set. strm.clear(flag) unsets everything _but_ flag.
> strm.setf(flags) doesn't mean `set flags' even though that's what it does.
> Then there is 'basic_ios' verses 'ios_base'. std::string::clear() does
> something completely different than std::stringstream::clear(). Reading a
> stream completely and correctly puts it into a fail state.
>
> Then there are the necessary complexities of the design to consider.
Apparently we have rather different approaches to learning about a
language and/or a library. My approach is to get an overview of the
whole language or library and then obtain the details once I need
them. Your approach seems to pick whatever looks like it could be
useful for a task and then twist it trying to make it what you want
it to. For the binary formatting stuff it would have been obvious that
there is no direct support and what to do about it. No need to touch
even a single formatting flag and only a need to adjust the open mode
for files.
>> For most
>> of the normal work you don't have to understand these hierarchies at
>> all! You just operate on an 'std::istream' or an 'std::ostream' - you
>> can even get a 'std::istreambuf_iterator' with an 'std::istream'.
>
> So what happens when I have a C-style file pointer returned from a
> library? Can I use that with C++ I/O?
Sure you can. You would wrap it into a stream buffer. Actually, this
is how 'std::basic_filebuf' is implemented on some systems...
> The advantage to that code is simple. It provides support for code
> completion.
Code completion is good for the knowledgable who needs to be reminded.
It is evil for everybody else. In summary, I think it is better removed
from development tools because the knowledgable are quick enough in
locating the stuff anyway.
> There are several things that could be
> done with the basic concept that would provide a more usable interface
> than the existing one.
Indeed, there are. Somehow I seriously doubt that the improvements
you can imagine have even a tangtial intersection with the improvements
I can imagine...
Julián Albo
> You seem to be confused between my making an observation with the
> objective of identifying problem areas so that improvements might be made,
> and the alse idea that I expect absolute adherence to some design
> principle. I was stating facts as I see them.
Then I think you are loosing your time. And ours.
--
Salu2
Dietmar Kuehl
> It was not at all intended to be insulting. I am baffled about how you
> could find that insulting.
I guess you didn't realize whom you were discussing with: P.J.Plauger
was a long term chair of the library working group and major parts of
the original standard C++ library specification were written by him.
You explicitly stated that the current specification is inferior to
other options and that it is just due to licensing issues. I consider
this a pretty severe insult.
> I was simply thinking about this statement by the
> inventor of the I/O streams:
>
> "A major goal in my original design was that it be extensible in
> interesting ways. In particular, in the stream library the streambuf class
> was an implementation detail, but in the iostream library I intended it to
> be a usable class in its own right. I was hoping for the promulgation of
> many streambufs with varied functionality. I wrote a few myself, but
> almost no one else did. I answered many more questions of the form "how do
> I make my numbers look like this" than "how do I write a streambuf". And
> textbook authors also tended to ignore streambufs. Apparently they did not
> share my view that the architecture of the input/output library was an
> interesting case study."
I think I can understand how Jerry was feeling about it and, without
knowing (I think I haven't seen this quote before) I helped correcting
this issue, e.g. by explaining how to implement stream buffer in UseNet
and in Nico's book.
> You are looking for insults where there are none. When I insult you, you
> will know it. Trust me.
I consider it an insult when saying to someone involved in the creation
of a specification that he did a far inferior job.
Steven T. Hatton
> Steven T. Hatton wrote:
>> In the US we have a traditional children's story called _Hansel and
>> Gretel_,
>
> I'd guess that it actually originates from Germany, at least with this
> name (which actually is "Hänsl und Gretel").
I assume you are familiar with the metaphor.
>> That is what was missing from std::streambuf<>.
>
> Well, the stream buffer iterators are entirely independent from
> 'std::streambuf'! They don't use any private or protected members but
> the normal public interface. Thus, it makes no sense to reference
> them from their members.
I would say that it does not follow a dominate concept in OO design. I do
believe it makes sense to do /something/ to show there is an association
between the stream buffer and the iterator. What that something is, I am
not sure.
Though it probably doesn't apply here, this situation is related to the
reason I suggested there be an additional access specifier called
'associated'. An associated member of a class would not have privileged
access to any of the class members. It would simply add the function to
the view we have of a class. I further suggested that classes which shared
a particular function, say an overload of the '*' operator, might be
derived from a common base which provided the operator. A vector and a
matrix might fit this category.
I submit that the fact that the iterator is external to the stream buffer
*_may_* indicate it lives at the wrong level of abstraction.
>> No, it merely shows where the beginning of the available data is. Figure
>> 13.5 and associated text.
>
> Neither the picture nor the text implies, however, that 'eback()' is
> in some form the beginning of the input. It the beginning of the input
> buffer but this is quite different from the input. For typical stream
> buffers the input comes to rest at least temporarily in the input
> buffer but that's all.
What about a file?
>> I'm trying to explain to you
>> something that you cannot see. That is, what the current C++ looks like
>> to a person who is learning it. I believe that is a valuable perspective
>> for a person involved in design to understand.
>
> I think I have a pretty good view of how C++ looks to people learning
> it, especially when it comes to the library part. ...and, indeed, it
> is non-trivial and it has its own idioms which are different from
> idioms used in other most other language. What is kind of news to me
> is that somebody learning a language insists in twisting it to become
> a different language...
Me?! I reject the accusation! This may not be perfect, but you tell me what
language it's written in:
#include <iostream>
#include <fstream>
#include <iomanip>
#include <sstream>
#include <string>
namespace hexlite {
using namespace std;
typedef string::const_iterator c_itr;
ostream& printline(c_itr start, c_itr stop, ostream& out) {
while(start<stop) out
<<setw(2)
<<(static_cast<unsigned int>(static_cast<unsigned char>(*start++)))<<"
";
return out;
}
ostream& dump(const string& dataString, ostream& out) {
ostream hexout(out.rdbuf());
hexout.setf(ios::hex, ios::basefield);
hexout.fill('0');
c_itr from (dataString.begin());
c_itr dataEnd (from + dataString.size());
c_itr end (dataEnd - (dataString.size()%16));
for(c_itr start = from; start < end; start += 16)
printline(start, start + 16, hexout)<<endl;
printline(end, dataEnd, hexout)<<endl;
return out;
}
}
int main(int argc, char* argv[]) {
if (argc < 1) { std::cerr<<"enter a file name"<<std::endl; return -1; }
std::ifstream inf(argv[1],std::ios::binary);
if(inf) {
std::ostringstream oss;
oss << inf.rdbuf();
hexlite::dump(oss.str(), std::cout);
return 0;
}
std::cerr <<"\nCan't open file:"<<argv[1]<<std::endl;
return -1;
}
That started out as 284 lines of "C++" code. There's a small amount of
functionality I would need to add to get it back to the original target
functionality. The original code had a problem which is the problem I was
trying to solve when I started this thread.
> Apparently we have rather different approaches to learning about a
> language and/or a library. My approach is to get an overview of the
> whole language or library and then obtain the details once I need
> them. Your approach seems to pick whatever looks like it could be
> useful for a task and then twist it trying to make it what you want
> it to.
I would not have gone nearly as deep into the I/O at this point had it not
been for the number of replies that were of the nature of:
'You idiot! Why are you using a char not an unsigned char?'
'You idiot! Why are you using a unsigned chare and not a char?'
'You idiot! Why are you using a stringstream and not a vector?'
etc.
> For the binary formatting stuff it would have been obvious that
> there is no direct support and what to do about it. No need to touch
> even a single formatting flag and only a need to adjust the open mode
> for files.
If I had found this earlier, it would have saved me a lot of time:
http://gcc.gnu.org/onlinedocs/libstdc++/27_io/howto.html
How to get an unadulterated bit stream is not very clear from having a
simple overview of the library. I also chose two bad places to look for
the information.
> Indeed, there are. Somehow I seriously doubt that the improvements
> you can imagine have even a tangtial intersection with the improvements
> I can imagine...
How many years of C and C++ experience do you have?
Steven T. Hatton
> Steven T. Hatton wrote:
>> It was not at all intended to be insulting. I am baffled about how you
>> could find that insulting.
>
> I guess you didn't realize whom you were discussing with: P.J.Plauger
> was a long term chair of the library working group and major parts of
> the original standard C++ library specification were written by him.
> You explicitly stated that the current specification is inferior to
> other options and that it is just due to licensing issues. I consider
> this a pretty severe insult.
No. I merely suggested there were some features of the other implementation
that had some attractiveness. I specifically made it clear that directly
emulating the Java I/O would break C++ I/O. If I challenged him to rethink
the current implementation, and he found that insulting, so be it.
> I think I can understand how Jerry was feeling about it and, without
> knowing (I think I haven't seen this quote before) I helped correcting
> this issue, e.g. by explaining how to implement stream buffer in UseNet
> and in Nico's book.
>
>> You are looking for insults where there are none. When I insult you, you
>> will know it. Trust me.
>
> I consider it an insult when saying to someone involved in the creation
> of a specification that he did a far inferior job.
But I did not say that. What I was specifically thinking about was the way
Java I/O interfaces fit together as "filters". That's the part I like. I
fully agree, the character level suff like reading in mixed character based
and numeric based data plain sucks in Java when compared to C++. Now, in
comparison to C... Consider something else, I bet my entire future on
learning C++, when I had a reasonable grasp of Java in a very lucrative
area to start with.
Dietmar Kuehl
> Though it probably doesn't apply here, this situation is related to the
> reason I suggested there be an additional access specifier called
> 'associated'. An associated member of a class would not have privileged
> access to any of the class members.
The problem is not that much how to name any access specifier or whatever.
The problem is that you would need to open the class everytime you added
something which could possibly used with that class. ... and this is just
the tip of the iceberg: in C++, you can even use many things never
directly intended to be used with a specific class. That is, if a class
is implemented to conform to a certain concept, it can be used everywhere
this concept is used.
> I submit that the fact that the iterator is external to the stream buffer
> *_may_* indicate it lives at the wrong level of abstraction.
Not at all! Actually, making it an internal aspect of a stream buffer
would probably be an error! In fact, it is quite common to create iterators
for entities which were never intended to be used with iterator, often due
to the fact that they were created when [at least STL] iterators were not
invented (e.g. something like UNIX readdir(3) has a fair chance to predate
the overall concept of an iterator but there are STL iterators using it).
C++ is not built around object orientation (anymore, at least) because it
turned out that object orientation does not solve all problems.
> What about a file?
Specifically files will come to rest only blockwise in the file buffer's
character buffer. Well, the exact size depends on the implementation but
processing files in multiples of the file system's block size is a natural
thing to do. Of course, a file buffer implemented on top of 'FILE*' which
is also a viable approach may never buffer more than just one character in
the file buffer and delegate the actual buffering to the 'FILE*'
functions.
>> What is kind of news to me
>> is that somebody learning a language insists in twisting it to become
>> a different language...
>
> Me?! I reject the accusation!
Apparently, there are two persons then who post under the name "Steve
T. Hatton". For a reference you might want to have a look at the
following articles where this other "Steve T. Hatton" at least wants to
discuss changes to C++:
<http://groups-beta.google.com/group/comp.std.c++/msg/e9fc2c6b9d11453e?hl=en&>
<http://groups-beta.google.com/group/comp.std.c++/msg/2e7702a4ff7b098d?hl=en&>
<http://groups-beta.google.com/group/comp.lang.c++/msg/f5bb61a26494adf9?hl=en&>
<http://groups.google.com/group/comp.lang.c++/msg/1ec45f94a87ac66d?hl=en&>
Interestingly enough, the last message is actually from this very thread.
I would have thought that you had realized that a different person is
posting under the name you do...
> I would not have gone nearly as deep into the I/O at this point had it not
> been for the number of replies that were of the nature of:
>
> 'You idiot! Why are you using a char not an unsigned char?'
> 'You idiot! Why are you using a unsigned chare and not a char?'
> 'You idiot! Why are you using a stringstream and not a vector?'
> etc.
Well, assuming you perception is that I called you implicitly an idiot,
I think you are wrong. Generally, I just point people into the correct
direction and possibly state why the thing they did didn't work. Of
course, this is made easier if they stated what they wanted rather than
showing their attempt at a solution: if the specification of the problem
is not stated it is essentially impossible to figure the exact goals out
from a wrong attempt.
> How many years of C and C++ experience do you have?
I don't know how this matters but it is fairly easy to find these numbers
out (with a little bit of calculation) since I have posted them in the
past, anyway. I'm programming about 15 years with C++ and I started C a
little bit earlier.
Dietmar Kuehl
>> I consider it an insult when saying to someone involved in the creation
>> of a specification that he did a far inferior job.
>
> But I did not say that.
You might want to reread the articles in this thread hierarchy. In a rush
I can't find a single statement which makes this statement but if I just
sum up what I reread right now, I come to the conclusion that you did.
> What I was specifically thinking about
What you think is something different from what people read in UseNet
from your writing.
> was the way Java I/O interfaces fit together as "filters". That's the
> part I like.
Great. C++ has this since ages. Of course, the C++ library distracts a
little from this fact by bundling up commonly used configurations into
a simple class.
Julián Albo
>>> What is kind of news to me
>>> is that somebody learning a language insists in twisting it to become
>>> a different language...
>>
>> Me?! I reject the accusation!
>
> Apparently, there are two persons then who post under the name "Steve
> T. Hatton". For a reference you might want to have a look at the
> following articles where this other "Steve T. Hatton" at least wants to
> discuss changes to C++:
Maybe the accusation rejected is to be learning C++ X-)
--
Salu2
Steven T. Hatton
> Steven T. Hatton wrote:
>>> I consider it an insult when saying to someone involved in the creation
>>> of a specification that he did a far inferior job.
>>
>> But I did not say that.
>
> You might want to reread the articles in this thread hierarchy. In a rush
> I can't find a single statement which makes this statement but if I just
> sum up what I reread right now, I come to the conclusion that you did.
If I was giving the impression that I believe there is something very
problematic with the C++ I/O implementation, then I communicated
effectively. The very fact that the statements quoted below[*] were made
indicates to me there are some serious shortcomings with C++ I/O.
When I read Stroustrup's treatment of I/O, he said "It is possible to define
streams for which the physical I/O is not done in terms of characters.
However, such streams are beyond the scope of the C++ standard and beyond
the scope of this book." - §21.2.1. Now, it's not clear to me what,
exactly, that means. Is he talking about streams where the data type is
other than type char, or streams where the data is not processed in terms
of characters sets? I suspect the answer would be "yes".
When I read Josuttis's book, I don't recall anywhere that clearly explains
the distinction between processing character based data and processing
"raw" data. All the details may be there, but I would be hard-pressed to
find them. One person told me that I should use the std::ostream::read()
and std::ostream::write() functions. But another person wrote this:
"... and, BTW, using 'std::istream::read()' and 'std::ostream::write()'
is almost certainly the *wrong* approach! These functions are an
historical mistake which should have been corrected in the standard:..."
Plauger's documentation, the place I looked first for an answer clued me
onto the idea that the openmode of the stream would be important, but it
did not discuss the ramifications of using formatting vs. non-formatting
functions. The impression it gave was that if I open a stream in binary
mode, it will faithfully reproduce the data exactly as it exists prior to
passing through the stream. The documentation then went on to discuss I/O
using functions from the C libraries.
I don't recally any discussion in Accelerated C++ about working with
unformatted binary data. A while back I tried reading the section in the
Standard, but I found it to be difficult to follow because of the wording
regarding the structure of the clause, I therefore never finished reading
the clause. I now believe that I would have benefitted by actual taking
the time to figure it all out.
I have been told that I should have been working with unsigned char, I've
also been told I was wrong to be working with unsigned char. I've been
told I can set the stream to binary and use the extractor functions if I
unset the formatting bits. I've also been told that this will not work
(and I believe there could be performance issues with this as well). See
above for as similar example with read() and write().
>> What I was specifically thinking about
>
> What you think is something different from what people read in UseNet
> from your writing.
Well, there is the problem that what people read into my posts is colored by
what other people write. When I read the comments of Jerry Schwartz
regarding the scarcity of new stream classes, in conjunction with a comment
suggesting that I was free to contribute my own, I suggested that part of
the reason there had not been as much contribution to the C++ Standard
Library was because of the nature of the licensing of libraries that I know
to exist. I was accused of arrogance for that. The fact of the matter is,
I was merely extrapolating from this:
"How is the Boost license different from the GNU General Public License
(GPL)? The Boost license permits the creation of derivative works for
commercial or non-commercial use with no legal requirement to release your
source code."
The next thing I know Plauger, and others are also accusing me of arrogance
for having made that statement.
>> was the way Java I/O interfaces fit together as "filters". That's the
>> part I like.
>
> Great. C++ has this since ages. Of course, the C++ library distracts a
> little from this fact by bundling up commonly used configurations into
> a simple class.
To some extent the C++ I/O library is a kludge. That is simply a result of
history. That's the way things happen when new designs are built directly
on top of older ones. There are good reasons people did things this way.
Nonetheless, it causes problems.
To a considerable extent, the C++ iostream classes _can_ be understood as
filters. The stream buffers represent an unfiltered stream, and the stream
classes represent filters. One thing that seems out of order about this
view is that the openmode is set at a higher level than the level at which
unformatted I/O is performed.
[*]
http://gcc.gnu.org/onlinedocs/libstdc++/27_io/howto.html#3
<quote>
Binary I/O
The first and most important thing to remember about binary I/O is that
opening a file with ios::binary is not, repeat not, the only thing you have
to do. It is not a silver bullet, and will not allow you to use the <</>>
operators of the normal fstreams to do binary I/O.
Sorry. Them's the breaks.
This isn't going to try and be a complete tutorial on reading and writing
binary files (because "binary" covers a lot of ground), but we will try and
clear up a couple of misconceptions and common errors.
First, ios::binary has exactly one defined effect, no more and no less.
Normal text mode has to be concerned with the newline characters, and the
runtime system will translate between (for example) '\n' and the
appropriate end-of-line sequence (LF on Unix, CRLF on DOS, CR on Macintosh,
etc). (There are other things that normal mode does, but that's the most
obvious.) Opening a file in binary mode disables this conversion, so
reading a CRLF sequence under Windows won't accidentally get mapped to a
'\n' character, etc. Binary mode is not supposed to suddenly give you a
bitstream, and if it is doing so in your program then you've discovered a
bug in your vendor's compiler (or some other part of the C++
implementation, possibly the runtime system).
Second, using << to write and >> to read isn't going to work with the
standard file stream classes, even if you use skipws during reading. Why
not? Because ifstream and ofstream exist for the purpose of formatting, not
reading and writing. Their job is to interpret the data into text
characters, and that's exactly what you don't want to happen during binary
I/O.
Third, using the get() and put()/write() member functions still aren't
guaranteed to help you. These are "unformatted" I/O functions, but still
character-based. (This may or may not be what you want, see below.)
</quote>
Steven T. Hatton
> Steven T. Hatton wrote:
>> Though it probably doesn't apply here, this situation is related to the
>> reason I suggested there be an additional access specifier called
>> 'associated'. An associated member of a class would not have privileged
>> access to any of the class members.
>
> The problem is not that much how to name any access specifier or whatever.
> The problem is that you would need to open the class everytime you added
> something which could possibly used with that class.
No. There would be no requirement that all operations on the class be done
through associated methods. This would simply be a means of collecting
things which are logically related into selfcontained units.
> ... and this is just
> the tip of the iceberg: in C++, you can even use many things never
> directly intended to be used with a specific class. That is, if a class
> is implemented to conform to a certain concept, it can be used everywhere
> this concept is used.
But if there were some hypothetical tag to indicate explicitly that a
certain concept were expressed by a given class, and another tag to
indicate that a different class requires that concept, then there might be
a way of automattically generating a mapping between the two kinds of
class. Such a technique might be useful as a means of meta-type-checking.
Say, for example, if your iterators have traits indicating their category,
your container, or other class could have similar traits indicating what
type of iteration they support. But then...
This is why I laugh when I think of template parameters as meta-types.
>> I submit that the fact that the iterator is external to the stream buffer
>> *_may_* indicate it lives at the wrong level of abstraction.
>
> Not at all! Actually, making it an internal aspect of a stream buffer
> would probably be an error! In fact, it is quite common to create
> iterators for entities which were never intended to be used with iterator,
> often due to the fact that they were created when [at least STL] iterators
> were not invented (e.g. something like UNIX readdir(3) has a fair chance
> to predate the overall concept of an iterator but there are STL iterators
> using it). C++ is not built around object orientation (anymore, at least)
> because it turned out that object orientation does not solve all problems.
eback(), gpter(), and egptr() /are/ iterators. They are, however, not
iterators that enforce the correct invariant for a user of the stream. I
understand the basic idea of what streambuf_iterators are doing. They are
enforcing an invariant for the stream while providing sequential access to
its elements. They exploit the public interface to do so.
>> What about a file?
>
> Specifically files will come to rest only blockwise in the file buffer's
> character buffer. Well, the exact size depends on the implementation but
> processing files in multiples of the file system's block size is a natural
> thing to do. Of course, a file buffer implemented on top of 'FILE*' which
> is also a viable approach may never buffer more than just one character in
> the file buffer and delegate the actual buffering to the 'FILE*'
> functions.
That makes sensse. Now, if I open a file, read it into, say a string
stream, then clear() it and set seekg back to the beginning, under normal
circumstances (file is not huge) I believe the file is typically read in
one operation, and worked on in memory. But I really don't know how that
relates to the structure of the iostreams implementation on my system. I
would need to read the code, which I'm not going to do right now.
>>> What is kind of news to me
>>> is that somebody learning a language insists in twisting it to become
>>> a different language...
>>
>> Me?! I reject the accusation!
>
> Apparently, there are two persons then who post under the name "Steve
> T. Hatton". For a reference you might want to have a look at the
> following articles where this other "Steve T. Hatton" at least wants to
> discuss changes to C++:
>
>
<http://groups-beta.google.com/group/comp.std.c++/msg/e9fc2c6b9d11453e?hl=en&>
>
<http://groups-beta.google.com/group/comp.std.c++/msg/2e7702a4ff7b098d?hl=en&>
>
<http://groups-beta.google.com/group/comp.lang.c++/msg/f5bb61a26494adf9?hl=en&>
> <http://groups.google.com/group/comp.lang.c++/msg/1ec45f94a87ac66d?hl=en&>
>
> Interestingly enough, the last message is actually from this very thread.
> I would have thought that you had realized that a different person is
> posting under the name you do...
None of that supports your assertion. In most of it, I made no suggestion
regarding any change to the language. Of particular interest is the post
discussing C++ classes that act like Java or C# classes, that is exactly
what I _did_not_ suggest.
>> I would not have gone nearly as deep into the I/O at this point had it
>> not been for the number of replies that were of the nature of:
>>
>> 'You idiot! Why are you using a char not an unsigned char?'
>> 'You idiot! Why are you using a unsigned chare and not a char?'
>> 'You idiot! Why are you using a stringstream and not a vector?'
>> etc.
>
> Well, assuming you perception is that I called you implicitly an idiot,
No. There have been some rather 'interesting' comments from others,
however.
> I think you are wrong. Generally, I just point people into the correct
> direction and possibly state why the thing they did didn't work. Of
> course, this is made easier if they stated what they wanted rather than
> showing their attempt at a solution: if the specification of the problem
> is not stated it is essentially impossible to figure the exact goals out
> from a wrong attempt.
>
>> How many years of C and C++ experience do you have?
>
> I don't know how this matters but it is fairly easy to find these numbers
> out (with a little bit of calculation) since I have posted them in the
> past, anyway. I'm programming about 15 years with C++ and I started C a
> little bit earlier.
I got my first formal introduction to digital I/O at the bit-level in 1979.
I may not know C++'s I/O that well, and I have not focused on I/O at that
level for the past two and a half decades, but I do have a solid background
in the subject.
Dietmar Kuehl
> But if there were some hypothetical tag to indicate explicitly that a
> certain concept were expressed by a given class, and another tag to
> indicate that a different class requires that concept, then there might be
> a way of automattically generating a mapping between the two kinds of
> class. Such a technique might be useful as a means of meta-type-checking.
We can agree on this. It is, however, not yet there but it is under
discussion: there have been two independent proposals to add concepts
to C++, mostly to do some form of concept checking and to remove the
need for certain kinds of meta programming. This does still not help
to locate e.g. the stream [buffer] iterators for streams because they
are implemented entirely independent from the stream. This can only be
located with some form of use dependency.
> eback(), gpter(), and egptr() /are/ iterators. They are, however, not
> iterators that enforce the correct invariant for a user of the stream.
They are an implementation detail of stream buffers used to communicate
between the base class and its derived classes. They are iterator but
they are only relevant to implementers of stream buffers. They should
never be used by something external to the stream buffer to access the
buffer (well, there is something for which they can be used to great
effect, namely for optimizations for segmented sequences; however, even
this optimization could be made to use a safe public interface).
>> Specifically files will come to rest only blockwise in the file buffer's
>> character buffer. Well, the exact size depends on the implementation but
>> processing files in multiples of the file system's block size is a
>> natural thing to do. Of course, a file buffer implemented on top of
>> 'FILE*' which is also a viable approach may never buffer more than just
>> one character in the file buffer and delegate the actual buffering to the
>> 'FILE*' functions.
>
> That makes sensse. Now, if I open a file, read it into, say a string
> stream, then clear() it and set seekg back to the beginning, under normal
> circumstances (file is not huge) I believe the file is typically read in
> one operation, and worked on in memory.
The file stream buffer ('std::basic_filebuf<..>') is likely to hold a
buffer which is a multiple of the corresponding file system block size,
possibly plus a few additional characters to provide a put back facility.
The user of the file stream buffer has no real control over the size of
the buffer, in particular 'pubsetbuf()' is only guaranteed to work when
turning off the buffer (i.e. the call 'pubsetbuf(0, 0)'). Thus, it is
unsafe to assume that the internal buffer of file stream buffer holds
any particular content, e.g. the whole file. As mentioned before, the
buffer may actually consist of just one character (the wording in the
standard seems to imply that the buffer has to hold at least one
character if there is any although I consider this restriction
unnecessary and actually an error in the specification), e.g. because the
actual buffering is done by a 'FILE*'.
>
<http://groups-beta.google.com/group/comp.std.c++/msg/e9fc2c6b9d11453e?hl=en&>
>>
>
<http://groups-beta.google.com/group/comp.std.c++/msg/2e7702a4ff7b098d?hl=en&>
>>
>
<http://groups-beta.google.com/group/comp.lang.c++/msg/f5bb61a26494adf9?hl=en&>
>>
<http://groups.google.com/group/comp.lang.c++/msg/1ec45f94a87ac66d?hl=en&>
>>
>> Interestingly enough, the last message is actually from this very thread.
>> I would have thought that you had realized that a different person is
>> posting under the name you do...
>
> None of that supports your assertion. In most of it, I made no suggestion
> regarding any change to the language.
My reading of those articles is that you indeed suggest changes to the
language (which in my reading includes the standard library).
Dietmar Kuehl
> If I was giving the impression that I believe there is something very
> problematic with the C++ I/O implementation, then I communicated
> effectively. The very fact that the statements quoted below[*] were made
> indicates to me there are some serious shortcomings with C++ I/O.
In my opinion there is only one major shortcoming with C++ I/O and this
is a lack of understanding. There seem to be only few people who really
understand the model or even want to understand it. This includes both
the technical details and the scope this library targets, although
neither is actually that hard to understand.
The first thing to note about C++ I/O (which also applies to C I/O) is
that it is very vague about things like standard input, standard output,
and files. There is a simple reason for this, though: the exact details
differ widely between different platforms and thus a definition of what
a file is and what kind of properties it has are essentially side
stepped. All what is assumed is that files have name (though no detail
is given how valid names look like) and consist of sequences of bytes
which can be read or written. When writing a file and reading it back
in later, the bytes read should be identical to those which have been
written if the files were opened in binary mode. If one or both of
these files were not opened in binary mode, the bytes read may differ
from those written.
Note, that this does not say anything about the actual bytes stored in
the file! Although it is reasonable to assume that these are identical
to those written, this is not necessarily the case and may be dependent
of the platform.
The next thing to note about C++ I/O is that it addresses essentially
text formatted I/O. It is neither intended nor particular well suited
for binary I/O although the latter can be done to some extend. This is
shown e.g by the fact that all I/O layers accessible to the user
operate in terms of characters, not in terms of bytes. Although the
normal character type ('char') is also often used to represent bytes,
it is important to keep these things mentally apart because the bytes
undergo a transformation to become characters, namely a locale specific
translation between bytes and characters. This is done at some lower
level in 'FILE* or 'std::basic_filebuf'. In both cases it is guaranteed
that this conversion is the identity conversion when using the "C"
locale. That is, to get an unchanged sequence of bytes useful for some
form of binary I/O you would need to open the file in binary mode and
suppress any locale specific character transformation by using the "C"
locale.
In particular the stream level hierarchy only makes sense for text
formatted I/O. Although it provides "unformatted" operations,
interaction between these and the formatted operations is a little bit
tricky (in particular, the formatted operations skip e.g. white space
characters while the unformatted do not; this is often a stumbling
block when switching between formatted and unformatted operations).
When you want to operate on binary files, it much more reasonable to
use stream buffers directly and provide a similar hierarchy to the
streams for reading a particular binary format.
Alf P. Steinbach
>
> In my opinion there is only one major shortcoming with C++ I/O and this
> is a lack of understanding.
...
>
> The next thing to note about C++ I/O is that it addresses essentially
> text formatted I/O. It is neither intended nor particular well suited
> for binary I/O although the latter can be done to some extend. This is
> shown e.g by the fact that all I/O layers accessible to the user
> operate in terms of characters, not in terms of bytes.
Well then, considering that most of C++ i/o text input has formally
undefined behavior, I guess a lack of understanding that point is the major
shortcoming you're talking about?
Sorry, I couldn't resist. ;-)
--
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
P.J. Plauger
news:-uednTTYbKF...@speakeasy.net...
> I have been told that I should have been working with unsigned char, I've
> also been told I was wrong to be working with unsigned char.
No, you were told that it is *unnecessary* to work with unsigned char.
There are also reasons to avoid this type, but we didn't get that
far.
> I've been
> told I can set the stream to binary and use the extractor functions if I
> unset the formatting bits. I've also been told that this will not work
> (and I believe there could be performance issues with this as well).
No, you were told to use *certain* extractors. Most are geared
toward formatted I/O and are hence irrelevant. What you believe
about performance is uninteresting until you make measurements
to support it.
> See
> above for as similar example with read() and write().
>
>>> What I was specifically thinking about
>>
>> What you think is something different from what people read in UseNet
>> from your writing.
>
> Well, there is the problem that what people read into my posts is colored
> by
> what other people write. When I read the comments of Jerry Schwartz
> regarding the scarcity of new stream classes, in conjunction with a
> comment
> suggesting that I was free to contribute my own, I suggested that part of
> the reason there had not been as much contribution to the C++ Standard
> Library was because of the nature of the licensing of libraries that I
> know
> to exist. I was accused of arrogance for that.
No, I accused you of arrogance for making the statement:
>> After you posted, I began thinking about why there hasn't been as much
>> contribution as one might expect to something as important as the C++
>> Standard Library.
(Despite Dietmar's comments, I did not take personal offense at this
statement. My contributions to the design of the Standard C++ library
have been very small. I'm more of an implementer and explainer. But
the statement certainly shows a lack of appreciation for the many
real, and sometimes highly inventive, contributions made by others
to the library. That, to me, reflects arrogance, stupidity, or both.)
I've read enough of your posts now to see that you are not a very
clear thinker. You are even worse at understanding what you read,
particularly if it challenges any of your preconcieved beliefs.
Thus, debating you is akin to shooting at liferafts.
P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com
Steven T. Hatton
> Steven T. Hatton wrote:
>> If I was giving the impression that I believe there is something very
>> problematic with the C++ I/O implementation, then I communicated
>> effectively. The very fact that the statements quoted below[*] were made
>> indicates to me there are some serious shortcomings with C++ I/O.
>
> In my opinion there is only one major shortcoming with C++ I/O and this
> is a lack of understanding. There seem to be only few people who really
> understand the model or even want to understand it. This includes both
> the technical details and the scope this library targets, although
> neither is actually that hard to understand.
Nothing is that hard to understand, once you understand it. ;) I believe
you are probably correct about it not being too terribly hard to
understand. I do believe it has some unfortunate features which make it
more difficult than it needs to be. This includes cryptic names,
counterintuitive control mechanisms, and, at least one uncommon use of a
commonly used term. That term being "binary".
> The first thing to note about C++ I/O (which also applies to C I/O) is
> that it is very vague about things like standard input, standard output,
> and files. There is a simple reason for this, though: the exact details
> differ widely between different platforms and thus a definition of what
> a file is and what kind of properties it has are essentially side
> stepped.
I can see where trying to specify a generic file descriptor could be
problematic. However, I can't imagine it poses anything close to the
variability that locales pose. Perhaps it really is something best left
outside of the C++ Standard. My first inclination was to suggest the
Standard could provide some kind of extensible framework, and use POSIX as
a default implementation. But, then that imposes a requirement on the
implementation which may not be easy to meet.
> All what is assumed is that files have name
I wonder if there is a place in the C++ Standard Library for something
similar to JNDI.
> (though no detail
> is given how valid names look like) and consist of sequences of bytes
> which can be read or written. When writing a file and reading it back
> in later, the bytes read should be identical to those which have been
> written if the files were opened in binary mode. If one or both of
> these files were not opened in binary mode, the bytes read may differ
> from those written.
Hmmm. I seems that statement should come with some further qualification
about the role of formating operators.
...
> The next thing to note about C++ I/O is that it addresses essentially
> text formatted I/O. It is neither intended nor particular well suited
> for binary I/O although the latter can be done to some extend. This is
> shown e.g by the fact that all I/O layers accessible to the user
> operate in terms of characters, not in terms of bytes. Although the
> normal character type ('char') is also often used to represent bytes,
> it is important to keep these things mentally apart because the bytes
> undergo a transformation to become characters, namely a locale specific
> translation between bytes and characters. This is done at some lower
> level in 'FILE* or 'std::basic_filebuf'. In both cases it is guaranteed
> that this conversion is the identity conversion when using the "C"
> locale. That is, to get an unchanged sequence of bytes useful for some
> form of binary I/O you would need to open the file in binary mode and
> suppress any locale specific character transformation by using the "C"
> locale.
I believe, by default, all my code will use the "C" locale unless I
specifically change it. I recall reading that using wchar_t instead of
char can improve performance for unformatted binary I/O, and I assume the
same qualifications hold for locale in that situation as well. No matter
whether I use char or wchar_t I have no guarantee as to the exact size nor
signedness of the objects of that type. I'm not sure what that means in
terms of reading or writing a "binary" file such as an ELF file which has
some embedded text.
> In particular the stream level hierarchy only makes sense for text
> formatted I/O. Although it provides "unformatted" operations,
> interaction between these and the formatted operations is a little bit
> tricky (in particular, the formatted operations skip e.g. white space
> characters while the unformatted do not; this is often a stumbling
> block when switching between formatted and unformatted operations).
> When you want to operate on binary files, it much more reasonable to
> use stream buffers directly and provide a similar hierarchy to the
> streams for reading a particular binary format.
This is the other part of what you said was not that hard to understand
about the library. It isn't that difficult to understand. I guess the
idea of providing a general stream template for arbitrary integral types
has too many complications, such as what happens if data of type char is
put into a stream of type unsigned long one unit at a time. Should it be
packed so that it uses optimal space? etc.
Trying to work with C-style I/O in conjunction with C++ I/O seems rather
difficult. For instance, if I have a library that uses printf(), I don't
believe I can get it to write to a C++ stream. I did come across this:
<quote>
http://www.channel1.com/users/bobwb/cppnotes/lec08.htm
8.8 Mixing in C I/O
C I/O functions can be intermixed with C++ I/O portably on a line by line
basis. To use a C++ file in a C context, an ostream or istream must be
constructed with a stdiobuf (a type of streambuf - header is stdiostream.h)
which is in turn constructed from a FILE *.
Example:
FILE *fp = fopen("test.dat", "r+");
stdiobuf buf(fp);
ostream str(buf);
</quote>
And this:
http://www.boost.org/libs/format/index.html
<quote>
Boost Format library
The format library provides a class for formatting arguments according to a
format-string, as does printf, but with two major differences :
* format sends the arguments to an internal stream, and so is entirely
type-safe and naturally supports all user-defined types.
* The ellipsis (...) can not be used correctly in the strongly typed
context of format, and thus the function call with arbitrary arguments is
replaced by successive calls to an argument feeding operator%
Steven T. Hatton
> "Steven T. Hatton" <chatt...@germania.sup> wrote in message
> news:-uednTTYbKF...@speakeasy.net...
>
[...]
>> I've been
>> told I can set the stream to binary and use the extractor functions if I
>> unset the formatting bits. I've also been told that this will not work
>> (and I believe there could be performance issues with this as well).
>
> No, you were told to use *certain* extractors. Most are geared
> toward formatted I/O and are hence irrelevant.
When I use the term "extractor" I am specifically talking about the
overloaded shift operator. That is the only place I've seen the term used
in conjunction with C++ I/O. I /have/ been told 'don't use the overloaded
shift operators because they are for formatted data', with no qualification
added. I *believe* I can use basic_istream<char> &
operator>>(basic_streambuf<char> *sb) to process unformatted binary data.
But I don't know all the consequences, or requirements of using it. This
program works to read in a file and output it. I ran it on the executable
produced by compiling it, and the result was a file of the same size which
also executed successfully. Furthermore diff indicated the input and
output are identical.
#include <iostream>
#include <fstream>
#include <sstream>
using namespace std;
int main(int argc, char * argv[])
{
if(argc!=2) { cerr<<"test filename"<<endl; return -1; }
string filename(argv[1]);
ifstream inf(filename.c_str(), ios::binary);
if(!inf) { cerr<<"can't open "<<filename<<endl; return -1; }
inf.unsetf(ios::skipws);
stringstream ss(ios::binary|ios::in|ios::out);
ss.unsetf(ios::skipws);
ss<<inf.rdbuf();
cout<<"read "<<ss.str().size()<<" bytes of "<<filename<<endl;
filename += ".tmp";
ofstream off(filename.c_str(), ios::binary);
if(!off) { cerr<<"can't open "<<filename<<endl; return -1; }
off.unsetf(ios::skipws);
off<<ss.rdbuf();
return 0;
}
Note that the program worked as stated above, even without the ios::binary
and unsetf(ios::skipws). Are these needed?
Can I safely assume the for used in the program will process the data
unaltered? That is, no transformation of CR/LF, etc.
> What you believe
> about performance is uninteresting until you make measurements
> to support it.
My suspicion may be unfounded since I don't fully understand the mechanisms
involved.
>>> After you posted, I began thinking about why there hasn't been as much
>>> contribution as one might expect to something as important as the C++
>>> Standard Library.
>
> (Despite Dietmar's comments, I did not take personal offense at this
> statement. My contributions to the design of the Standard C++ library
> have been very small. I'm more of an implementer and explainer. But
> the statement certainly shows a lack of appreciation for the many
> real, and sometimes highly inventive, contributions made by others
> to the library. That, to me, reflects arrogance, stupidity, or both.)
That is an absurd statement. My comments in no way provide any indication
of my assessment of the amount of work that has gone into the library that
currently exists. There are many features that could be in the library
that aren't. There are many possible explanations for these features not
to be there. I was merely speculating as to one possible reason why they
are not included.
There is also another possible reason why many people would withold valuable
contributions to the C++ Standard Library.
[...]
I am not the subject of this newsgroup. Please see the FAQ for information
regarding the intended content of the newsgroup and proper etiquette when
posting. You can find a copy of the FAQ here:
http://www.parashift.com/c++-faq-lite/
Steven T. Hatton
>
> I can see where trying to specify a generic file descriptor could be
> problematic. However, I can't imagine it poses anything close to the
> variability that locales pose. Perhaps it really is something best left
> outside of the C++ Standard. My first inclination was to suggest the
> Standard could provide some kind of extensible framework, and use POSIX as
> a default implementation. But, then that imposes a requirement on the
> implementation which may not be easy to meet.
Whoops! I got to thinking about that statement, and realized that, AFAIK,
you can't get a filename from a POSIX file descriptor.
Steven T. Hatton
> Dietmar Kuehl wrote:
>> This is a broken link. Anyway, you don't want to educate me about the
>> IOStreams or stream buffer class hierarchies, do you? I know pretty
>> well how the C++ I/O mechanisms work, you know. ... and I think they
>> are pretty simple to use once you have accepted the basic ideas of
>> using extractors and inserters and/or the use of sequences.
>
> And knowing which iterators are available, as well as what flags to use to
> modify the behavior of the stream. Also relevant is the distinction
> between formatted and unformatted operations. It is well advised also to
> understand that unformatted streams may still alter the underlying data,
> and that formatted operations may alter the data in a so-called "binary"
> stream. It is easy to put the stream into "stupid" format states where
> the wrong combination of flags is set causing the numeric format to behave
> as if the default were set. strm.clear(flag) unsets everything _but_ flag.
> strm.setf(flags) doesn't mean `set flags' even though that's what it does.
> Then there is 'basic_ios' verses 'ios_base'. std::string::clear() does
> something completely different than std::stringstream::clear(). Reading a
> stream completely and correctly puts it into a fail state.
>
> Then there are the necessary complexities of the design to consider.
Add this to the: 'Damn! why did that do that?' list. std::hex is a
manipulator. ios::hex is a flag. std::ostream will accept either one,
with different results. Please note that I have not said there is anything
fundamentally wrong with C++ I/O, there's just a bunch of that kind of
stuff.