ifstream performance vs. fread()
Damien Cymbal
This is probably more implementation specific than I should be posting
here, but I'm hoping to garner a little bit more well-rounded feedback
then if I just post to the ms-specific groups.
The real subject of this message really is: "Can MSVC++ 6.0 fstream
implementation really be this slow?"
I was experiencing dreadful performance with an app (in fact the Java
prototype was running faster than the C++ version, talk about wanting to
hide under a rock....)
Anyway, I traced it down to the file i/o. As a quick hack, I replaced a
few functions using ifstream.read() with fopen()/fread() just for
comparison sake and not really expecting much, and performance jumped way
beyond whatever I expected. As a test, I just wrote a simple app that
read a file as fast as it could using a 16K buffer, optionally either
using streams (ios::in|ios::binary) or fread() ("rb"). In all cases, the
results were not even close. Finally, I compiled my little test app with
cygwin GCC 3.2 on the same machine and those stream/C results are much
more in order of what I expected.
For example sake, reading a 46MB file:
MSVC fread ~250ms
MSVC ifstream ~6550ms
GCC fread ~375ms
GCC ifstream ~410ms
So, have others also seen this discrepancy? It seems crazy to me and I
keep thinking I must be missing something somewhere. Am I resigned to not
using streams for file i/o in MSVC 6? Any first hand experience with
MSVC 7 as far this being resolved?
dc
---
[ comp.std.c++ is moderated. To submit articles, try just posting with ]
[ your news-reader. If that fails, use mailto:std...@ncar.ucar.edu ]
[ --- Please see the FAQ before posting. --- ]
[ FAQ: http://www.jamesd.demon.co.uk/csc/faq.html ]
tom_usenet
Cymbal) wrote:
>Happy New Year All.
>The real subject of this message really is: "Can MSVC++ 6.0 fstream
>implementation really be this slow?"
There is a bug in MSVC6's library that causes buffering to be
needlessly disabled for files opened by name.
See the <fstream> section here:
http://www.dinkumware.com/vc_fixes.html
For some reason (there were some legal wranglings around Dinkumware's
library IIRC) those fixes never made it into a service pack.
As a workaround, I think you can open files using the constructor,
rather than with the "open" member function. e.g. not:
ofstream ofs;
ofs.open("myfile.txt");
but:
ofstream ofs("myfile.txt");
You can also fix it by upgrading to VC 7 (or better, 7.1) or buy the
upgrade VC6 library from www.dinkumware.com
Tom
Stephen Howe
> few functions using ifstream.read() with fopen()/fread() just for
> comparison sake and not really expecting much, and performance jumped way
> beyond whatever I expected. As a test, I just wrote a simple app that
> read a file as fast as it could using a 16K buffer, optionally either
> using streams (ios::in|ios::binary) or fread() ("rb"). In all cases, the
> results were not even close. Finally, I compiled my little test app with
> cygwin GCC 3.2 on the same machine and those stream/C results are much
> more in order of what I expected.
>
> For example sake, reading a 46MB file:
>
> MSVC fread ~250ms
> MSVC ifstream ~6550ms
> GCC fread ~375ms
> GCC ifstream ~410ms
>
> So, have others also seen this discrepancy?
Yes I have. I don't see how any of the C++ iostream library in your
compilers case could be faster than stdio as it built on top of stdio (a
design decision as it makes it portable).
I can't do any better than referring to the thread "Maximum throughput" on
microsoft.public.vc.stl. See
http://groups.google.com/groups?safe=images&ie=UTF-8&oe=UTF-8&as_ugroup=micr
osoft.public.vc.stl&as_usubject=maximum%20throughput&lr=&hl=en
and read it in its entirety. You might find that
std::ifstream if;
:
if.rdbuf()->pubsetbuf(NULL, 16384);
helps on doing a successful open (particularly if reading sequentially in
small chunks).
> It seems crazy to me and I
> keep thinking I must be missing something somewhere. Am I resigned to not
> using streams for file i/o in MSVC 6? Any first hand experience with
> MSVC 7 as far this being resolved?
That I don't know. I would be interested to find out if the library has
changed.
Stephen Howe
Tristan
> I can't do any better than referring to the thread "Maximum throughput" on
> microsoft.public.vc.stl. See
>
>
http://groups.google.com/groups?safe=images&ie=UTF-8&oe=UTF-8&as_ugroup=micr
> osoft.public.vc.stl&as_usubject=maximum%20throughput&lr=&hl=en
>
As usual, too long links get always broken in more parts ... a best solution
could be generating a link using makeashorterlink.com
The following is the link to the discussion above:
http://makeashorterlink.com/?G42E21FE2
Dimitris Kamenopoulos
===================================== MODERATOR'S COMMENT:
I approved the initial article by mistake. I'm approving this response,
since it is a response to something in the newsgroup, but, unless
this discussion drifts to something on-topic, further discussion should
go to another newsgroup.
===================================== END OF MODERATOR'S COMMENT
Damien Cymbal wrote:
> Anyway, I traced it down to the file i/o. As a quick hack, I replaced a
> few functions using ifstream.read() with fopen()/fread() just for
> comparison sake and not really expecting much, and performance jumped way
> beyond whatever I expected. As a test, I just wrote a simple app that
> read a file as fast as it could using a 16K buffer, optionally either
> using streams (ios::in|ios::binary) or fread() ("rb"). In all cases, the
> results were not even close. Finally, I compiled my little test app with
> cygwin GCC 3.2 on the same machine and those stream/C results are much
> more in order of what I expected.
>
> For example sake, reading a 46MB file:
>
> MSVC fread ~250ms
> MSVC ifstream ~6550ms
> GCC fread ~375ms
> GCC ifstream ~410ms
Have you tried compiling with optimizations turned on/debug turned off?
Thore B.Karlsen
Cymbal) wrote:
Yes, and I've seen much worse in my measurements. I no longer use
streams for anything but test code, because for the compilers I have to
use streams are intolerably slow. Can't say I miss streams much, though.
--
Be seeing you.
Scott Mayo
> The real subject of this message really is: "Can MSVC++ 6.0 fstream
> implementation really be this slow?"
I'm probably the heretic on this issue, but if you need speed, stay away
from STL. The C runtime has been optimized and tested and gone over
with a fine tooth comb for many years. It's fast and very stable.
Of course the STL does many wonderful things. If you want your
polymorphing multiphasic autosnarkling object to magically persist itself
with one line of code, you generally don't use fwrite(). But if you just
want to slam bits out fast, stick with the name you learned to trust over
20 years ago: C.
Charles Krug
Did the OP measure and determine from his profiler output that this is
actually the problem?
What I had to do ONCE was create my own istream-alike when it mattered.
I kept the istream interface because it was familiar and, at the end of
the excercise, I'd retained type safety and gained performance.
This was a VERY odd duck piece of code (interfacing with unusual
hardware).
You may simply find that the fstream version is doing ONE thing that you
can avoid in this ONE case, which would be simpler than having two
distinct interfaces to worry about.
W Randolph Franklin
>
> I'm probably the heretic on this issue, but if you need speed,
> stay away from STL. The C runtime has been optimized and tested
> and gone over with a fine tooth comb for many years. It's fast
> and very stable.
YMMV. One version of g++ implemented fread as a loop calling
getc! I discovered this by testing the reading of a large file in
both gcc and g++. The latter was about 20 times slower than the
former.
The concept of transparency in C and C++ is such a nice idea. I'm
quite looking forward to its realization.
Is there any way to see the generated code in a format a little
higher level than assembler?