[Boost-users] [iostreams] why doesn't file_sink flush?

[Andrew Schweitzer]

I'm trying to chain debugging logs together with a filtering_ostream.
I've discovered that I need to call flush() on the filtering_ostream to
get characters to appear in console in a timely fashion. This flush()
call doesn't causes the file to be flushed. After some digging, I've
discovered that the boost::iostreams::file_sink doesn't have the
flushable_tag (or implement flush). std::fstream does have a flush
function, and if I use it directly (not chained into a
filtering_ostream), the flush call works as you might expect.

So what is the appropriate way of implementing this? Chaining multiple
logs together seems like a great way to use iostreams library. And out
of curiosity, why doesn't file_sink implement flushable_tag?

Here is the code I was using:

#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/stream.hpp>
#include <boost/iostreams/tee.hpp>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/categories.hpp>
#include <boost/iostreams/concepts.hpp>
#include <boost/thread.hpp>
#include <boost/thread/xtime.hpp>
#include <iostream>

int main(int argc, char** argv)
{
using namespace std;
using namespace boost;
using namespace boost::iostreams;

filtering_ostream fo;

file_sink fsink("out.txt");
tee_filter<file_sink> tee_fsink(fsink);
fo.push(tee_fsink);

fo.push(cout);

int i = 0;
while(true)
{
fo << i << " ";
//without this, I don't see "one print / second"
//with it, I only see them on cout, but not in out.txt.
fo.flush();

i++;

boost::xtime xt;
boost::xtime_get(&xt, boost::TIME_UTC);
xt.sec += 1;
boost::thread::sleep(xt);
}

return 0;
}

tx

Andy

_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[eg]

Andrew Schweitzer wrote:
> I'm trying to chain debugging logs together with a filtering_ostream.
> I've discovered that I need to call flush() on the filtering_ostream to
> get characters to appear in console in a timely fashion. This flush()
> call doesn't causes the file to be flushed. After some digging, I've
> discovered that the boost::iostreams::file_sink doesn't have the
> flushable_tag (or implement flush). std::fstream does have a flush
> function, and if I use it directly (not chained into a
> filtering_ostream), the flush call works as you might expect.
>
> So what is the appropriate way of implementing this? Chaining multiple
> logs together seems like a great way to use iostreams library. And out
> of curiosity, why doesn't file_sink implement flushable_tag?
>

I am not sure if it is exactly your case, but you may wish to read the
thread at:
http://thread.gmane.org/gmane.comp.lib.boost.user/33355

In particular, read the posts by Jonathan Turkanis (iostreams author).
It may (or may not?) shed some additional light on flush and your needs.

[Andy Schweitzer]

eg wrote:
> Andrew Schweitzer wrote:
>> I'm trying to chain debugging logs together with a filtering_ostream.
>> I've discovered that I need to call flush() on the filtering_ostream
>> to get characters to appear in console in a timely fashion. This
>> flush() call doesn't causes the file to be flushed. After some
>> digging, I've discovered that the boost::iostreams::file_sink doesn't
>> have the flushable_tag (or implement flush). std::fstream does have a
>> flush function, and if I use it directly (not chained into a
>> filtering_ostream), the flush call works as you might expect.
>>
>> So what is the appropriate way of implementing this? Chaining multiple
>> logs together seems like a great way to use iostreams library. And out
>> of curiosity, why doesn't file_sink implement flushable_tag?
>>
>
>
> I am not sure if it is exactly your case, but you may wish to read the
> thread at:
> http://thread.gmane.org/gmane.comp.lib.boost.user/33355
>
> In particular, read the posts by Jonathan Turkanis (iostreams author).
> It may (or may not?) shed some additional light on flush and your needs.

Thanks, I've read that, but it doesn't directly answer my questions. In
particular, a) fstream flush seems to work fine, so I'm curious why
file_sink doesn't call it and b) given that you can't flush a file_sink
what is the appropriate implementation - it seems like there would be
one that is relatively straight-forward.

[eg]

Good questions.
I dont know why file_sink or file_descriptor_sink are not currently
implemented with the flushable_tag.

It looks like it would be very straightforward to add.

My usage of iostreams has always involved the gzip filters, which are
not flushable either, so I haven't run into this and don't have a
solution... other than adding flush support to file_sink.

Is there more to it than adding the flushable_tag, and adding a flush
function to basic_file_sink and basic_file<Ch>, the latter simply
calling pimpl->file_.flush() ?

[Andy Schweitzer]

eg wrote:
> Andy Schweitzer wrote:
>> eg wrote:
>>> Andrew Schweitzer wrote:
>
>>
>> Thanks, I've read that, but it doesn't directly answer my questions.
>> In particular, a) fstream flush seems to work fine, so I'm curious why
>> file_sink doesn't call it and b) given that you can't flush a
>> file_sink what is the appropriate implementation - it seems like there
>> would be one that is relatively straight-forward.
>
>
> Good questions.
> I dont know why file_sink or file_descriptor_sink are not currently
> implemented with the flushable_tag.
>
> It looks like it would be very straightforward to add.
>
> My usage of iostreams has always involved the gzip filters, which are
> not flushable either, so I haven't run into this and don't have a
> solution... other than adding flush support to file_sink.
>
> Is there more to it than adding the flushable_tag, and adding a flush
> function to basic_file_sink and basic_file<Ch>, the latter simply
> calling pimpl->file_.flush() ?

That's a good idea. I don't know, I don't know the library well enough.
That seemed like more code... but maybe a better idea. I tried this,
wrapping fstream. Not sure if there's a reason not to do this:

class flushable_file_sink
{
public:
flushable_file_sink(const std::string& path,
BOOST_IOS::openmode mode = BOOST_IOS::out)
:m_fs(path.c_str(), mode), m_strName(path), m_mode(mode)
{
}

flushable_file_sink(const flushable_file_sink& rOther)
{
m_strName = rOther.m_strName;
m_mode = rOther.m_mode;
rOther.m_fs.close();
m_fs.open(m_strName.c_str(), m_mode);
}

struct category
: output_seekable,
device_tag,
closable_tag,
flushable_tag
{ };

typedef char char_type;

std::streamsize write(const char* buf, std::streamsize n)
{
m_fs.write(buf, n);
return n;
}

bool flush()
{
m_fs.flush();
return true;
}

void close()
{
m_bOpen = m_fs.is_open();
m_fs.close();
}

void close(std::ios_base::openmode mode)
{
m_fs.close();
}

std::streamsize read(char_type* s, std::streamsize n)
{
m_fs.read(s, n);
return n;
}

std::streampos seek( stream_offset off, BOOST_IOS::seekdir way,
BOOST_IOS::openmode which =
BOOST_IOS::in | BOOST_IOS::out )
{
m_fs.seekp(off, way);
m_fs.seekg(off, way);
return off;
}

private:
mutable fstream m_fs;
string m_strName;
BOOST_IOS::openmode m_mode;
bool m_bOpen;
};

[eg]

Andy Schweitzer wrote:
> eg wrote:
>> Andy Schweitzer wrote:
>>> eg wrote:
>>>> Andrew Schweitzer wrote:
>>

> That's a good idea. I don't know, I don't know the library well enough.
> That seemed like more code... but maybe a better idea. I tried this,
> wrapping fstream. Not sure if there's a reason not to do this:
>
> class flushable_file_sink
> {

<snip>

I don't see why not. You just need to make sure the lifetime of your
stream outlives the filter chain which I think the iostreams file_sink
does for you (see:
http://www.boost.org/doc/libs/1_38_0/libs/iostreams/doc/classes/file.html).

Having said that, it might be nice to add a feature request to make
file_sink and file_descriptor_sink "flushable" so that future users
could benefit without having to write their own.

Do you want to create a New Ticket for this at:
https://svn.boost.org/trac/boost/

[Andy Schweitzer]

eg wrote:
>
> Having said that, it might be nice to add a feature request to make
> file_sink and file_descriptor_sink "flushable" so that future users
> could benefit without having to write their own.
>
> Do you want to create a New Ticket for this at:
> https://svn.boost.org/trac/boost/

Done!

https://svn.boost.org/trac/boost/ticket/2998

attached implementation of flushable_file_sink copied from file_sink.

I tried file_descriptor_sink as well, but so far seems to be having some
problems...

I didn't assign ticket to anyone. Should I do that?

[eg]

Andy Schweitzer wrote:
> eg wrote:
>

> https://svn.boost.org/trac/boost/ticket/2998
>

Thanks.

>
> I didn't assign ticket to anyone. Should I do that?

I dont think it is necessary. I believe it defaults to the "owner" of
the component.