Capturing everything sent to stdout, stderr, cout, cerr, clog

[Frederick Virchanza Gotham]

I mentioned a few weeks ago on this newsgroup about how I'm combining two programs into one.

Now I've put a GUI in there too, and so I want to capture all the output from the two programs and display the output in a text box on the screen.

Capturing everything sent to "printf" and "puts" can be achieved in GNU by using "fopencookie" and then changing the value of stdout, as follows:

FILE *const f = fopencookie( ........ );
stdout = f;
stderr = f;

I thought that this would automatically also capture everything from cout, cerr and clog, but it doesn't -- I reckon because under the bonnet, 'cout' is using the file descriptor 1 instead of using a FILE*. I tried constructing a streambuf from the FILE* returned from 'fopencookie' and then assigning this streambuf to cout, but this didn't work -- presumable because 'fileno' doesn't work on the FILE* returned from fopencookie.

So I wrote a new kind of streambuf class:

class streambuf_redirect : public std::streambuf {

using std::streambuf::streambuf;

virtual std::streamsize xsputn(char const *const s, std::streamsize const count)
{
return fwrite(fp_from_fopencookie,1u,count,s);
}
};

This works, and so now I am able to capture all the output sent to standard output by printf, puts, fprint(stderr,...), cout <<, clog <<, cerr <<.

Everything works fine except for one: It doesn't capture the output from "puts". It works on "fputs(stdout,....)", but not on "puts". I've compiled and linked everything with -fno-builtin but that doesn't fix it. I've tried taking the address of 'puts' in a volatile function pointer and then using the function pointer to invoke it, but that doesn't work either.

I've gone on the Github for glibc and compared the implementations of puts and fputs side by side, and I can see why I'm failing to capture puts.

Anyone got any ideas?

[Frederick Virchanza Gotham]

On Sunday, March 12, 2023 at 12:42:06 PM UTC, Frederick Virchanza Gotham wrote:
>
> It doesn't capture the output from "puts".

I figured it out. I can capture the output from puts if I remove unprintable characters afterward:

for ( std::size_t i = 0u; i < s.size(); ++i )
{
using std::isprint;
using std::isspace;

char unsigned const c = static_cast<char unsigned>(s[i]);

if ( false == (isprint(c) || isspace(c)) )
{
s.erase(i,1u);
--i;
}
}

There must be some dodgy bytes in there -- I'll check what they are another time. For now I have a fix to keep me going.

[Frederick Virchanza Gotham]

On Sunday, March 12, 2023 at 1:03:41 PM UTC, Frederick Virchanza Gotham wrote:
>
> For now I have a fix to keep me going.

And to finish it off, I'm able to capture anything written to file descriptor 1(=stdout) or 2(=stderr) with:

ssize_t write(int fd, void const *buf, size_t count)
{
static ssize_t (*const p)(int,void const*,size_t) = reinterpret_cast<ssize_t(*)(int,void const*,size_t)>( ::dlsym(RTLD_NEXT, "write") );

if ( (fd < 1) || (fd > 2) ) return p(fd,buf,count);

extern ssize_t writer(void *cookie, char const *buffer, size_t size); // defined in stubs.cpp.unique_rule

return writer(nullptr, static_cast<char const*>(buf), count);
}

So now I can capture output from:
(1) Standard C functions like printf, puts, fwrite(stdout), fprintf(stderr)
(2) Standard C++, cout <<, cerr <<, clog <<
(3) POSIX functions, write(1), write(2)

[Kenny McCormack]

In article <a47c7ef5-250d-4101...@googlegroups.com>,
Frederick Virchanza Gotham <cauldwel...@gmail.com> wrote:

>On Sunday, March 12, 2023 at 1:03:41PM UTC, Frederick Virchanza Gotham wrote:
>>
>> For now I have a fix to keep me going.
>
>And to finish it off, I'm able to capture anything written to file descriptor
>1(=stdout) or 2(=stderr) with:

First, I generally enjoy your posts. I think you do a good job of "owning
the regs".

That said, it does sometimes seem like you're making a moutain out of a
molehill.

Third, I didn't follow all the ins and outs of your efforts so far, but it
does seem to me that fopencookie() (which I'd never heard of before, and
which looks, er, interesting...) is overkill.

Why not just use freopen() ?


--
There are two kinds of Republicans: Billionaires and suckers.
Republicans: Please check your bank account and decide which one is you.

[Frederick Virchanza Gotham]

On Monday, March 13, 2023 at 3:46:58 PM UTC, Kenny McCormack wrote:
>
> Why not just use freopen() ?

freopen only allows you to redirect to a file (you have to give it the name of the file). I realise that this gives us a plethora of possibilities on Linux because 'everything is a file', but a few things aren't files. The handle you get back from 'fopencookie' is a FILE*, and so if you use 'fileno' on that handle and to try get a file descriptor, it fails -- because there's no such file.

You can see how I used 'fopencookie' to redirect stdout on the mailing list for proposals for the C++ Standard:

https://lists.isocpp.org/std-proposals/2023/03/6038.php

[even andersen]

søndag 12. mars 2023 kl. 13:42:06 UTC+1 skrev Frederick Virchanza Gotham:
> I mentioned a few weeks ago on this newsgroup about how I'm combining two programs into one.
>
>

...
> stdout = f;
Not sure the above is legal.

As far as I know this cannot be done in neither C nor C++, it can be done in posix

(For reference, and since the question was asked, not really c++ though)
(For instance https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html)

You might want to work on the right hand side of the filedescriptor table, assuming:
(FILE *) / iostream ---> <file descriptor> ---> (client side object) ---> | ---> (kernel side object)

with this model you would look at:
pipe(2)
dup(2)
dup2(2)
(dup3(2))
close(2)
read(2)
write(2)
flockfile(3) / funlockfile(3)
fflush(3)
fdopen(3)

This way you would:
Keep stdout as is
Keep fileno(stdout) as is
Redirect what fileno(stdout) refers to

stdout -> | fileno(stdout) | (some object)
| fd_1 | ^
| pipe_wr | >---V
| pipe_rd | <---V

stdout -> | fileno(stdout) | (rewired to refer to pipe_wr)
| fd_1 | (refers to old stdout object)
| pipe_wr | >---V
| pipe_rd | <---V


1) Duplicate fileno(stdout) to keep it around (create a new fd to the client side object) (dup)
2) Create a pipe, with a read end and a write end (create two fds to a new client side object) (pipe)
3) Change fileno(stdout) to refer to the write end of the pipe above (dup2)
4) Read from the read end of the pipe,
-> this is where you capture what is written to stdout, and fileno(stdout)
5) Process what you read, and write it to the filedescriptor you duplicated in 1)


The problem with this approach is that the object layout/processing layout is not specified for C or C++.
Another weak point is that std::cout etc does not necessarily write to stdout/fileno(stdout)

(Not really c++ though)

And if you want it standardized, try this pipeline
(buffered layer) -> (file descriptor layer) -> (client side object) -> (kernel side object)

.. or write a library