System.out and System.err
Mike Schilling
thread-safe; if we write to them from more than one threads, the worst
that will happen is output getting interleaved but not lost, nor will
exceptions be caused by unsynchronized access (e.g., we don't expect
internal bufferr-copying routines to get confused.) Looking, though,
I can't find this guarnteed anywhere. Can anyone point to an offical
statement about it?
Jan Thomä
<mscotts...@hotmail.com> said:
Well the only "official" statement I have found looking at the
implementation of PrintStream which System.out and System.err are
internally. The write( .... ) methods are all internally synchronized,
so they will not produce weird results when multiple threads access
them:
public void write(int b) {
try {
synchronized (this) {
ensureOpen();
out.write(b);
if ((b == '\n') && autoFlush)
out.flush();
}
}
catch (InterruptedIOException x) {
Thread.currentThread().interrupt();
}
catch (IOException x) {
trouble = true;
}
}
Andreas Leitgeb
> On 2008-10-02 00:09:52 +0200, "Mike Schilling"
><mscotts...@hotmail.com> said:
>> I think we all assume that access to System.out and System.err is
>> thread-safe; if we write to them from more than one threads, the worst
>> that will happen is output getting interleaved but not lost,
> Well the only "official" statement I have found looking at the
> implementation of PrintStream which System.out and System.err are
> internally. The write( .... ) methods are all internally synchronized,
If (e.g. on unix) you start some java-application like this:
java MyApp > /tmp/MyAppLog 2> /tmp/MyAppLog
(with ">", not ">>" !)
I could imagine that they might still overwrite each other, despite
being thread-safe each on its own.
Haven't tried it, though... maybe it's safe, anyway. I just wouldn't
bet on it.
Andreas Leitgeb
> If (e.g. on unix) you start some java-application like this:
> java MyApp > /tmp/MyAppLog 2> /tmp/MyAppLog
> (with ">", not ">>" !)
Meanwhile I did the test (with non-java prog "tee", but surely
java-progs writing both to stdout and -err also behave same way):
$ echo foobar | tee /dev/stderr
foobar
foobar
$ echo foobar | tee /dev/stderr >bar 2>bar
$ cat bar
foobar
$
> I just wouldn't bet on it.
Now I do. Any takers? :-)
Sabine Dinis Blochberger
Hm. I'd say yes, because it's pipes and one program run. As opposed to
repeated runs, where it would replace the file each time.
Jan Thomä
<a...@gamma.logic.tuwien.ac.at> said:
> If (e.g. on unix) you start some java-application like this:
>
> java MyApp > /tmp/MyAppLog 2> /tmp/MyAppLog
>
> (with ">", not ">>" !)
> I could imagine that they might still overwrite each other, despite
> being thread-safe each on its own.
Well but this is something completely out of the control of the
program. Blaming the program for this would be like blaming it for not
being able to write on a defective disc...
Andreas Leitgeb
> Andreas Leitgeb wrote:
>> $ echo foobar | tee /dev/stderr >bar 2>bar
>> > I just wouldn't bet on it.
>> Now I do. Any takers? :-)
>
> Hm. I'd say yes, because it's pipes and one program run. As opposed to
> repeated runs, where it would replace the file each time.
What pipes are you talking about?
The one between cat and tee isn't relevant.
$ tee <<<foobar /dev/stderr >bar 2> bar
does just the same.
There was also no mention of repeated runs.
The question was: if writing to System.out and to System.err wildly mixed,
can that result in loss of data?
It can, but only if both these channels(streams) are externally redirected
to the same plain file (and without the O_APPEND flag set).
You then have two separate filedescriptors each with its own write-cursor,
and whoever writes last to a specific position will overwrite data that
was previously written by the other filedescriptor.
I don't know if that scenario is relevant to the OP, though.
Wojtek
But now you are replying on the OS to manage concurrency. This has
nothing to do with Java.
Depending on the OS strange things may happen, or, one or the other
will get an error that the file is locked by another process.
--
Wojtek :-)
Andreas Leitgeb
> On 2008-10-02 11:56:33 +0200, Andreas Leitgeb
><a...@gamma.logic.tuwien.ac.at> said:
>
>> If (e.g. on unix) you start some java-application like this:
>>
>> java MyApp > /tmp/MyAppLog 2> /tmp/MyAppLog
>>
>> (with ">", not ">>" !)
>> I could imagine that they might still overwrite each other, despite
>> being thread-safe each on its own.
>
> Well but this is something completely out of the control of the
> program.
Indeed it is. Let's recall Mike's words:
... the worst that will happen is output getting interleaved
but not lost, ...
Now, depending on external conditions, the application itself
may overwrite its previous output to System.err by outputting
to System.out and/or vice versa.
Output *can* get lost, while the application is still running,
and *without* anyone *else* removing or truncating the file at
the same time.
class Test {
public static void main(String[] args)
{
System.out.print("@");
for (char ch='a'; ch <= 'z';) {
System.err.print(""+ch+ch); ch++;
System.out.print(""+ch+ch); ch++;
}
}
}
// I was too lazy to look at the javadocs for String on how
// better to create a string of instances of a given char.
Let it run on console and you see each char twice, ditto
when redirecting >bar 2>&1, but if you redirect both
individually to the same file, then each but the z will
appear only once.
> Blaming the program for this would be like blaming it for not
> being able to write on a defective disc...
I didn't *blame* anyone or anything for anything.
Nigel Wade
> Jan Thomä <jant...@janthomae.de> wrote:
>> On 2008-10-02 11:56:33 +0200, Andreas Leitgeb
>><a...@gamma.logic.tuwien.ac.at> said:
>>
>>> If (e.g. on unix) you start some java-application like this:
>>>
>>> java MyApp > /tmp/MyAppLog 2> /tmp/MyAppLog
>>>
>>> (with ">", not ">>" !)
>>> I could imagine that they might still overwrite each other, despite
>>> being thread-safe each on its own.
>>
>> Well but this is something completely out of the control of the
>> program.
>
> Indeed it is. Let's recall Mike's words:
> ... the worst that will happen is output getting interleaved
> but not lost, ...
The output from the program isn't getting lost. It is getting lost *after* it
has left the program. You could just as easily redirect one of the channels to
the /dev/null bit-bucket and claim that demonstrated the output could get lost.
>
> Now, depending on external conditions, the application itself
> may overwrite its previous output to System.err by outputting
> to System.out and/or vice versa.
You have not demonstrated that the program is overwriting its own output. That
is being done by shell redirection.
>
> Output *can* get lost, while the application is still running,
> and *without* anyone *else* removing or truncating the file at
> the same time.
>
But someone else *is* doing it, the shell. The application is not even writing
to a file...
> class Test {
> public static void main(String[] args)
> {
> System.out.print("@");
> for (char ch='a'; ch <= 'z';) {
> System.err.print(""+ch+ch); ch++;
> System.out.print(""+ch+ch); ch++;
> }
> }
> }
> // I was too lazy to look at the javadocs for String on how
> // better to create a string of instances of a given char.
>
> Let it run on console and you see each char twice, ditto
> when redirecting >bar 2>&1, but if you redirect both
> individually to the same file, then each but the z will
> appear only once.
>
>> Blaming the program for this would be like blaming it for not
>> being able to write on a defective disc...
> I didn't *blame* anyone or anything for anything.
But you ascribed the problem to the application, when the fault lies in shell
mis/redirection.
--
Nigel Wade
Mike Schilling
> Jan Thom� <jant...@janthomae.de> wrote:
>> On 2008-10-02 11:56:33 +0200, Andreas Leitgeb
>> <a...@gamma.logic.tuwien.ac.at> said:
>>
>>> If (e.g. on unix) you start some java-application like this:
>>>
>>> java MyApp > /tmp/MyAppLog 2> /tmp/MyAppLog
>>>
>>> (with ">", not ">>" !)
>>> I could imagine that they might still overwrite each other,
>>> despite
>>> being thread-safe each on its own.
>>
>> Well but this is something completely out of the control of the
>> program.
>
> Indeed it is. Let's recall Mike's words:
> ... the worst that will happen is output getting interleaved
> but not lost, ...
But it's a wholly different situation: here you have two streams
writing to the same external resource, not multithreaded access to the
same stream.
Andreas Leitgeb
I never meant to attribute any fault or guilt to any application.
I imagined the OP wondering, why some of the redirected output of
his application disappeared, and therefore asking here, whether
it's his own fault, for not having cared for synchronisation himself.
I just offered another explanation (outside of Java's scope) for a
symptom that at first glance may look like a lack of synchronisation
inside a Java-app.
Your comparision with /dev/null is principially valid, but the
results of that would be easily recognizeable, which I think is
not true for my suggested cause.
PS:
the Java-application may also have opened some file twice and
assigned the two streams to System.out and System.err. So it
doesn't really require any shell-magic around it.
So, the real point is: if two streams *independently* write to
the same disk-file, then there is a chance, that the result *may
look* as if something went wrong with the streams themselves,
even though it didn't.
Andreas Leitgeb
> But it's a wholly different situation: here you have two streams
> writing to the same external resource, not multithreaded access to the
> same stream.
Your original text mentioned both .out and .err and multiple
threads.
I took it as: one thread writes to .out, another one to
.err ... and that was my interpretation of the problem.
The only way they could possibly affect each other, would
be if they were redirected to the same file ... voila.
Ok, ok, ok, As you clarified, I got it wrong.
So I "solved" a different unstated problem. ;-)
John B. Matthews
Jan Thom� <jant...@janthomae.de> wrote:
> On 2008-10-02 00:09:52 +0200, "Mike Schilling"
> <mscotts...@hotmail.com> said:
>
> > I think we all assume that access to System.out and System.err is
> > thread-safe; if we write to them from more than one threads, the
> > worst that will happen is output getting interleaved but not lost,
> > nor will exceptions be caused by unsynchronized access (e.g., we
> > don't expect internal bufferr-copying routines to get confused.)
> > Looking, though, I can't find this guarnteed anywhere. Can anyone
> > point to an offical statement about it?
>
> Well the only "official" statement I have found looking at the
> implementation of PrintStream which System.out and System.err are
> internally. The write( .... ) methods are all internally
> synchronized, so they will not produce weird results when multiple
> threads access them:
[...]
Interesting. IIUC, not all implementations are so well behaved, though.
For some years, Michael B. Feldman has taught students of Ada and Java
about threads using code similar to that below, which he kindly shares
with us.
<code>
import java.util.Random;
/**---------------------------------------------------------------
* Demo of simple thread class
* Last Modified: March 2008
* @author Michael B. Feldman, mfeldman at gwu.edu
*--------------------------------------------------------------*/
public class ShowThreads {
public static void main(String args[]) {
SimpleThread ThreadA = new SimpleThread("A", 5, 1);
SimpleThread ThreadB = new SimpleThread("B", 7, 21);
SimpleThread ThreadC = new SimpleThread("C", 4, 41);
SafeScreen.clearScreen();
ThreadA.start();
ThreadB.start();
ThreadC.start();
}
} // end ShowThreads
/**---------------------------------------------------------------
* Simple Java thread example
* Last Modified: March 2008
* @author Michael B. Feldman, mfeldman at gwu.edu
*--------------------------------------------------------------*/
class SimpleThread extends Thread {
private static final Random random = new Random();
private String name = "Default";
private int count = 0;
private int column = 0;
// constructor
public SimpleThread(String name, int count, int column) {
this.name = name;
this.count = count;
this.column = column;
}
// run method invoked when 'start' called
public void run() {
for(int i = 1; i <= count; i++) {
int nap = random.nextInt(7) + 1;
SafeScreen.write(name + " naps " + nap + " secs", i, column);
try {
sleep(nap * 1000);
}
catch(InterruptedException e) {} // ignored
}
}
} // end class SimpleThread
/**---------------------------------------------------------------
* Thread-safe Mini-terminal controller for vt100 (ANSI) terminals
* Translated from C to Java February 2001
* @author Michael B. Feldman, mfeldman at gwu.edu
*--------------------------------------------------------------*/
class SafeScreen {
public static synchronized void clearScreen() {
System.out.print("\033" + "[2J");
}
public static synchronized void write(
String item, int row, int col) {
System.out.println("\033" + "[" + row + ";" + col + "f" + item);
}
} // End of class SafeScreen
</code>
[The code is Professor Feldman's; any errors are mine.]
--
John B. Matthews
trashgod at gmail dot com
home dot woh dot rr dot com slash jbmatthews
Sabine Dinis Blochberger
> Sabine Dinis Blochberger <no....@here.invalid> wrote:
> > Andreas Leitgeb wrote:
> >> $ echo foobar | tee /dev/stderr >bar 2>bar
> >> $ cat bar # -> only one foobar
> >> > I just wouldn't bet on it.
> >> Now I do. Any takers? :-)
> >
> > Hm. I'd say yes, because it's pipes and one program run. As opposed to
> > repeated runs, where it would replace the file each time.
>
> What pipes are you talking about?
> The one between cat and tee isn't relevant.
> $ tee <<<foobar /dev/stderr >bar 2> bar
> does just the same.
>
Hm, my knwoledge on these is admittedly fuzzy - aren't the System.out
and System.err named pipes of the OS?
> There was also no mention of repeated runs.
> The question was: if writing to System.out and to System.err wildly mixed,
> can that result in loss of data?
> It can, but only if both these channels(streams) are externally redirected
> to the same plain file (and without the O_APPEND flag set).
>
> You then have two separate filedescriptors each with its own write-cursor,
> and whoever writes last to a specific position will overwrite data that
> was previously written by the other filedescriptor.
>
I get that now, thanks.
Andreas Leitgeb
> Andreas Leitgeb wrote:
>> What pipes are you talking about?
>> The one between cat and tee isn't relevant.
>> $ tee <<<foobar /dev/stderr >bar 2> bar
>> does just the same.
> Hm, my knwoledge on these is admittedly fuzzy - aren't the System.out
> and System.err named pipes of the OS?
Not necessarily. The Java .out and .err (and .in, too) are of course
just streams (which are Java objects implementing a certain interface).
They *may* (directly or indirectly) wrap OS filedescriptors, (but could
as well be entirely independent from the OS, like ByteArrayOutputStreams).
So far you surely knew that, anyway.
Filedescriptors on Unix can refer to pipes, sockets, devices, and just
as well also plain files (and probably even more).
>> I don't know if that scenario is relevant to the OP, though.
No, as it turned out, it wasn't. Doesn't matter; maybe,
someone else has picked up something useful from it.
Arne Vajhøj
I don't think it is guaranteed to be so.
It is well known that some well known implementations are
synchronizing, which is why it for many many years has been
considered where bad practice to use System.out.println for
debug output in multi threaded server context (Java EE).
Arne