Test for a warning

[Paweł Biernat]

I would like to write a test case verifying that a function call results in a warning.  I have seen that there are tests for exceptions, but I was unable to find a test for warining in the docs.

[Paweł Biernat]

A little example

using Base.Test

function f(x)
    if x < 0
        warn("x should be non-negative")
    end
    return x
end

@test_warning f(-1)  #this is the test I am looking for

[Stefan Karpinski]

There is currently no way to do this.

[Jameson Nash]

You could redirect_stderr and test for content written

[Laszlo Hars]

Could someone help with redirecting stderr? For example, the following code does not get the error message shown in the Julia console in Windows 7, Julia Version 0.3.0-prerelease+3789:

~~~

stderr_orig = STDERR

rd, wr = redirect_stderr()

1^-1

close(wr)

eof(rd)

close(rd)

out = readall(rd)

redirect_stderr(stderr_orig)

~~~

[Mike Innes]

function with_out_str(f::Function)

  orig_stdout = STDOUT

  rd, wr = redirect_stdout()

  f()

  redirect_stdout(orig_stdout)

  return readavailable(rd)

end

macro with_out_str(expr)

  :(with_out_str(()->$expr)) |> esc

end

You can use this as

@with_out_str begin

  ... code ...

end

But I think you'll need to change "stdout" to "stderr" in the above definition to capture warnings.

[Laszlo Hars]

I am confused: I can change all the "out"s to "err"s, but what variable will contain the error message to be inspected? The error messages seem to only appear in the console, nowhere else.

[Leah Hanson]

~~~

errormsg = @with_out_str begin

  ... code ...

end

~~~

should put the error message into errormsg.

-- Leah

[Mike Innes]

The macro expression itself will return the output string. So if you type it into a repl you'll get something like

julia> @with_err_str warn("foo")

"\e[1m\e[31mWARNING: foo\n\e[0m"

If you want to capture that string you'd just assign it to a variable:

julia> err = @with_err_str warn("foo");

One thing to be careful of is that those ANSI codes may not be included when running the tests.

[Jameson Nash]

Your code looks generally fine, except that you close the read end before actually reading the data. Also, you don't catch the error, so it aborts execution early. Try wrapping you user code in a try/catch block. 

(Ps can we please get rid of readavailable before 0.3. Also, this is why the function form of redirect_stdio was in my list of features to include in 0.3)

[Laszlo Hars]

I must be doing something stupid. This is what I typed into the console:

~~~

function er(f::Function)

  orig_stderr = STDERR

  rd, wr = redirect_stderr()

  f()

  redirect_stderr(orig_stderr)

  return readavailable(rd)

end

macro er(expr)

  :(er(()->$expr)) |> esc

end

err = @er begin 1^-1 end

err

~~~

I only get:

ERROR: err not defined

Could it be some problem with the Windows console or the new REPL? (I coded similar functions a couple of months ago, and they did work. Not anymore)

[Laszlo Hars]

...can someone at least verify that the 0.3.0 Julia console output in Windows is not written to STDOUT, and error messages are not written to STDERR? (The results of print() do appear on STDOUT)

[Jameson Nash]

julia> begin

       stderr_orig = STDERR

       rd, wr = redirect_stderr()

       try

       warn("HI")

       finally

       redirect_stderr(stderr_orig)

       close(wr)

       out = readall(rd)

       println(length(out),out)

       end

       end

25WARNING: HI

in you example, code, throwing an error does not cause anything to be written to stdout, it is a break in control flow that gets caught by the REPL, skipping execution of all of the rest of your code.

Note that the calls to both redirect_stderr and close are necessary before readall, because we have multiple handles to the write end of the pipe, and need to close all of them before the read end will receive the EOF signal

[Laszlo Hars]

Thanks, Jameson. That explains, why I cannot catch errors, only warnings. That is, the following code will not (and I verified, it does not) work.

~~~

begin

stderr_orig = STDERR

rd, wr = redirect_stderr()

try

error("HI") ###

finally

redirect_stderr(stderr_orig)

close(wr)

out = readall(rd)

println(length(out),out)

end

end

~~~

Is there a trick to catch the error message and return it in a variable?

[Laszlo Hars]

...furthermore: I naively thought the same code would capture the console output in a variable. It does not. What am I still missing?

~~~

begin

stdout_orig = STDOUT

rd, wr = redirect_stdout()

try

1+2

finally

redirect_stdout(stdout_orig)

close(wr)

out = readall(rd)

println((length(out),out))

end

end

~~~

RESULT:

(0,"")

3

That is, the "out" variable is empty, but the result of 1+2, "3" is magically printed in the console window. Again, a very similar piece of code used to capture the console output a couple months ago with the then current 0.3.0 nightly.

[Laszlo Hars]

...and this is what I meant by saying that print() does produce data on STDOUT (but regular console output does not):

~~~

begin

stdout_orig = STDOUT

rd, wr = redirect_stdout()

print(1+2)

redirect_stdout(stdout_orig)

close(wr)

out = readall(rd)

println((length(out),out))

end

~~~

RESULT (as expected):

(1,"3")

[Laszlo Hars]

We could summarize our finding so far about the Julia console output

~ For Windows, with the latest nightly 0.3.0 Julia version

- Warnings are written to STDERR, they can easily be redirected, captured to a variable

- Error messages might or might not get written to STDERR, but they cannot be captured to a variable, because the code execution aborts beforehand [bad]

-- I also failed to get error messages via another thread redirecting STDERR

- The output of print() is written to STDOUT, which can be redirected and captured. This does not seem to matter, because we can directly print to a pipe

- The console output is not written to STDOUT as if it were printed: it cannot be captured to a variable [bad]

Could someone verify these? Especially the last bullet point looks suspect.

I'd like to find workarounds for error messages and regular console output. We can then log them in files, or pass them to other applications via the clipboard.