Stack trace refers to wrong or nonexistent line number?

[Ben Crowell]

Hi, all,

I'm a brand-new julia user trying to convert some matlab code to julia. When I run the code, I am currently getting a stack trace that looks like this:

    0 ERROR: LoadError: TypeError: non-boolean (Array{Bool,2}) used in boolean context
 in tvd at /home/bcrowell/tvd/tvd.jl:145
 in tvd at /home/bcrowell/tvd/tvd.jl:53
 in include at ./boot.jl:254
 in include_from_node1 at loading.jl:133
 in process_options at ./client.jl:306
 in _start at ./client.jl:406
while loading /home/bcrowell/tvd/test.jl, in expression starting on line 3

Am I correct in assuming that in output like "...tvd.jl:145", the 145 is supposed to be a line number in tvd.jl? I couldn't figure out what was wrong, but I noticed that when I fiddled with the code in attempts to fix or track down the error, the 145 was always 145, even when it seemed like it should have changed. I then started trying to make a minimal example, and I ended up with code that was only 114 lines long, but the error was still claimed to be at line 145 -- a line that doesn't exist. This is with julia  0.4.0-dev.

Normally I would not post a question on a newsgroup like this without first whittling my problem down to a minimal example, but this seeming inaccuracy of the line numbers is making it hard for me to do so. I've therefore posted the 114-line version of the code below, just so people can see what I'm talking about and try to reproduce the behavior if they wish.

Without expecting anyone to debug my whole program for me ...

...am I correct that the 145 is supposed to be a line number?
...is this a known issue?

Thanks in advance,

  Ben

===============================================================================================================

# converted from the matlab function tvdip() at http://www.maxlittle.net/software/ , GPL v2
function tvd(y, lambda, display=true, stoptol=1e-3, maxiter=60)
# Search tuning parameters
ALPHA     = 0.01   # Backtracking linesearch parameter (0,0.5]
BETA      = 0.5    # Backtracking linesearch parameter (0,1)
MAXLSITER = 20     # Max iterations of backtracking linesearch
MU        = 2      # t update

N = length(y)    # Length of input signal y
M = N-1          # Size of Dx

# Construct sparse operator matrices
I1 = speye(M,M) # sparse identity matrix
O1 = spzeros(M,1) # column vector initialized with zero
D = [I1 O1]-[O1 I1] # negative differentiation operator, 1 on main diag, -1 to right

DDT = D*D' # second derivative operator, 2 on main diag, -1 on diagonals two steps above and below it
Dy  = D*y

# Find max value of lambda
lambdamax = maximum(abs(DDT\Dy)) # x=A\B ... solves Ax=B

L = length(lambda)
x = zeros(N, L)
s = zeros(L, 1)
E = zeros(L, 1)

# Optimization variables set up once at the start
z    = zeros(M,1)   # Dual variable
mu1  = ones(M,1)    # Dual of dual variable
mu2  = ones(M,1)    # Dual of dual variable

# Work through each value of lambda, with hot-restart on optimization
# variables
for l = 1:L
   
    t    =  1e-10
    step =  Inf
    f1   =  z-lambda[l]
    f2   = -z-lambda[l]

    # Main optimization loop
    s[l] = 1

    for iters = 0:maxiter

        DTz  = (z'*D)'
        DDTz = D*DTz
        w    = Dy-(mu1-mu2)

        # Calculate objectives and primal-dual gap
        pobj1 = 0.5*w'*(DDT\w)+lambda[l]*sum(mu1+mu2)
        pobj2 = 0.5*DTz'*DTz+lambda[l]*sum(abs(Dy-DDTz))
        pobj = min(pobj1,pobj2)
        dobj = -0.5*DTz'*DTz+Dy'*z
        gap  = pobj - dobj

        if display
            @printf("%5d %7.2e %7.2e %7.2e\n", iters, pobj, dobj, gap)
        end

        # Test duality gap stopping criterion
        if gap <= stoptol
            s(l) = 1
            break
        end

        if step >= 0.2
            t = max(2*M*MU/gap, 1.2*t)
        end

        dmu1    = mu1
        dmu2    = mu1

        resDual = rz
        resCent = [-mu1.*f1-1/t; -mu2.*f2-1/t]
        residual= [resDual; resCent]

        # Perform backtracking linesearch
        step = 1
        for i = 1:length(dmu1)
          if dmu1[i]<0
            step = minimum( step, 0.99*min(-mu1[i]./dmu1[i]) ) # I get an error on this line, even if I comment it out!?
          end
        end
        for i = 1:length(dmu2)
          if dmu2[i]<0
            step = minimum( step, 0.99*min(-mu2[i]./dmu2[i]) )
          end
        end

    end

    x(:,l) = y-D'*z
    E(l) = 0.5*sum((y-x(:,l)).^2)+lambda[l]*sum(abs(D*x(:,l)))

    # We may have a close solution that does not satisfy the duality gap
    if iters >= maxiter
        s(l) = 0
    end
   
    if display
        if s(l)
            @printf("Solved to precision of duality gap %5.2e\n", gap)
        else
            @printf("Max iterations exceeded - solution may be inaccurate\n")
        end
    end

end

end

(x,E,s,lambdamax) = tvd([10, 11, 9, 10],1)

[Isaiah Norton]

Unfortunately, this is a known issue. Part of the problem is that we don't currently do a very good job keeping track of source line transitions when doing macro expansion and function inlining (see e.g. issue #1334 on the github issue tracker). However, I get slightly more useful output on a build using a recent LLVM version where we are able to generate somewhat better debug location information (we haven't bumped to the newer version by default due to other major TODOs):

julia> include("/tmp/test.jl")

    0 ERROR: LoadError: TypeError: non-boolean (Array{Bool,2}) used in boolean context

 in tvd at ./printf.jl:145

 in tvd at /tmp/test.jl:4

 in include at ./boot.jl:254

 in include_from_node1 at ./loading.jl:184

while loading /tmp/test.jl, in expression starting on line 114

Poking around a bit more (with a debugger), it looks like the issue is that your 'gap` variable on line 56 is an array. So, the @printf fails. If `display=false` then the error is instead on 63 when comparing the scalar 'stoptol' to the 'gap' variable.

[Tim Holy]

Short of better line numbers (which we all want), a useful trick that often
works is to put `@show L` at the top of your loop; this will reveal which L is
causing the problem. Then, comment out the loop and just set L to this value.

More generally, you're doing a lot of computation at the global scope, and
this is going to be slow. Put things in functions:
http://docs.julialang.org/en/release-0.3/manual/performance-tips/

Best,
--Tim

[Ben Crowell]

Thanks, Isaiah, for spending so much time to help me with this problem -- much appreciated!!

  Ben