Create formatted string

[Dominique Orban]

Sorry if this is a RTFM, but I can't find the answer in the documentation or on the web. I may have missed it. I come from Python where I can build strings with formatted data using a syntax like

s = "pi=%7.1e" % acos(-1)

How do I accomplish that in Julia? @printf doesn't do the job because it doesn't return anything:

julia> s = @printf("%7.1e", 3.14)
3.1e+00
julia> s

Thanks.

[John Myles White]

@sprintf

[Dominique Orban]

Thank you! Such a basic operation could feature a bit more prominently in the documentation.

[Dominique Orban]

As a follow-up question, why is the following not allowed?

julia> fmt = "%8.1e";

julia> @sprintf(fmt, 3.1415)
ERROR: first or second argument must be a format string

I don't see how it's different from

julia> @sprintf("%8.1e", 3.1415)

What's the appropriate syntax?

Thanks.

[John Myles White]

@sprintf is a macro, not a function. It doesn't evaluate its inputs: it just rewrites the inputs into something else (usually less readable) that carries out the actual computation. You can see what it does using the macroexpand function:

julia> macroexpand(quote @sprintf("%8.1e", 3.1415) end)
:(begin # none, line 1:
Base.Printf.sprint(#63#io->begin # printf.jl, line 783:
begin
#59#out = #63#io
#60###x#3463 = 3.1415
local #61#neg, #57#pt, #58#len, #62#exp
if Base.Printf.isfinite(#60###x#3463)
Base.Printf.ini_dec(#60###x#3463,2)
#61#neg = Base.Printf.NEG[1]
#62#exp = Base.Printf.-(Base.Printf.POINT[1],1)
(Base.Printf.-(Base.Printf.-(1,Base.Printf.|((#62#exp Base.Printf.<= -100),(100 Base.Printf.<= #62#exp))),#61#neg) Base.Printf.> 0) && Base.Printf.write(#59#out,' ')
#61#neg && Base.Printf.write(#59#out,'-')
Base.Printf.write(#59#out,Base.Printf.DIGITS[1])
Base.Printf.write(#59#out,'.')
Base.Printf.write(#59#out,Base.Printf.+(Base.Printf.pointer(Base.Printf.DIGITS),1),1)
Base.Printf.write(#59#out,'e')
Base.Printf.print_exp(#59#out,#62#exp)
else
Base.Printf.write(#59#out,begin # printf.jl, line 141:
if Base.Printf.isnan(#60###x#3463)
" NaN"
else
if (#60###x#3463 Base.Printf.< 0)
" -Inf"
else
" Inf"
end
end
end)
end
Base.Printf.nothing
end
end)
end)

-- John

[Mike Innes]

That @sprintf is a macro sort of explains why using a run-time value doesn't work in the same way, but it isn't really the reason since @sprintf(fmt, val) could work in principle – it would just have to delegate to a function if its argument isn't a compile-time string.

If using a run-time string is particularly useful to you, I'd suggest opening an issue about this, since it appears to be missing functionality.

[Milan Bouchet-Valat]

Le samedi 12 avril 2014 à 04:11 -0700, Mike Innes a écrit :

That @sprintf is a macro sort of explains why using a run-time value doesn't work in the same way, but it isn't really the reason since @sprintf(fmt, val) could work in principle – it would just have to delegate to a function if its argument isn't a compile-time string.

If using a run-time string is particularly useful to you, I'd suggest opening an issue about this, since it appears to be missing functionality.

I had argued printf() and sprintf() should be functions taking non-standard string literals, instead of being macros. Then you could also pass them a (let's say) Format object created at runtime if needed. But I didn't have a concrete use case -- maybe you have one. See
https://github.com/JuliaLang/julia/issues/5747


Regards

[Kevin Squire]

I have a port of a BSD printf function which works at about half the speed of the @printf macro. I was hoping to make it more functional/less ugly, but I'll see if I can't get it into a pull request, at least, or in a package if it's not accepted. 

I'll also point out Dahua's Formatting.jl package, which offers python-style formatting. 

Cheers,

  Kevin

[Stefan Karpinski]

Note that to match the functionality of our @printf macro, it has to handle different types of arguments correctly, which is non-trivial. Not saying it's impossible, but it isn't easy.

[Dominique Orban]

I think that would be a very worthy addition though. A use case for me is to read and write sparse matrices to file in Rutherford-Boeing and Harwell-Boeing format. Those formats are really born out of the Fortran 77 school. They consist in a header that gives Fortran format strings to be used to read the integer and real data (or complex) data to follow. So we would still need something that translates Fortran formats to C formats, and that's another story. A band-aid solution could be to interface the C library RBio (http://www.cise.ufl.edu/research/sparse/RBio) but a Julia implementation would be more elegant. For what it's worth, here's my Python implementation: https://github.com/dpo/pyorder/blob/master/pyorder/tools/hrb.py.

[Stefan Karpinski]

I have yet to see an example that can't be handled with compile time format generation and evaling function definitions.

[Jeff Waller]

Likewise I am having problems with @sprintf

Is this because @sprinf is macro?  The shorthand of expanding a printf with format the contents of an array is desirable.  I would have expected the ... operator to take an array of length 2 and turn it into 2 arguments.

   

    julia> X=[1 2]
   1x2 Array{Int64,2}:
    1  2

    julia> @sprintf("%d%d",1,2)
    "12"

    julia> @sprintf("%d%d",X...)
    ERROR: @sprintf: wrong number of arguments

    julia> @sprintf("%d%d",(1,2)...)
    ERROR: @sprintf: wrong number of arguments

    julia> @sprintf("%d",X...)
    ERROR: error compiling anonymous: unsupported or misplaced expression ... in function anonymous
    in sprint at io.jl:460
    in sprint at io.jl:464

    julia> macroexpand(quote @sprintf("%d%d",X...) end)
    :($(Expr(:error, ErrorException("@sprintf: wrong number of arguments"))))

[John Myles White]

As far as the macro is concerned, the splat isn’t executed: it’s just additional syntax that gets taken in as a whole expression.

The contrast between how a function with splatting works and how a macro with splatting works might be helpful:

julia> function splat(a, b...)
println(a)
println(b)
return
end
splat (generic function with 2 methods)

julia> splat(1, 2, 3)
1
(2,3)

julia> splat(1, [2, 3]...)
1
(2,3)

julia> macro splat(a, b...)
println(a)
println(b)
:()
end

julia> @splat(1, 2, 3)
1
(2,3)
()

julia> @splat(1, [2, 3]...)
1
(:([2,3]...),)
()


— John

[Dominique Orban]

So what's the preferred Julia syntax to achieve what I meant here:

julia> fmt = "%8.1e";
julia> @sprintf(fmt, 3.1415)
ERROR: first or second argument must be a format string

[Mike Innes]

It occurs to me that, if you really need this, you can define

sprintf(args...) = eval(:@sprintf($(args...)))

It's not pretty or ideal in terms of performance, but it will do the job.

fmt = "%8.1e"

sprintf(fmt, 3.141) #=> " 3.1e+00"

[Stefan Karpinski]

Please don't do this – or if you do and your program is amazingly slow, then consider yourself warned. You can define a custom formatting function pretty easily:

julia> fmt = "%8.1e"

"%8.1e"

julia> @eval dofmt(x) = @sprintf($fmt, x)

dofmt (generic function with 1 method)

julia> dofmt(1)

" 1.0e+00"

julia> dofmt(123.456)

" 1.2e+02"

The difference is that you compile the function definition with eval *once* and then call it many times, rather than calling eval every time you want to print something.

[Jeff Waller]

That's pretty cool, but I have a followup.

Does this mean

1) Writing macros is implicitly harder because they don't deal with (splatted) collections as easy as functions?

2) This is not really important because it hardly is ever done?

3) sprintf should be re-written as a function?

4) sprintf should be modified to deal with this?

5) sprintf can't be modified easily to deal with this and still be efficient (see #1-#3)

[Jameson Nash]

I would pick (4): sprintf should be modified to deal with it.

My reason: sprintf knows how many arguments it is expecting, so if it
sees a `...` on the last argument, it could easily emit code to
extract the elements that it needs


Also (1) / (2): splatting arguments is something that happens in a
function call at runtime. A macro is neither since it deals with the
AST (the form of the code) before it gets executed, at compile time.
Since X doesn't exist at compile time, `X...` can't be expanded yet
either. However, the sprintf macro can choose to take steps to
accommodate this syntax, or it can choose to make it an error.
Currently it's just undefined, probably because nobody has tried it
before, and then opened an issue when it didn't work.

[Stefan Karpinski]

Up for grabs issue: https://github.com/JuliaLang/julia/issues/6520. If anyone is interested in doing a bit of metaprogramming, this is a good opportunity.

[Jeff Waller]

On Monday, April 14, 2014 10:58:15 AM UTC-4, Stefan Karpinski wrote:

Up for grabs issue: https://github.com/JuliaLang/julia/issues/6520. If anyone is interested in doing a bit of metaprogramming, this is a good opportunity.

Oh!  Me, I mean I'm saying all these things, I should also invest too.  But I hardly know where to start.   

[Stefan Karpinski]

The macros are defined here:

https://github.com/JuliaLang/julia/blob/master/base/printf.jl#L750-L784

It should be a matter of detecting that there's a splat happening – keep in mind that splats can be mixed with other arguments – and emitting the appropriate code in the macro. It may be possible by adding a single line that does something along these lines:

julia> argsʹ = tuple(args...)

(1,2,3,4,5)

and then uses argsʹ instead of args afterwards. You'd also have to change the argument count checking in that case to be done at runtime instead of macro expansion time.

[Jeff Waller]

On Monday, April 14, 2014 11:36:40 AM UTC-4, Stefan Karpinski wrote:

The macros are defined here:

https://github.com/JuliaLang/julia/blob/master/base/printf.jl#L750-L784

Do I need to git the most up-to-date source for Julia as well and make a language development environment?  What's normally done here?

[Tony Kelman]

You can work from a binary installation to test your changes locally, but you'll need to rebuild the "system image" for changes to Julia code in base to take effect. Last I checked this detail is not explicitly documented anywhere, adding a sentence or two to CONTRIBUTING.md about this would be a good idea. On Windows there's a batch file bin/prepare-julia-env.bat to do this for you. Not sure whether the equivalent exists for the Ubuntu PPA or OS X dmg binary packages.

[Dominique Orban]

How would one go about benchmarking a set of implementations like those?

[Stefan Karpinski]

I'm not sure what you mean, but doing things in a loop and timing it is the normal way. The lack of usefulness of my answer may be indicative that I don't understand the question.

[John Myles White]

I think the question is how to time a proposed sprintf() function vs. the existing @sprintf macro.

 -- John

[Dominique Orban]

Here are some timings comparing @printf with the proposed @eval option. I also wanted to try a variant that calls libc's printf directly. I came up with this implementation: https://gist.github.com/11000433. Its "advantage" is that you can print an array's address using %p (for what it's worth).

I didn't find a way around another @eval due to what ccall expects its arguments to look like. I thought it would be easy to call libc's printf, but it wasn't! (Also I'm sure cprintf should be more sophisticated than it currently is, but for now, it does what I need.)

Running time_printf.jl on my Macbook pro gives the following timings:

   macro      eval      libc
8.06e-05  5.91e-02  6.63e-03

These are averages over 1000 calls. The call to libc's printf isn't doing too badly compared to the simpler @eval proposed by Stefan. But I'm wondering if it's possible to avoid the @eval in cprintf and call the C function directly?!

Are there other options?

I'm all for performance but when it comes to printing, convenience and flexibility are also a must in my opinion. Because printing is inherently inefficient, I'm willing to accept performance hits there.

Many thanks for all the help.

ps: Shoudln't @time return the execution time?

[Tony Fong]

For formatting just one number into a string (instead of multi-arg sprintf) you can use sprintf1 in https://github.com/tonyhffong/NumFormat.jl. It's somewhat close to standard macro speed.

[Dominique Orban]

This is an old question about computed format strings, but it's still biting me. I've been following your suggestion and I defined

print_formatted(fmt, args...) = @eval @printf($fmt, $(args...))

Now I am in a situation where fmt is computed inside a function, and my function executes in roughly 26 seconds when using the above print_formatted(). If I cheat and pretend I know the format beforehand and use @printf, the function executes in under one tenth of a second!

My question:

Once the format is known, is there a way to take advantage of this fact performance-wise? What I mean is something of the form

   print_formatted(args...) = @eval @printf($fmt, $(args...));  # now fmt is known and fixed

Unfortunately, defining the above after fmt has been computed isn't any faster than the first print_formatted. Is there a better option?

Falling back on my old c_printf (that calls libc directly; see a previous message in this thread) proves to be the fastest option so far after @printf, with a run time of just under one second.

Thanks!

[Tony Fong]

Try Formatting.jl? (https://github.com/lindahua/Formatting.jl)