Format function usage

[Letitgo]

Hi, GoNum experts,

I'm trying to understand https://godoc.org/github.com/gonum/matrix/mat64#Format function.

the code below doesn't match the expectation of multiple lines of matrix format.

package main
 
import (
	"fmt"
	"github.com/gonum/matrix/mat64"
)
 
func main() {
	m := mat64.NewDense(2, 2, []float64{7.0, 0.5, 0.5, 1.0})
	fmt.Printf("%#v\n", *m)
	fmt.Printf("%v\n", *m)
}

Output:
mat64.Dense{mat:blas64.General{Rows:2, Cols:2, Stride:2, Data:[]float64{7, 0.5, 0.5, 1}}, capRows:2, capCols:2}
{{2 2 2 [7 0.5 0.5 1]} 2 2}

I found the "%#v", "%v" in https://github.com/gonum/matrix/blob/master/mat64/format_test.go, but the result is different.


Moreover, format_test.go is not runnable in my github client.  Did I miss anything?
$ go test format_test.go

# command-line-arguments
./format_test.go:15: undefined: Matrix
FAIL    command-line-arguments [build failed]

[Dan Kortschak]

Format is a stub of sorts. I have been thinking about fixing that, but there are complications I have not figured out how to overcome nicely (go syntax formatting mainly). If you use the fm struct from the test file in external code, you need to replace Matrix with mat64.Matrix.

[Letitgo]

ah.., sorry for the wrong command line in my previous mail, it should be.
go test -run format_test.go
that's no problem.


okay, it's working by adding *mat64.Dense type.

type fm struct {
        Matrix *mat64.Dense
        margin int
}

func (m fm) Format(fs fmt.State, c rune) {
        if c == 'v' && fs.Flag('#') {
                fmt.Fprintf(fs, "%#v", m.Matrix)
                return
        }
        mat64.Format(m.Matrix, m.margin, '.', fs, c)
}

func main() {
        m := mat64.NewDense(3, 3, []float64{1, 2, 3, 4, 5, 6, 7, 8, 9})
        mt := fm{Matrix: m, margin: 3}
        fmt.Printf("%v\n", mt)

[Sonia Keys]

+1

I'm looking at gonum/matrix now after not paying very close attention for a while and this was one of the first things I ran into.  I did something similar:

package main


import (
    "fmt"


    "github.com/gonum/matrix/mat64"
)

type M struct {
    *mat64.Dense
}


func (m M) Format(f fmt.State, c rune) {
    mat64.Format(m.Dense, 0, '.', f, c)
}


const n = 3


func main() {
    m := mat64.NewDense(n, n, nil)
    for i := 0; i < n; i++ {
        m.Set(i, i, 1)
    }
    fmt.Println(M{m})
}

[Dan Kortschak]

Sonia, do you have any suggestions for how this API should look?

Probably we should add a prefix parameter a la the prefix parameter for encoding/json. I would like to add this method to each of the matrix types we provide - presumably with either sensible defaults or with a formatting option method setter (and associated fields), but the issue I haven't figured out how to get around cleanly is how to prevent fmt.Formatter switching from clobbering %#v printing - which I want to keep.

[Sonia Keys]

Oh rats, didn't reply all.  Sorry.

---------- Forwarded message ----------
From: Sonia Keys <soni...@gmail.com>
Date: Wed, Aug 5, 2015 at 9:17 PM
Subject: Re: [gonum-dev] Format function usage
To: Dan Kortschak <dan.ko...@adelaide.edu.au>

Hmm, I don't know exactly what happens with %#v.  I see the doc says it uses GoStringer but I'm not sure of the implications at the moment.  Anyway, the problem is a conflict with using # for the dot function?  How about using space for the dot function instead of #?  Saving a space for a sign doesn't seem as meaningful in this context, whereas the other four flags have standard functions that people might more plausibly want.

What would the prefix parameter do?  Would this be added before the vertical bar on each line?

I wrestled with custom formatters for a library of mine, https://github.com/soniakeys/sexagesimal.  The API there is a little crazier because I defined a whole bunch of custom verbs, which we shouldn't have to do here.  Also, it's crazier because I wanted a format overflow error but I didn't want to compromise the efficiency of my computational types.  I ended up creating a parallel set of types for formatting that embed the computational types.  The idea is to create some efficient values, do some computations, then wrap them in the formattable types just when they need to be output.  It looks a little bulky in godoc, but it turns out to feel fairly smooth in use.  In our case here, the format types might hold the extra data that needs to be passed to mat64.Format.  So my example before would become something like,

 func main() {

    const n = 3

    m := mat64.NewDense(n, n, nil)

    for i := 0; i < n; i++ {

        m.Set(i, i, 1)

    }

    // use once

    fmt.Println(mat64.FmtDense{"  (pre)  ", m, 0, '.'})

    // or

    fm := mat64.FmtDense{m: m} // construct and hold on to it if you like.

    fm.Prefix = "  (pre)  " // change fields as needed

    // and maybe the zero value for dot could mean '.'

    fmt.Printf("% f/n", fm) // with space flag invoking dot function

}

[Dan Kortschak]

On Wed, 2015-08-05 at 21:20 -0400, Sonia Keys wrote:
> Anyway, the problem is a conflict with using # for the dot function?

No, the problem is that fmt.Formatter switching happens before the check
for fmt.GoStringer and blocks the possibility of doing the default %#v
action. In biogo.matrix, I did a trick where you print the dereferenced
pointer value - which doesn't implement fmt.Formatter - and prefix with
'&'. This won't work here for reasons relating to the much wider range
of types we deal with. The consequence of this is that to get %#v
behaviour, we need to reimplement that ourselves. I don't like this
idea.

[Brendan Tracey]

Sorry if I'm missing the bigger picture; I've mostly avoided fighting with fmt.

It seems to me there are two fmt cases we'd like to support. We'd like to support debugging (inside and outside mat64), and we'd also like a nice way to for the users to see what the data is.  With debugging mat64, it's very useful to have simple/nonexisting interference with fmt. I want to see what Go thinks the type is, and I want to be able to see the full data array (even those entries beyond the rows and columns). When seeing what the data is, I want it laid out in some table format that's nice.

It seems that it is hard to override fmt while still supporting the first use case. Instead, how about providing a function to support the second case. We could have

func Sprintf(a Matrix, verb string) string

Where the verb is a proxy for the normal formatting verb. The basic case is to lay the data out as a matrix
[1 2 3
 4 5 6
 7 8 9]
or whatever. %e, %f, %g all do what you'd expect, along with the %5.6e-esque verbs. A # would append the extra information, so something like
Type: *mat64.Dense
Rows: 3
Cols: 2
Data:
[1 2
3 4
5 6]

We could also have a %u (unused) which would mark the areas of the data not in use. So
mat64.Sprintf(mat64.NewTriDense(3, true, nil), "%u)
Would print
[0 0 0
*  0 0
*  * 0]

Finally, it could be nice to support indexing help, so something ilke
        0    1   2
0 [ 10   3    4]
1 [  3    5    7]

As an interesting side note, in computational fluid dynamics, one often talks about the numerical solution "blowing up". I had thought this was because the iterative procedure feeds on itself producing larger and larger solutions, so using the "blowing up" of getting larger and larger. However, some of the older Stanford professors say it was because in the early days NaN was printed by the computer as *. When the numbers got too large, the computer would just print out a bunch of asterisks, making it look like someone had blown up all of your numbers with a bomb.

[Brendan Tracey]

Oh, to add, the actual name Sprintf may be confusing as the behavior is not the same as fmt.Sprintf. We could find an alternate name (perhaps just Format).

[Dan Kortschak]

On Wed, 2015-08-05 at 21:56 -0700, Brendan Tracey wrote:
> func Sprintf(a Matrix, verb string) string

This is one of the options that I have been considering for the past
year. It feels to me like a cop-out.

[Dan Kortschak]

How would the formatter know this?

On Wed, 2015-08-05 at 21:56 -0700, Brendan Tracey wrote:

[Brendan Tracey]

It would type switch and be documented to only work on mat64 values. On non-mat64 values it wouldn’t know and therefore would print the matrix like normal.

> --
> You received this message because you are subscribed to the Google Groups "gonum-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gonum-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Dan Kortschak]

And what would the format of used elements be? It would be nice if verb
modifiers we extendible, but they aren't.

[Brendan Tracey]

I should probably spend more time with the fmt package before adding to this discussion, but oh well. Here’s what my reaction at the moment is to do.

// Format returns the matrix as a nicely formatted string
func Format(a Matrix, verb string) string{
}

Now, inside Format, we can parse the verb string. It will be “[number][.number]rune”. The numbers mean what they mean in fmt, i.e. specify the printing precision. The verbs also mean what they mean in fmt. Naively, this could be done with

var s string
r, c := a.Dims()
for i := 0; i < r; i++{
for j := 0; j < c ; j++{
s += fmt.Sprintf(verb, a.At(i,j))
}
}

Obviously it needs a lot of error checking and doing the actual pretty printing part.

If we wanted to support the “#” we parse it out ourself and then print the header if it’s there. As we are parsing the verb ourselves, we could define whatever language we wanted in the formatting string, so, for example, the space as you suggest in the PR. Therefore,

mat64.Format(a, “%# .5e)

would print all of the entries of a with five decimal places, and those that are known to mat64 to be by definition zero have a . instead of a value. This would be coded by parsing the flag, seeing there is a # and a ‘ ‘, thus printing the header, printing the dots, and passing “%.5e” to Sprintf on all of the elements.

This is somewhat covoluted, so another option is to have
func Format(a Matrix, matVerb string, elemVerb string)

Where elemVerb is what is passed to fmt.Sprintf on each of the values and matVerb are our special verb on printing matrices (dot, header, etc.). This more cleanly establishes what aligns with fmt and what is different.

Lastly, I know there are good reasons to not fully construt a string which is why the fmt.State.

[Dan Kortschak]

On Thu, 2015-08-06 at 22:03 -0600, Brendan Tracey wrote:
> // Format returns the matrix as a nicely formatted string
> func Format(a Matrix, verb string) string{
> }

If this happens, the parameters should be in the reversed order.

> Now, inside Format, we can parse the verb string. It will be
> “[number][.number]rune”. The numbers mean what they mean in fmt, i.e.
> specify the printing precision. The verbs also mean what they mean in
> fmt. Naively, this could be done with
>
> var s string
> r, c := a.Dims()
> for i := 0; i < r; i++{
> for j := 0; j < c ; j++{
> s += fmt.Sprintf(verb, a.At(i,j))
> }
> }
>
> Obviously it needs a lot of error checking and doing the actual pretty
> printing part.

This is the body of the current Format.

> If we wanted to support the “#” we parse it out ourself and then print
> the header if it’s there. As we are parsing the verb ourselves, we
> could define whatever language we wanted in the formatting string, so,
> for example, the space as you suggest in the PR. Therefore,
>
> mat64.Format(a, “%# .5e)
>
> would print all of the entries of a with five decimal places, and
> those that are known to mat64 to be by definition zero have a .
> instead of a value. This would be coded by parsing the flag, seeing
> there is a # and a ‘ ‘, thus printing the header, printing the dots,
> and passing “%.5e” to Sprintf on all of the elements.

I don't think # and ' ' should work together. # only works with v in
fmt.

> This is somewhat covoluted, so another option is to have
> func Format(a Matrix, matVerb string, elemVerb string)
>
> Where elemVerb is what is passed to fmt.Sprintf on each of the values
> and matVerb are our special verb on printing matrices (dot, header,
> etc.). This more cleanly establishes what aligns with fmt and what is
> different.
>
> Lastly, I know there are good reasons to not fully construt a string
> which is why the fmt.State.

I'm beginning to think that we are trying to do too much.

[Brendan Tracey]

How about

Format(header bool, verb string, a Matrix) string

Header == true prints
Type: (whatever the %T output is)
Rows:
Cols:

verb is what is passed per element.

Doesn’t have to be named Format.