gonum + vgo

Sebastien Binet

unread,

May 11, 2018, 8:02:06 AM5/11/18

to gonum-dev

hi there,

I haven't managed to get the attention of anybody on this vgo related (meta) issue:

- https://github.com/gonum/gonum/issues/454

golang/go#23970: x/vgo: allow cross-module internal imports based on import path prefix
golang/go#23960: x/vgo: module path contains non-final version element v1

I suspect/hope the first one will get resolved eventually (it's blocking for us because of our gonum/internal package)

the second one doesn't seem to get much attention from Go devs.

should we ping them back for feedback or should we just bite the bullet and do a `s/v1/x/` ?

-s

Dan Kortschak

unread,

May 11, 2018, 8:39:07 AM5/11/18

to Sebastien Binet, gonum-dev

The first one is not blocking for us, it's blocking for golang/go if
they care, but gonum/internal and all the other old packages are
deprecated and won't be changed.

The second one is troubling. I think pinging is in order. There is no
justification for forcing us to change this.

On Fri, 2018-05-11 at 14:01 +0200, Sebastien Binet wrote:
> hi there,
>
> I haven't managed to get the attention of anybody on this vgo related
> (meta) issue:
> - https://github.com/gonum/gonum/issues/454
>
>

> - golang/go#23970 <https://github.com/golang/go/issues/23970>:

> x/vgo:
> allow cross-module internal imports based on import path prefix

> - golang/go#23960 <https://github.com/golang/go/issues/23960>:

Sebastien Binet

unread,

May 11, 2018, 9:00:33 AM5/11/18

to Dan Kortschak, gonum-dev

On Fri, May 11, 2018 at 2:39 PM Dan Kortschak <dan.ko...@adelaide.edu.au> wrote:

The first one is not blocking for us, it's blocking for golang/go if
they care, but gonum/internal and all the other old packages are
deprecated and won't be changed.

The second one is troubling. I think pinging is in order. There is no
justification for forcing us to change this.

done.

-s

speter

unread,

May 11, 2018, 1:52:34 PM5/11/18

to Sebastien Binet, Dan Kortschak, gonum-dev

I'm far from being a vgo expert, but based on reading https://research.swtch.com/vgo-intro and some faqs, I believe the current gonum.org import path scheme is fundamentally incompatible with vgo: in vgo, version numbers in the import path go *under* the module path, where the module path is the path for go.mod. I think this is the primary reason for the error: v1 occurs before the module top, which doesn't make sense from a vgo perspective: in vgo the unit of version management is the module, and vgo cannot control versions at a level that is not the module top, so it cannot do anything with "gonum.org/v1".

In addition, in vgo, by convention, v0 and v1 is implicit and usually not included in the import path (not sure if it's still acceptable to state it explicitly or not).

So it seems likely to me that if you want to allow using vgo to manage versions, there is need to adjust import paths to vgo's convention. Also, you may want to reconsider using a custom domain: pre-vgo you needed a custom domain for versioning within a single repo (I've been using gopkg.in for my personal projects), but with vgo this will not be necessary, you can use different versions with a github import path, so you may want to reconsider just using the github paths as the canonical import paths in the future. Of course, you can also keep using the custom domain if you like, just with a different path scheme, where the version number doesn't come before the module root.

Peter

--
You received this message because you are subscribed to the Google Groups "gonum-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gonum-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sebastien Binet

unread,

May 22, 2018, 8:38:38 AM5/22/18

to Dan Kortschak, gonum-dev

hi,

hum... it doesn't seem pinging did bring more attention.

and now, `vgo` has been accepted.

- https://www.reddit.com/r/golang/comments/8l3y4k/the_vgo_prposal_is_accepted/

- https://github.com/golang/go/issues/24301#issuecomment-390766926

what should we do ?

-s

Brendan Tracey

unread,

May 22, 2018, 11:57:23 AM5/22/18

to gonum-dev

Thanks Dan for pinging on the issue.

I haven't been on top of this as I should be, so I'd like to check I understand.

- As of right now, it seems like we need to break our current system, instead of "gonum.org/v1/gonum", we would likely just have "gonum.org/gonum".

- We currently don't have any versions at all. While we would like to be at version 1, I don't think we're there yet (though we're getting closer). These changes do imply we should try and have some minor version tags, 0.1, 0.2 etc. to at least work with vgo somewhat

- Let's say we do get a version 1, but in the future we hove to a version 2. This would look like "gonum.org/gonum/v2", with imports like "gonum.org/gonum/v2/mat" . It would be represented on git with a version tag, but the code would still live in "github.com/gonum/gonum"?

- Let's say we eventually make the root `gonum` a package for some reason. How would a user use the v2 version of this? It seems like the import path would be simply "gonum.org/gonum/v2", so does that mean all import statements have to be written as `import gonum "gonum.org/gonum/v2" `? That seems really ugly.

Thanks.

Sebastien Binet

unread,

May 22, 2018, 12:11:02 PM5/22/18

to Brendan Tracey, gonum-dev

On Tue, May 22, 2018 at 5:57 PM Brendan Tracey <tracey....@gmail.com> wrote:

Thanks Dan for pinging on the issue.

I haven't been on top of this as I should be, so I'd like to check I understand.

- As of right now, it seems like we need to break our current system, instead of "gonum.org/v1/gonum", we would likely just have "gonum.org/gonum".

I think I'd rather have it `gonum.org/x/gonum` and `gonum.org/x/plot` (to more easily organize handlers and end-points on the gonum.org web site end).

but, yeah, if nobody changes the current behaviour of `vgo` (or go-1.11), dropping `gonum.org/v1/xxx` would be required.

- We currently don't have any versions at all. While we would like to be at version 1, I don't think we're there yet (though we're getting closer). These changes do imply we should try and have some minor version tags, 0.1, 0.2 etc. to at least work with vgo somewhat

yes.

but we should tag them as v0.1.0, v0.2.0, etc.. to better follow what vgo expects.

(for go-hep, I was using v0.1, v0.2, ... but this confused vgo. had to add an extra `.0`...)

- Let's say we do get a version 1, but in the future we hove to a version 2. This would look like "gonum.org/gonum/v2", with imports like "gonum.org/gonum/v2/mat" . It would be represented on git with a version tag, but the code would still live in "github.com/gonum/gonum"?

yes.

- Let's say we eventually make the root `gonum` a package for some reason. How would a user use the v2 version of this? It seems like the import path would be simply "gonum.org/gonum/v2", so does that mean all import statements have to be written as `import gonum "gonum.org/gonum/v2" `? That seems really ugly.

I believe one would just have to write:

```

import (

"gonum.org/gonum/v2"

"gonum.org/gonum/mat/v2"

)

```

but I am not completely sure about the `mat` import.

also, the vgo story for mono-repos isn't completely clear to me:

- one go.mod file for each "sub" package ?

- one go.mod file per mono-repo, at the root ?

- what happens if somebody tries to:

```

import (

"gonum.org/x/gonum/mat/v2"

"gonum.org/x/gonum/stat/v3"

)

```

assuming stat/v3 is compatible with mat/v2, is it legit? wouldn't this confuse vgo's storage somehow?

-s

Brendan Tracey

unread,

May 22, 2018, 12:21:19 PM5/22/18

to Sebastien Binet, gonum-dev

On May 22, 2018, at 10:10 AM, Sebastien Binet <bi...@cern.ch> wrote:

On Tue, May 22, 2018 at 5:57 PM Brendan Tracey <tracey....@gmail.com> wrote:
Thanks Dan for pinging on the issue.

I haven't been on top of this as I should be, so I'd like to check I understand.

- As of right now, it seems like we need to break our current system, instead of "gonum.org/v1/gonum", we would likely just have "gonum.org/gonum".
I think I'd rather have it `gonum.org/x/gonum` and `gonum.org/x/plot` (to more easily organize handlers and end-points on the gonum.org web site end).

I agree with the principle, though I don’t know about “x”, as in go land that is “experimental”. We could have “std” or “c” (for code) or something.

but, yeah, if nobody changes the current behaviour of `vgo` (or go-1.11), dropping `gonum.org/v1/xxx` would be required.

- We currently don't have any versions at all. While we would like to be at version 1, I don't think we're there yet (though we're getting closer). These changes do imply we should try and have some minor version tags, 0.1, 0.2 etc. to at least work with vgo somewhat
yes.
but we should tag them as v0.1.0, v0.2.0, etc.. to better follow what vgo expects.
(for go-hep, I was using v0.1, v0.2, ... but this confused vgo. had to add an extra `.0`...)

- Let's say we do get a version 1, but in the future we hove to a version 2. This would look like "gonum.org/gonum/v2", with imports like "gonum.org/gonum/v2/mat" . It would be represented on git with a version tag, but the code would still live in "github.com/gonum/gonum"?
yes.

- Let's say we eventually make the root `gonum` a package for some reason. How would a user use the v2 version of this? It seems like the import path would be simply "gonum.org/gonum/v2", so does that mean all import statements have to be written as `import gonum "gonum.org/gonum/v2" `? That seems really ugly.
I believe one would just have to write:
```
import (
"gonum.org/gonum/v2"
"gonum.org/gonum/mat/v2"
)
```

but I am not completely sure about the `mat` import.
also, the vgo story for mono-repos isn't completely clear to me:
- one go.mod file for each "sub" package ?
- one go.mod file per mono-repo, at the root ?
- what happens if somebody tries to:
```
import (
"gonum.org/x/gonum/mat/v2"
"gonum.org/x/gonum/stat/v3"
)
```
assuming stat/v3 is compatible with mat/v2, is it legit? wouldn't this confuse vgo's storage somehow?

I believe you are misinterpreting how the import path should look. From the proposal document

"The packages imported as my/thing/sub/pkg, my/thing/v2/sub/pkg, and my/thing/v3/sub/pkg come from major versions v1, v2, and v3 of the module my/thing, but the build treats them simply as three different packages.”

So based on my reading it would be “gonum.org/c/gonum/v2/mat” and “gonum.org/c/gonum/v3/stat”

My confusion about the bullet point is wrong, as I misread the Go spec. The Go spec says the default identifier is what is listed in the package clause in the imported package, and has nothing to do with the identifier used in the import clause. So, “mat” would still have a default import of “mat” no matter what you tack on to the end.

Sebastien Binet

unread,

May 22, 2018, 12:52:49 PM5/22/18

to Brendan Tracey, gonum-dev

On Tue, May 22, 2018, 18:21 Brendan Tracey <tracey....@gmail.com> wrote:

On May 22, 2018, at 10:10 AM, Sebastien Binet <bi...@cern.ch> wrote:

On Tue, May 22, 2018 at 5:57 PM Brendan Tracey <tracey....@gmail.com> wrote:
Thanks Dan for pinging on the issue.

I haven't been on top of this as I should be, so I'd like to check I understand.

- As of right now, it seems like we need to break our current system, instead of "gonum.org/v1/gonum", we would likely just have "gonum.org/gonum".
I think I'd rather have it `gonum.org/x/gonum` and `gonum.org/x/plot` (to more easily organize handlers and end-points on the gonum.org web site end).

I agree with the principle, though I don’t know about “x”, as in go land that is “experimental”. We could have “std” or “c” (for code) or something.

No, 'x' stands for 'extra'.

'exp' stands for 'experimental'.

That's why we import 'golang.org/x/exp/rand' instead of 'golang.org/x/rand' :)

You're probably right.

But we can't really test it because of this 'v1' issue :/

-s

My confusion about the bullet point is wrong, as I misread the Go spec. The Go spec says the default identifier is what is listed in the package clause in the imported package, and has nothing to do with the identifier used in the import clause. So, “mat” would still have a default import of “mat” no matter what you tack on to the end.

--
You received this message because you are subscribed to the Google Groups "gonum-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gonum-dev+...@googlegroups.com.

Sebastien Binet

unread,

May 25, 2018, 8:50:49 AM5/25/18

to Dan Kortschak, gonum-dev

On Fri, May 11, 2018 at 2:39 PM Dan Kortschak <dan.ko...@adelaide.edu.au> wrote:

The first one is not blocking for us, it's blocking for golang/go if
they care, but gonum/internal and all the other old packages are
deprecated and won't be changed.

The second one is troubling. I think pinging is in order. There is no
justification for forcing us to change this.

ok, now that golang/go#23960 has been fixed, I tried to vgo-ize `gonum.org/v1/gonum/floats`.

unfortunately:

$> go get -u golang.org/x/vgo

$> cd gonum.org/v1/gonum/floats

$> /bin/rm -f go.mod; touch go.mod

$> vgo install -v .

vgo: resolving import "gonum.org/v1/gonum/internal/asm/f64"

vgo: finding gonum.org/v1/gonum (latest)

vgo: adding gonum.org/v1/gonum v0.0.0-20180525030818-2a09e81714c8

floats.go:13:2: use of internal package not allowed

so... golang/go#23970 *is* a blocker.

unless we go with a big module for the whole github.com/gonum/gonum mono-repo: