Vanity import path vs canonical import path

[meta keule]

Hi,

can somebody help me with a definition of vanity import path vs canonical import path?

While I found lots of discussions around them, I could not find a clear definition.

Thank you,

metakeule

[Ian Lance Taylor]

On Tue, May 27, 2014 at 2:27 PM, meta keule <marcre...@googlemail.com> wrote:
>
> can somebody help me with a definition of vanity import path vs canonical
> import path?
>
> While I found lots of discussions around them, I could not find a clear
> definition.

Most import paths refer directly to some code hosting repository, as
in "code.google.com/p/go.tools/go/types". A vanity import path would
be something like "golang.org/p/go.tools/go/types". golang.org is not
actually a code hosting site, but the idea is that it would host some
sort of redirector that would point to the real code hosting site.
That would permit, for example, moving go.tools from code.google.com
to github.com without breaking programs that use it.

Ian

[meta keule]

From the documentation, I looks like import paths are just strings and go get is the only
tool looking for urls.

So, if I get you right, vanity import path is just a fancy way to say "an import
path that makes use of the fact that go get supports redirects"?

I find it confusing, because "vanity import path" suggests that it is a property of the import path.
Now it looks more like a property of go get and a server configuration or do I miss something?

[Ian Lance Taylor]

On Tue, May 27, 2014 at 3:04 PM, meta keule <marcre...@googlemail.com> wrote:
>
> From the documentation, I looks like import paths are just strings and go
> get is the only
> tool looking for urls.
>
> So, if I get you right, vanity import path is just a fancy way to say "an
> import
> path that makes use of the fact that go get supports redirects"?
>
> I find it confusing, because "vanity import path" suggests that it is a
> property of the import path.
> Now it looks more like a property of go get and a server configuration or do
> I miss something?

You are correct.

We call it a vanity path because it means I could tell everybody to
import their package from

ianlancetaylor.com/awesome

without having to actually host the source code control myself.

Ian

[meta keule]

hmm, I see the benefit of being able to switch the hosting service beneath.
But from a users perspective it looks like some (private) domain might sooner
get unavailable than a rotten repo on github.

Maybe a central vanity.golang.org redirector could solve those problems.
In addition to have configurable redirects, it also could clone the repos
regularily and serve them on its own when the original repo has disappeared?

[meta keule]

Ok with the brand new go 1.4 release I try to use the

Canonical import path feature as in

   package loop // import "gopkg.in/metakeule/loop.v1"

Now the problem is: If I use a vanity import path as canonical import path
(in the example I want the user to force the gopkg.in version variant)
that is in fact hosted elsewhere (here on github.com/metakeule/loop)
I can't develop anymore inside my $GOPATH/src/github.com/metakeule/loop directory, because
 'go build' and 'go install' complain and don't compile anymore.

However developing inside  $GOPATH/src/gopkg.in/metakeule/loop.v1 would make no sense
for obvious reasons.

Any ideas how to force go build and go install to ignore the canonical import for development?

Otherwise gopkg.in and canonical import paths could be a powerful combo.

But this problem would occur for anyone having a canonical import pointing to an vanity import
(and wasn't it the reason for canonical imports in the first place?).

[Karol Mieczysław Marcjan]

W dniu sobota, 13 grudnia 2014 23:55:34 UTC+1 użytkownik meta keule napisał:

However developing inside  $GOPATH/src/gopkg.in/metakeule/loop.v1 would make no sense
for obvious reasons.

For the sake of being explicit: What exactly is so bad about developing code in the
$GOPATH/src/gopkg.in/metakeule/loop.v1 directory?

-- Karol Marcjan

[meta keule]

Well the point of using git is to develop in one repository/working copy and have different tags and branches
and be able to switch them.

So working in $GOPATH/src/gopkg.in/metakeule/loop.v1
for a bugfix in v1 and then developing in $GOPATH/src/gopkg.in/metakeule/loop.v2
for a bugfix in v2 and so on gets tiresome since the working copies would get out of sync fast
and it is not easy to keep track, switch branches and tags etc.

[minux]

On Sat, Dec 13, 2014 at 6:41 PM, meta keule <marcre...@googlemail.com> wrote:

Well the point of using git is to develop in one repository/working copy and have different tags and branches
and be able to switch them.

So working in $GOPATH/src/gopkg.in/metakeule/loop.v1
for a bugfix in v1 and then developing in $GOPATH/src/gopkg.in/metakeule/loop.v2
for a bugfix in v2 and so on gets tiresome since the working copies would get out of sync fast
and it is not easy to keep track, switch branches and tags etc.

This is one more reason not to use versioning. (FTR, I'm against the versioning approach.

keep api compatible is a virtue)

Just develop in ../loop.v1 for v1 branch, and develop in ../loop.v2 for the v2 branch.

[meta keule]

The problem exists for everyone using vanity import paths.

Like mentioned by Ian Lance Taylor a vanity import path is a redirect
with the purpose of delivering the package under an other path than
the hosting service. Thats what gopkg.in is doing and it is working fine.

The canonical path then can enforce the usage of the vanity path.

But for the development it makes no sense to develop in a redirect path.

The gopkg.in only makes this visible and is for illustration.

It would be better IMHO, if 'go build' and 'go install' only would respect the
canonical path as an option, since they are only used by the developers of the package
and 'go get' should invoke 'go install' with the option that prevents installation via non canonical paths since
'go get' is used primary by users of the package (developers would clone a repository).

The versioning approach can make sure that the api is compatible while making it possible to break the api
when needed (semantic versioning). Canonical paths could be used to enforce the communication about
what compatible version is used and even prevent imports of  two different versions when necessary (initialization code that should be run only once) while allowing it when it doesn't matter.

This could be a proper way to express dependencies between libraries if the canonical paths would be implemented as proposed above.

[Robert Melton]

On Sun, Dec 14, 2014 at 5:10 AM, meta keule <marcre...@googlemail.com> wrote:
> But for the development it makes no sense to develop in a redirect path.

You haven't convinced me this is true in the least. The reason I use
gopkg is so I can have major version splits that are no longer
backwards compatible. Between public projects, gopkg is the "real"
location for all intents and purposes. You can develop a project in
"$GOPATH/src/gopkg.in/robertmeta/library.v1" (which pushes to
github.com/robertmeta/library on branch v1) and then you can
concurrently develop a separate (breaking) featureset in
"$GOPATH/src/gopkg.in/robertmeta/library.v2" (which pushes to
github.com/robertmeta/library on branch v2)... you imply this is
inheriently bad because... why exactly?

--
Robert Melton | http://robertmelton.com

[Manlio Perillo]

Il giorno sabato 13 dicembre 2014 23:55:34 UTC+1, meta keule ha scritto:

Ok with the brand new go 1.4 release I try to use the

Canonical import path feature as in

   package loop // import "gopkg.in/metakeule/loop.v1"

Now the problem is: If I use a vanity import path as canonical import path
(in the example I want the user to force the gopkg.in version variant)
that is in fact hosted elsewhere (here on github.com/metakeule/loop)
I can't develop anymore inside my $GOPATH/src/github.com/metakeule/loop directory, because
 'go build' and 'go install' complain and don't compile anymore.

However developing inside  $GOPATH/src/gopkg.in/metakeule/loop.v1 would make no sense
for obvious reasons.

Any ideas how to force go build and go install to ignore the canonical import for development?

Not tested, but I may try it in the near future:

1) Create a new personal src/metakeule directory.

2) Keep all your git projects here.

3) Create soft links to your vanity import path, e.g.

    ln -s src/metakeule/loop src/gopkg.bin/metakeule/loop.v1

You can also create a `go develop` tool that does this.

If you need multiple versions, you can clone the repository instead of symlink it.

> [...]

Regards  Manlio Perillo 

[meta keule]

In the case of gopkg.in some reasons why development in $GOPATH/src/gopkg.in/robertmeta/library.v1
is  suboptimal:

- inconsistency between unversioned and versioned package paths:
unversioned package paths have same path as hosting site, versioned not
which leads to the fact that in a team everbody developing on a package must
know that once a canonical path is introduced they have to change their gopath layout

- there is no way to clone from gopkg.in/robertmeta/library.v1 and commit to it. you have to clone from
github as repo committer and that into a different path than the github repo

- for N versions you end up with N working/developing copies

- to merge a patch to N versions all working copies must have the latest version, so just must go to N
working copies and pull/merge into them. Now with several developers, each of them must do this

- how does any of the developers know that a new version has been tagged? must go to the hosting site to find out
with only one repository there is just a regular pull and look at the tags

- if the canonical path changes for whatever reason (something better than gopkg.in appears) all developers have to reconfigure their working copies again.

[Karol Mieczysław Marcjan]

W dniu niedziela, 14 grudnia 2014 11:50:34 UTC+1 użytkownik Manlio Perillo napisał:

Not tested, but I may try it in the near future:

1) Create a new personal src/metakeule directory.

2) Keep all your git projects here.

3) Create soft links to your vanity import path, e.g.

    ln -s src/metakeule/loop src/gopkg.bin/metakeule/loop.v1

You can also create a `go develop` tool that does this.

If you need multiple versions, you can clone the repository instead of symlink it.

> [...]

Regards  Manlio Perillo 

go build/install do not follow symlinks, AFAIR. On the other hand, I haven't tried
them when they pointed to another location in the $GOPATH directory...

-- Karol Marcjan

[meta keule]

I might be wrong, but I think it was mentioned on this list that soft links are rejected by the
go commands. Maybe someone can confirm.

> Not tested, but I may try it in the near future:

[meta keule]

BTW, I made a ticket for this issue: https://github.com/golang/go/issues/9310

Am Dienstag, 27. Mai 2014 23:27:02 UTC+2 schrieb meta keule:

[Sebastien Binet]

alternatively, you could use go's build constraints to enable/disable
vanity import paths for during development:

$ cat pkg.go
package foo

func Foo() {}

$ cat release.go
// +build !dev

package foo // import "example.org/foo"

so, now 'go get github.com/user/foo' won't work, but 'go get
example.org/foo' will.
and during development, 'go build -tags=dev' and 'go install -tags=dev' will.

(you can put those into a Makefile if that's too long to type)

hth,
-s

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Robert Melton]

On Sun, Dec 14, 2014 at 6:00 AM, meta keule <marcre...@googlemail.com> wrote:
> - inconsistency between unversioned and versioned package paths:
> unversioned package paths have same path as hosting site, versioned not
> which leads to the fact that in a team everbody developing on a package must
> know that once a canonical path is introduced they have to change their
> gopath layout

Umm... the goals of the project have changed if you are switching to a
stable API and gopkg.in -- I think telling the team is an acceptable
cost, at the very least... your developers will probably want to know
such a thing.

> - there is no way to clone from gopkg.in/robertmeta/library.v1 and commit to
> it. you have to clone from
> github as repo committer and that into a different path than the github repo

Yes... there is, and it is a standard part of working on Go projects
on github. You use "go get" to grab the project, as per normal. Then
you add your fork of it to the remotes and you work in the original
projects directory ($GOPATH/src/gopkg.in/robertmeta/library.v1). So
you work, work, zug, zug, you push to your fork and put in a pull
request to the canonical repo & branch. The only additional step here
is having to track down the canonical repo, which is trivial.

> - for N versions you end up with N working/developing copies

This is a feature... it lets your move forward with breaking changes,
this is the entire point.

> - to merge a patch to N versions all working copies must have the latest
> version, so just must go to N
> working copies and pull/merge into them. Now with several developers, each
> of them must do this

What? If the patch is common, cherry pick between the branches...
this is already a part of (ANY) doing branched development. Why would
each developer have to do it... one developer does it and the rest
just pull his changes... I am so confused.

> - how does any of the developers know that a new version has been tagged?
> must go to the hosting site to find out
> with only one repository there is just a regular pull and look at the tags

Again, you act like the developers are in sealed rooms in some sort of
riddle. New versions of software have to be discovered, it isn't
magic... announcements, mailing lists, etc... how did you discover go
1.4 was out?

> - if the canonical path changes for whatever reason (something better than
> gopkg.in appears) all developers have to reconfigure their working copies
> again.

Something better than gopkg.in would be changing the vanity URL, not
the canonical one... and yes, developers would have to point to the
new location -- this isn't some crippling task that is beyond the
abilities of mere mortals -- this is exceptionally simple book
keeping.

[meta keule]

@Robert Melton

Sorry but I got the impression that you don't read my comments carefully.

>> - for N versions you end up with N working/developing copies

> This is a feature... it lets your move forward with breaking changes,
this is the entire point.

That is nonsense. With gopkg.in you always end up having different clones of a repo and yes that is a feature.
But this has nothing to do with developing inside different clones.
The latter does not make any sense: You are switching directories and manually pulling different times
for no reason (other than your crappy development structure).
There is a reason we have branches and tags in git. It allows us to stay inside the same directory and switch the
branch. And all the tools, editors, IDEs and whatnot that depends on the path still can be used.
That is a huge benefit you are throwing out of the window in your model for no reason.

[Robert Melton]

So, you already put forth that you have to have two directories (for
library.v1 and library.v2) so we are in agreement. These HAVE to
exist... but you have an issue with doing development in them, because
they must be pointed to a different remote name. You would rather do
development in $GOPATH/src/canonical/place branch v2... but, those
changes wouldn't be seen until you do another go get -u on the
gopkg.in version (because your code, as it should, references the
gopkg.in version). Have you really simplified your development any?
you didn't save any space (takes up more), you added a 3rd place to
manage... you avoided having to have a remote under them library.v1
and library.v2 ... at the cost of having to do a go get -u on every
single change... I don't like that tradeoff.

[meta keule]

Why do you think that I have to import every version that is supported
somewhere inside another package?

It is perfectly fine to just use v.10 and support v.1-v.9
There for I just need library.v10 inside my GOPATH
and the main repo and switch branches inside the main repo.

I don't need go get to develop. Just my VCS.

Where is the problem?

[meta keule]

I would propose to continue the discussion inside the ticket:
https://github.com/golang/go/issues/9310

Thank you.

Am Dienstag, 27. Mai 2014 23:27:02 UTC+2 schrieb meta keule:

[meta keule]

While this is  nice hack, that currently means to no longer
be able to use gofmt nor goimports since they both
do not support --tags.

In fact they both hang if the repo path does not
match the canonical path.

I have filed an issue: https://github.com/golang/go/issues/9314

[Manlio Perillo]

I just tried it now, and go build *do follow* symlinks.

[manlio@arch ~]$ go version

go version go1.4 linux/amd64

Regards   Manlio Perillo

[dhaw...@gmail.com]

Is there anyway to make canonical import path case insensitive?

package loop // import "github.com/testproject/testrepo"

Above package fails if saved under path "gopath/src/github.com/testProject/testRepo"

Thanks,

Dhawal

[Alex Bligh]

On 24 May 2016, at 19:12, dhaw...@gmail.com wrote:

> Is there anyway to make canonical import path case insensitive?
>
> package loop // import "github.com/testproject/testrepo"
>
> Above package fails if saved under path "gopath/src/github.com/testProject/testRepo"

Why would you want to save it under the wrong name?

--
Alex Bligh