Forking/transfering a repository and import paths

[Sotirios Mantziaris]

Hi,

i want to move a repo from my github account to another one. Goal is to have a new import path for the new forked repository.

There are 2 ways of achieving the move:

• Forking
• Transfer repository

Is it possible to fork a repo and change the import path of the repository?

If the transfer option is chosen we just have to change all imports in the code, which severs the ties for the originating project.

Is it possible to have:

• both repos
• every repo with it's own import path
• code exchange between them

What are the options?

[Wagner Riffel]

Go has nothing to do with github, you can have any import path and how many repos you like as long as it exists on your file system inside $GOPATH/src

-wgr

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Robert Engels]

I think he is pointing out the problem I’ve asked about many times that using the specified import path so ‘go get’ works is a problem. If I want to move my repos to another account all referencing code breaks. 

The import paths need a logical reference. 

Think Java and package names. The package name remains constant, where it is located/retrieved from changes all the time (classpath)

I am assuming I’ve always misunderstood this, but nothing that was ever stated cleared it up, so I just went with, well I guess I don’t understand and move along...

[Sotirios Mantziaris]

exactly that robert. :)

--

Regards,

S. Mantziaris

[Sebastien Binet]

On Wed, Dec 12, 2018 at 4:46 PM Sotirios Mantziaris <smant...@gmail.com> wrote:

exactly that robert. :)

On Wed, Dec 12, 2018 at 5:43 PM Robert Engels <ren...@ix.netcom.com> wrote:

I think he is pointing out the problem I’ve asked about many times that using the specified import path so ‘go get’ works is a problem. If I want to move my repos to another account all referencing code breaks. 

The import paths need a logical reference. 

Think Java and package names. The package name remains constant, where it is located/retrieved from changes all the time (classpath)

I am assuming I’ve always misunderstood this, but nothing that was ever stated cleared it up, so I just went with, well I guess I don’t understand and move along...

On Dec 12, 2018, at 9:36 AM, Wagner Riffel <wgrr...@gmail.com> wrote:

Go has nothing to do with github, you can have any import path and how many repos you like as long as it exists on your file system inside $GOPATH/src

-wgr

On Wed, Dec 12, 2018, 10:13 AM Sotirios Mantziaris <smant...@gmail.com> wrote:

Hi,

i want to move a repo from my github account to another one. Goal is to have a new import path for the new forked repository.

There are 2 ways of achieving the move:

• Forking
• Transfer repository

Is it possible to fork a repo and change the import path of the repository?

If the transfer option is chosen we just have to change all imports in the code, which severs the ties for the originating project.

Is it possible to have:

• both repos
• every repo with it's own import path
• code exchange between them

What are the options?

for compatibility sake, I usually fork instead of transfering the repository.

ie:

$> go get github.com/user/pkg

$> cd $GOPATH/src/github.com/user/pkg

$> git remote add new https://github.com/new-place/pkg

$> git checkout -b new-branch

$> find . -name "*.go" -type f -exec sed -i -e 's|github.com/user/pkg|github.com/new-place/pkg|g' {} \;

$> git add .

$> git commit -m "all: migration to github.com/new-place/pkg"

hth,

-s

[Robert Engels]

Yes, but you still need to maintain both repos or things will break. 

I am pretty sure that the correct solution is to decouple the package from its location. And a global Go registry can tell Go get where that package can currently be found. 

Then the package maintainer informs the global registry where the package is currently located. 

The the import statement uses the logical package name. 

For compatibility reasons the initial logical name could be the current import path. 

Seems straightforward to me. 

[Burak Serdar]

On Wed, Dec 12, 2018 at 9:08 AM Robert Engels <ren...@ix.netcom.com> wrote:
>
> Yes, but you still need to maintain both repos or things will break.
>
> I am pretty sure that the correct solution is to decouple the package from its location. And a global Go registry can tell Go get where that package can currently be found.

I agree with decoupling package from its location. I disagree with the
global registry idea. That's the same thing they did with maven
repositories in Java.

Package aliases, defined locally, is a much neater solution with no
global authority. Each project can define package aliases with
location information, and packages of the project can import those
aliases. This doesn't prevent "global" package registries, but
localized package information is much more manageable in my opinion.

[Sebastien Binet]

On Wed, Dec 12, 2018 at 5:08 PM Robert Engels <ren...@ix.netcom.com> wrote:

Yes, but you still need to maintain both repos or things will break. 

really? I am probably missing something.

usually when you migrate a repo it's to leave one on the side of the road.

by forking you sever all ties b/w the 2 repos and can apply breaking API changes.

I am pretty sure that the correct solution is to decouple the package from its location. And a global Go registry can tell Go get where that package can currently be found. 

Athens (or any GOPROXY-based + Go modules solution) might help there.

 - https://github.com/gomods/athens

--

[Robert Engels]

I agree and suggested that long ago....

Still, for the most part maven works pretty well and simplifies a lot of things. The global registry is not too different from godocs as a global repo of documentation...

[Louki Sumirniy]

I have done this many times. Some repositories are more of a pain than others to move. I was amused to learn after having done this with github.com/btcsuite/btcd, that the btcjson, btcec, btcutil and btclog repos all were separate but quite tightly bound both to each other and it took me about half a day to fix it. Part of my solution was copying those repos, removing the .git folder. I could have rather forked all of them, but those 4 in particular and the main btcd were very tangled and hard to separate.

My opinion is that there should be a simple way to refer to orthogonal repos, and sub-folders, with simple relative paths. So instead of "github.com/btcsuite/btcd/blockchain" I can just say "./blockchain" or instead of "github.com/btcsuite/btclog" I can say "../btclog" and the rest is inferred from the module spec and/or gopath location. 

I use Visual Studio Code (it has the best go toolchain integration I am aware of) and its search-in-repository functions are quite good, but it's easy to make a mistake and accidentally cast too wide a net, and the poor-man's-cut-down-for-no-reason regex searching doesn't help either.

It's not difficult, just tedious, and it would be nice if there was relative paths allowed for imports.

[Sotirios Mantziaris]

I was under the impression that go modules would do that kind of separation.
When creating a module you define the module path. If the project is hosted in github, bitbucket or anywhere else should not matter.
when calling go get github.com/user/application you refer just to the path where the repo is hosted. the import path should be the module path in the go.mod file.
Unfortunatelly i am not that experienced with modules. Is someone more experienced in go modules?

[Robert Engels]

I’ll admit I thought that modules would solve this problem as well, which is why I stopped worrying about it, but in the module examples I’ve reviewed I don’t see it addressing this, but I’m probably wrong. I think a clear example of how this is addressed with modules would be very beneficial. 

--

[Dave MacFarlane]

Relative import paths was a feature in old versions of Go and
explicitly removed *because* of Go modules:

https://github.com/golang/go/issues/26645
https://github.com/golang/go/issues/27224

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
- Dave

[Tamás Gulácsi]

With go modules you can import whatever/btcutil and force to be ../btcutil (see the replace directive).

[Sotirios Mantziaris]

would this one the problem of forking or transfer to another account? The go module path would still be the old one.
Like mentioned before it would be nice if the hosting path and the module path could be different e.g. hosting path github.com/mantzas/test module path "test".

On Thu, Dec 13, 2018 at 12:03 AM Tamás Gulácsi <tgula...@gmail.com> wrote:

With go modules you can import whatever/btcutil and force to be ../btcutil (see the replace directive).

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/E_TK2n9E-to/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Regards,

S. Mantziaris

[Ian Lance Taylor]

On Wed, Dec 12, 2018 at 8:08 AM Robert Engels <ren...@ix.netcom.com> wrote:
>
> I am pretty sure that the correct solution is to decouple the package from its location. And a global Go registry can tell Go get where that package can currently be found.

You don't need a global registry. Or, rather, the global registry can
be DNS. You can set up your own trivial redirector, using a meta tag,
as discussed at https://golang.org/cmd/go/#hdr-Remote_import_paths .
Or you can use an existing general purpose redirector such as
gopkg.in. Either way you can enforce this with an import comment as
discussed at https://golang.org/cmd/go/#hdr-Import_path_checking .

Ian

[robert engels]

It was called “sneaker-net” in my day… what a PITA

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[robert engels]

The modules Wiki https://github.com/golang/go/wiki/Modules is 68 pages long… not sure how such a simple need, solved many times before, ends up needing 68 pages to describe its usage. (I realize that a lot of the text is the FAQ, but still, if you need that long a FAQ, there are probably too many questions…)

[Sotirios Mantziaris]

Hi, if i understood correctly this solves the location of the repository path but the repo path will still point to the old repo.
It would be nice if the repo location could be anywhere and the import path anything e.g. hosted on any machine with the root package name be something different  (location agnostic).

The mod file or the project could have simple module name e.g. module patron 

and in the project that imports it: require patron v0.7.4      github.com/mantzas/patron.

When you `go get` a project for import you provide the location and `go get` could read the module name from the mod file and import it.

This way you could solve the fork problem by just `go get` from another location. forking would work.

--

Regards,

S. Mantziaris

[Sotirios Mantziaris]

If i understand it correctly the proposed solutions does not solve the problem of forking repos.
The ideal solution would be to have a import path agnostic of the repo location so that a fork could be up and downstream compatible.

[Nathan Fisher]

TLDR

> Is it possible to fork a repo and change the import path of the repository?

Not in one step with the github “click to fork”. You need to do one of the following:

1. Create a new empty remote repo, remap imports, and push to the new remote.
2. Fork, update imports, push.
3. Assuming vanity URL is used, fork and then update the vanity urls meta tag.

> Is it possible to have:
> both repos

yes but I would try to deprecate one as “archived/historical”.

> every repo with it's own import path

yes, it does by nature of being different urls

> code exchange between them
yes, but I’d avoid if possible. You’ll need to remap imports to match the repo so it’s not a simple push to both remotes.

—

You can use gofmt to remap the import paths. If you’re aim is to “sync” repos (I’m not clear why you’d want this aside from a clearance process for making corp stuff public) then you could encode the commands in a script or make target and trigger them as part of a “publishing process”. Relative import paths were available at one point but they are now deprecated/unavailable. In theory it allowed ignorance of the SCMs root at the sacrifice of not being go gettable and probably a number of other issues which resulted in its removal.

Godoc isn’t essential infrastructure in my opinion, you can serve docs locally in a pinch and I often do this while working offline in planes, trains, and automobiles. Introducing a registry however adds a failure domain/security concern that can break or be compromised. If githubs down and your repo is on github it’s the same failure domain so you’re arguably no worse/better off if your deps are there. Committing vendor where possible minimises the failure domains further and generally yields faster CI builds as a side benefit IME.

Another option might be abusing vanity urls if the objective is to support ongoing development in both repos (eg returning different meta tags based on a work IP vs not or a local /etc/hosts override to a different vanity url host). I’m not terribly familiar with them but in theory it should allow the same import path in both repos but it would likely result in some confusing issues. I don’t think it’s a great idea but it might work for your scenario.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

- sent from my mobile

[Sotirios Mantziaris]

I will try some of the suggestion in this topic to see what works out.

I will open a feature request in the golang repo for forking support. 

[Nikolay Bryskin]

Hello, are there any updates on this issue? I've faced this problem too when I tried to make my own fork of a large go library with a lot of self-imports inside - do I actually need to rename all these imports inside for it even to compile?

[Sean Liao]

If you intend to have a permanent fork distinct from the original, then yes, you do need to rename everything to match its new module identity.

- sean

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3b42f072-ae99-4f67-bec4-a71d3683e55dn%40googlegroups.com.