Is there a breakage in the latest weekly, or I have missed something?
The relative import worked in the previously weekly, but is broken
again in RC1.
can't load package: /home/robert/Code/go/src/pkg/javascriptcore/
base.go:7:8: local import "./strings" in non-local package
that doesn't appear to be correct. i have a counter-example here that
compiles at tip just fine. trimmed down to basics, my usage is:
$ mkdir test test/test1 test/test2
$ cd test
$ cat > test1/test1.go
package test1
type T int
$ cat > test2/test2.go
package test2
import "./../test1"
type T2 test1.T
$ cat > test.go
package main
import (
"./test2"
"./test1"
"fmt"
)
func main() { var v test2.T2; var v1 test1.T; fmt.Printf("%T %T\n", v, v1) }
$ go build .
$ ./test
test2.T2 test1.T
$
This is precisely what has been disabled.
If I understood the rationale behind the change, packages imported
using relative paths were being re-built every time, instead of using
the installed package. Relative paths have the problem that you need
to know what they are relative to. They are less portable. So, to
avoid that people use them, they were disabled. However, they still
work when used by files that you build outside of GOPATH.
I'm still trying to figure out if there is any way to organize my code
so that I can distribute a snapshot of the repository and it can be
built by the go tool without having to modify any source file.
--
- yiyus || JGL .
> Finally, I'm consistently surprised by people arguing that relative
> important paths are a problem because you need to know what they are
> relative to. If I'm inside a package 'a', then import "./b" should be
> equivalent to "a/b". It is not that hard, and programmers handle this
> every day. Frankly, if this doesn't work it will be a huge pain.
It's a bit more complicated in Go for a couple of reasons. The compiler
records the path to all imported packages, and the linker uses that
information when linking. That means that the linker has to be able to
find the import, even though the linker may be running in a different
directory. More seriously, because different packages are permitted to
have the same name in the package statement at the start of the file,
the compiler/linker distinguish different packages with the same name
based on the import path. That is, if one Go file imports "./b" and
another imports "a/b", those are considered to be two different packages
that happen to have the same name.
Ian
> Thanks for the reply, but I'm afraid I still don't understand the
> difficulty. If I'm in a directory (say a/b), and I change directory
> (say ./c), the OS says normalizes the path. My new directory is a/b/
> c, not ./c.
Normalizing the path can be complex on Windows, when using an
automounter, or in the presence of symlinks when the symlink is the
preferred path. Turning a relative path into an absolute path is rarely
a good idea when communicating between programs that may be separated by
time, such as, in this case, the compiler and the linker.
Ian
Maybe I'm missing something, but I don't think it is so complex.
The path can perfectly be normalized by the go tool. The relative path
doesn't need to be turned into an absolute path, only to a full import
path. We already have the import path of the package, which is passed
as the argument to -p, we only need to pass the same argument to -D.
I don't know about gcc, but the gc compiler and linker would not need
any change, just get the right arguments, which we already know well.
I think this patch should be enough (inlined, just 4 lines). I did not
do too many tests, and only in linux, but I don't think it should get
much more complex:
diff -r 160ec5506cb7 src/cmd/go/pkg.go
--- a/src/cmd/go/pkg.go Sat Mar 17 22:19:57 2012 -0700
+++ b/src/cmd/go/pkg.go Mon Mar 19 20:04:49 2012 +0100
@@ -291,7 +291,7 @@
// The localPrefix is the path we interpret ./ imports relative to.
// Synthesized main packages sometimes override this.
- p.localPrefix = dirToImportPath(p.Dir)
+ p.localPrefix = p.ImportPath
if err != nil {
p.Incomplete = true
@@ -351,6 +351,9 @@
if path == "C" {
continue
}
+ if !p.local && build.IsLocalImport(path) {
+ path = pathpkg.Join(p.ImportPath, path)
+ }
p1 := loadImport(path, p.Dir, stk, p.build.ImportPos[path])
if p1.local {
if !p.local && p.Error == nil {
Please don't take my analogy with directories so literally! If I'm
compiling package "a", than import "a/b" and import "./b" should be
equivalent. The relative import should be joined with the package
currently being compiled.
If relative package imports are not supported, it will be a barrier to
forking code.
Yes, it might be non-obvious, but absolute import path are more portable.
--
Aram Hăvărneanu
There is an ambiguity when compiling c, if c imports both a and a/b.
> 2) I want to be able to fork a repository.
>
> 3) I want to be able to push to or pull from a forked repository.
Fork at the vcs level, by creating a new branch. Iff you decide you
want to diverge from upstream your package should get a new
import path anyway and packages that decide to use your version
need to be updated as well.
It probably takes less time to write a sed one-liner that changes the
import path than the time it took me to write this sentence.
> Disallowing relative imports break both #2 and #3 (well, you could
> cherry-pick, but we've complicated what should be a simple process).
Relative import paths break #1.
There's also an indirection mechanism that allows to create short
import paths, I'm not sure if it's documented or not.
--
Aram Hăvărneanu
Hmmm, I feel like a [citation needed] stamp is needed to support you
assertion that writing sed commands to update package imports is
easier. I have a situation right now where using absolute imports
will cause me head-aches. Nothing that can't be fixed, but
difficulties nevertheless.
1) I'll emphasize this again. I want import "a/b" and import "./b" to
be equivalent when compiling package "a". There is no ambiguity here.
2) I want to be able to fork a repository.
3) I want to be able to push to or pull from a forked repository.
Disallowing relative imports break both #2 and #3 (well, you could
cherry-pick, but we've complicated what should be a simple process).
I really appreciate gofmt, but does the community really feel the need
to dictate how users organize their projects? Especially since
authors are free to ignore the feature?
It is a 4 line patch. Once. On the other side is forcing every
forked library to require an addition edit to correct import paths,
and complicating every push or pull.
You are doing nothing wrong, but that does not mean relative imports
are wrong. They can be useful.
For example, in the github.com/cznic/dns repository there are several
packages. Some of these packages import other packages from the same
repository. If those import paths were relative I could fork that
repository and it would be ready for the go tool, without any
modification. I could also make a tarball and build it with the go
tool from anywhere inside or outside of GOPATH, without touching
anything. You cannot do that now, you have to edit the code.
I think the original problem is that a repository without subpackes is
not location dependent, but as soon as you refactor some code in a
subpackage your location is fixed because of the import path. There
may be some better solution to this problem that using relative paths,
but I don't think sed is that solution.
Eg:
You have app1 and pkg1 in repo/, you can install app1 and pkg1 from
GOPATH and import repo/pkg1 from any other package. You can also go
build app1 and pkg1 from any arbitrary location, or go run app1.
At some moment you decide to refactor some common code from app1 and
pkg1 into pkg2. You would prefer to keep your code usable from any
location. The obvious solution is to import ../pkg2 instead of
repo/pkg2. But, if you do that, then pkg1 can no longer be built in
GOPATH. If, instead, you use an absolute path, you have lost all the
freedom you had to build or run from any location. How do you handle
this situation?
+1 for relative import path. The compiler should do normalizing relative path, and it's not too headache.
If the current compiling pkg is x, then the import path "./y" should normalized as "x/y" (by the compiler).
Without relative
imports, forking a repository that has multiple packages requires
extra work to fix the import paths. Pushing or pulling between
repositories also now involve extra work.
I was replying to the question of what relative paths are good for.
Indeed, writing a fix is not a big deal, but applying it can be a
pain. How would you distribute your code in a tarball if it has
subpackages? Either you say in your README or somewhere else that the
source has to be extracted to a specific location inside GOPATH or you
need to apply a fix before building with the go tool.
It is fine for me if relative paths are not allowed, I can live with
it, but I find unfortunate creating subpackages introduce this
limitation. I will have to consider if I prefer to fix my code to a
location or live with some duplicated code.
Go packages should be complete units (libraries) and not simple
namespace things, so If two packages should be used together they
should be the same.
> A lot of the original criticism came because people where using
> relative paths to import packages that lived outside of their GOPATH.
> I cannot comment on that use case, as it is not something I ever used.
>
> For my 2 cents (which does not seem to be amounting to much with the
> community), relative paths are useful for acessing sub-packages. They
> are perhaps useful for accessing the parent package. Without relative
> imports, forking a repository that has multiple packages requires
> extra work to fix the import paths. Pushing or pulling between
> repositories also now involve extra work.
GOPATH:
src
github.com
user1
pkg1 --> import ( ../pkg2 )
pkg2 --> import ( ../../user2/pkg1
user2
pkg1
--
André Moraes
http://andredevchannel.blogspot.com/
There is an exception to this, and that is when some common code is
factored out. For example, the package
code.google.com/p/go.crypto/openpgp/errors is meant to be used by the
other packages in that directory. I would not consider such a package
a complete unit, but a subpackage, part of a bigger project.
If this package could be imported like "../errors" then I could build
the packages inside openpgp from anywhere. I can accept that I cannot
do this, but I'd like to hear a good reason of why this is worse than
having to edit the original code.
I find strange having to edit source files just to build a project
from a different location (I find even stranger that most people here
seem to be fine with it). Relative paths could easily fix this. If
relative paths are considered harmful then, fair enough, that's the
wrong approach. Maybe there is other, more elegant solution and maybe
not. But can we agree that, if possible, not having to attach the code
to a fixed location is a desirable property?
> GOPATH:
> src
> github.com
> user1
> pkg1 --> import ( ../pkg2 )
> pkg2 --> import ( ../../user2/pkg1
> user2
> pkg1
What's your point? If this is the only concern, the go tool could
protect against relative paths escaping the current repository.
If the feeling against relative paths is so strong, why aren't they
completely removed? I'm sure we can find other solutions to compile
subpackages outside of GOPATH.
It is complete if we consider that it's purpose it to define errors
messages, it's a small purpuse but it's complete. But I agree that
"complete unit" dependes of how somebody see the purpose of the
package.
> If this package could be imported like "../errors" then I could build
> the packages inside openpgp from anywhere. I can accept that I cannot
> do this, but I'd like to hear a good reason of why this is worse than
> having to edit the original code.
>
> I find strange having to edit source files just to build a project
> from a different location (I find even stranger that most people here
> seem to be fine with it). Relative paths could easily fix this. If
> relative paths are considered harmful then, fair enough, that's the
> wrong approach. Maybe there is other, more elegant solution and maybe
> not. But can we agree that, if possible, not having to attach the code
> to a fixed location is a desirable property?
I think the NodeJS loading and search is a example of how things can
become complex with relative paths, even if you know how nodejs load
libraries you still waste time thinking about where modules are coming
from.
With Go, you just need to know the value of GOPATH.
You can have multiple GOPATH's and switching from one version of a
library to another is just a matter of GOPATH=/my/custom/path. Also in
POSIX systems ls can help with that.
> If the feeling against relative paths is so strong, why aren't they
> completely removed? I'm sure we can find other solutions to compile
> subpackages outside of GOPATH.
How?
For example, with a new build option:
$ cd $HOME/project1 # not in GOPATH
$ go build -D 'repo.com/user/project1 -> /home/user/project1' ./...
(the right part of -> may default to $PWD if omitted and, of course,
this option should not be allowed for the install command).
Only the go tool would have to be modified to make this work and,
taking into account this would allow to remove all the support for
relative paths (and its currently non-existing documentation), it may
be a simplification.
I'm sure there are other ways I have not thought about, probably better.
> Can you provided a concrete harm to your code that would come from allowing this?
I can't think o any harm that "./my sub package" will have, but
allowing "./" and not "./../" don't look right (personal opinion).
If "../" are allowed then the import path will become very complex to
understand simply by looking at it, and in this case, i prefer
absolute paths over relative ones. (another personal opinion).
Now, a possible solution to your problem:
Import the complete path of your project (JavaScriptCore bindings if I
am not wrong).
In bitbucket.org/rj/javascriptcore-go/base.go
import "bitbucket.org/rj/javascriptcore-go/strings"
Then if somebody want's to contribute code with you they need to
install your project under they GOPATH folder and change it.
If somebody want's to fork your code and do some changes and never
submit that again, they can:
Just use your path and never push their changes
Just use your path and add their remote repository as a valid URL in
.hg/hgrc and push their changes.
I don't see any problem here.
import (
"../../.././pk1"
)
func main() { pkg1.SayHello() }
Can you guess, just by reading this what is pkg1?
import (
"example.com/repo.git/terminal/pkg1"
)
func main() { pkg1.SayHello() }
>
>> You can have multiple GOPATH's and switching from one version of a
>> library to another is just a matter of GOPATH=/my/custom/path. Also in
>> POSIX systems ls can help with that.
>
> That does not help when working with forked packages.
Yes it does, I forked a GLFW package made my own changes, sent pull
request and didn't had to change the examples packages of the original
author.
That allowed me to write "go test ./..." to test if my changes broke something.
If the language allows it then people could write and this will be
valid and everybody will need to live with the possibility of finding
such code someday.
>
> I think relative import is important, even if only for subpackages (ie. not
> alowing ..), but if the language tries to control all wrong usage of things,
Allowing "./" isn't too bad and it's probably the only valid use of
relative paths that I can see.
But considering that GOPATH can change from one env to another by just
calling GOPATH="/some/path/here" then the abscense of relative paths
don't make things too complicated.
First, that is not a concrete example, because you've stripped away
all context. Certainly while editing a package, I should know the
name of the package. If I'm editing "example.com/repo.git/terminal',
I would have no trouble seeing that the import is error. If the
import was correct, I would have no trouble determining the actual
target. Anyone who can use the cd command on the command-line should
be able to handle a relative import.
Second, you've picked a pathological case. You might as well be
worried that I will cryptographically encrypt all the function names
in my package. (Don't tempt me. I've been meaning to find a use for
Go's native unicode identifiers...)
> Yes it does, I forked a GLFW package made my own changes, sent pull
> request and didn't had to change the examples packages of the original
> author.
Did you duplicate all of your dependencies? No? Fortunately you can
add multiple directories in your GOPATH. Oh wait, now I've also got a
copy of the original repository in the GOPATH, so I'll have a
conflict. Not a problem, I'll create a third directory, move my local
copy of the original repository there... I'm not saying you can not
make it work, I'm asking why this is suppose to be easier than using
relative import paths.
Complexity happens. It is dangerous to throw away a tool just because
you have not *personally* had a use for it yet.
I am not throwing anything out because I don't have a use.
I have forked repos which I make changes,
I have local code that is private right now but I will publish it later,
I have a use for "./pkg" and for "../otherPackage"
I just think that the benefits of relative packages are reduced by
it's problems (in the long run).
And pathological cases can become common cases.
"Almost never happens" == "can happens sometimes"
I am not even on the core language team, I just exposed a different
point of view that should be considered, my opinion goes to the toilet
if Go-lang team et. all decide that it's wrong, I will not fork the
compiler just because I don't like "../../" :)
You missed Ian's comment.
> therefore appears that the decision is what it is. I'll stop using
> sub-packages as a tool to organize my code.
--
André Moraes
http://andredevchannel.blogspot.com/
I don't follow.If I fork, for example, github.com/someuser/repos and then my repos address becomes github.com/divoxx/repos. If i want it to be able to compile and run the tests in my machine I have to update all refs to the old repos.Then if I send a pull request, I need to make sure those changes are not included in the changeset, so I have to revert back to the old ref or do some clever branching/rebasing to not incude it.
If I fork, for example, github.com/someuser/repos and then my repos address becomes github.com/divoxx/repos. If i want it to be able to compile and run the tests in my machine I have to update all refs to the old repos.
You don't need to update the refs. Clone the code where the go tool expects to find it. Here's an example assuming that $GOPATH is a single directory.$ mkdir -p $GOPATH/src/github.com/someuser$ cd $GOPATH/src/github.com/someuser$ git clone g...@github.com:github.com/divoxx/repos.git$ cd repos$ go install... edit .. test ... edit ...$ git push origin master
Ok, that would work. It just feel weird that when cloning the repos you'd have to know the file structure you'd have to clone.
If you clone a repository, it seems likely that you want to edit the files. You need to know the structure to edit the files.
Supposing I have a fork with a feature that haven't been merged back yet but people still want to use my fork.
They can use git clone as in my previous example.
2012/3/22 Rodrigo Kochenburger <>
--
~f
Well and when someone forks github.com/divoxx/repos.git it will not build.
It wil build if the repo is cloned to the location expected by the go tool. Try the suggestions from hash and me and see for yourself.
No, You don't.
Solution one, use symlinks
go get github.com/divoxx/repos
mkdir -p $GOPATH/src/github.com/someuser
cd $GOPATH/src/github.com/someuser
ln -s $GOPATH/src/github.com/divoxx/repos
This way you use your forked code with the same path of the upstream code.
Solution two, use git remote
go get github.com/divoxx/repos
go get github.com/someuser/repos
cd $GOPATH/src/github.com/someuser/repos
git remote add myfork ssh://g...@github.com/divoxx/repos
git pull myfork master
This way you use your forked code with the same path of the upstream code.
Both ways work perfectly fine for me when working on somebody else code.
If, you need to use both your code and the remote code, you can set
two different GOPATH variables and switch between them (GOPATH=.... go
<name your command here>)
Or just use two different import paths, this one makes a lot of sense,
since your forked code will not need to be compatible with the
upstream, they should be loaded from different paths.
>
> Then if I send a pull request, I need to make sure those changes are not
> included in the changeset, so I have to revert back to the old ref or do
> some clever branching/rebasing to not incude it.
People seem to confuse the intent. The goal is not to allow users to
circumvent the GOPATH. I simply want import "./subpkg" to be
equivalent to import "github.com/user1/pkg1/subpkg" when I'm compiling
"github.com/user1/pkg1".
How, I really can't see the point.
Git have branches, and with "git checkout -b yourbranch" you switch
your working copy.
The "go" tool sees only your working copy. You can have as may
branches/remote refs as you want and just switch them with "git
checkout".
>
> The suggested workaround of "mimic the original file path" isn't always
> workable; in this case, I want to have the original path living there (so I
> can refer to it as I experiment on my forked copy).
If you are only making some "experiments" just change your GOPATH and
point to the experimental tree. It's even better this way since you
would have more control over what is in it.
> I guess the takeaway from this is "don't use subpackages if you want to
> allow easy forking"... which is awfully disappointing.
No, this isn't the takeway. The takeway is: if forking is so important
that you can't change a env-var, then use makefiles and gc toolchain,
instead of "go tool". The compiler/linker allows you to do that.
I'd be happy to change GOPATH to accomplish this but I don't quite see
how that will work (without also editing the files for change the
imports). Perhaps you could explain a bit further?
GOPATH=/home/user/Stable
go get github.com/user/lib1
Now, use the upstream version of the library in your production app.
GOPATH=/home/user/Testing
go get github.com/user/lib1
cd $GOPATH/src/github.com/user/lib1
git add remote myrepo g...@github.com/otheruser/lib1
git pull myrepo master
Now use your version of the library in your experimental application
In both cases the import path in your .go files is the same. No need
to change anything in the code.
But when the go tool is running, it will load the packages from
different locations, since GOPATH would be different.
I'm not proposing that the Go import style needs to change, but github
is rather popular, and offering a simple cookbook approach to making
its default workflow mesh more easily would seem to be a good thing
for would-be Go experimenters.
Would it be possible to have a way to tell the go tool to remap an
import within a given tree (perhaps via the .go file approach that
someone else proposed)... e.g., for the examples below
go get github.com/user/lib1
go get github.com/otheruser/lib1
What if there was a config file of the form
github.com/otheruser/lib1/.go
IMPORTMAP "github.com/user/lib1" = "/path/to/local/otheruser/lib1"
?
I infer that this sort of "local config file" approach has been
considered and rejected, but it's not clear to me why; it seems like a
much simpler and cleaner solution to this sort of issue. (Again,
apologies for coming in late to this thread.)
2012/4/17 André Moraes <and...@gmail.com>:
But I don't want to edit them "forever" -- the whole issue is that
they have to be edited once (to get things to build locally), but when
I push back to github, I can either (1) leave them edited to refer to
my fork [in which case, pulling the change back to the original source
of the fork means they have to be edited back], or (2) change them
back to the original source of the fork [in which case, my fork will
need re-editing locally every time I sync]. Both of these options are
workable, but neither very appealing.
The important thing for the go tool is to: "Be able to build a Go
program without writing config files".
If the package owner really wants to allow easy forking, he could
write makefiles for that purpose or use another build tool.
It may break when somebody tries to "go get" your package.
...which would be neatly solved if we had relative import paths,
right? <ducking>
> If the package owner really wants to allow easy forking, he could
> write makefiles for that purpose or use another build tool.
But given typical github workflows, that's equivalent to saying "If
you are interested in writing multi-contributor projects in Go using
Github, you can't use the go build tool without extra effort -- use
makefiles instead." Surely there's a friendlier approach.
On Tue, Apr 17, 2012 at 12:45 PM, Jan Mercl <0xj...@gmail.com> wrote:
> Are you talking about the case when you want to both publish a go get enabled fork and being able to
> make pull requsts from that repo towards the forked-from tepo? That's not possible in the case of "intra"
> repo imports, fortunately that doesn't sound like a common scenario to me.
It's *not* a common scenario currently... but then, it couldn't be,
since it's not currently possible with the combination of "go get" and
github fork/pull.
I assertion is that it would become a common scenario if it was a bit
less painful to combine the two.
Maybe is just me, but when I decide to work on a Go project that isn't mine,
I first "go get" the upstream package and later add the fork using git
remote add ....
If the person wan'ts to first "go get" the forked repo and later add
the upstream then it's a problem.
Robert Johnstone <r.w.jo...@gmail.com> writes:> Thanks for the reply, but I'm afraid I still don't understand the
> difficulty. If I'm in a directory (say a/b), and I change directory
> (say ./c), the OS says normalizes the path. My new directory is a/b/
> c, not ./c.Normalizing the path can be complex on Windows, when using an
automounter,
or in the presence of symlinks when the symlink is the
preferred path.
Turning a relative path into an absolute path is rarely
a good idea when communicating between programs that may be separated by
time, such as, in this case, the compiler and the linker.
Ian