different package name but same module name (example: forking a package)

[ben]

Hi,

Is it possible for two different packages to contain identically named modules?

For instance, if I fork a package Galaxy.jl, and then 

> Pkg.clone("github/fork/url/Galaxy.jl.git", GalaxyBlue)

(So that both the original package and my forked version live side-by-side in my .julia repository.) How do I then use GalaxyBlue? "using GalaxyBlue" and "require("GalaxyBlue")" both give me "ERROR: GalaxyBlue" not found.

I guess I am generally confused about package names v. module names and I was not able to find documentation on this.

Thanks!

Ben

[John Myles White]

I think the core insight is that a package has two components:

(1) A module called GalaxyBlue
(2) A file, in a searchable package directory like Pkg.dir(), that’s called something like GalaxyBlue/src/GalaxyBlue.jl.

For 2, that means you have a directory (usually a repo) containing the package code, a subdirectory called src and then a file called PACKAGENAME.jl, which will be automatically loaded. That file should contain the definition of your module.

I’m not sure, but I suspect that Pkg.clone with a second argument isn’t changing the content of that file.

— John

[ben]

Thanks, John. 

I am still a little confused:

(1) Does the main module of a package have to be named like the package?

(2) What is the priority of modules/packages when, eg, calling `import Galaxy` if there is both a package and a module named Galaxy (see example below)?

(3) What is the correct use of import/using/require? 

(4) What is the point of the (documented) syntax `Pkg.clone("github/fork/url/Galaxy.jl.git", "GalaxyBlue")`? Is it deprecated? Is it exactly equivalent to a git clone from terminal?

Here is an example which can happen in practice and which is very unclear to me:

[Package Galaxy]

module Galaxy

  (stuff)

  module Star

    using Galaxy

    (stuff)

  end

end

[Forked Package GalaxyBlue]

module Galaxy

  (stuff)

  module Star

    using Galaxy              <-------------------------------- what happens here? is it going to use the forked version or load the package Galaxy?

    (stuff)

  end

end

(I found a somewhat related discussion here:

https://groups.google.com/forum/#!topic/julia-users/5TAUsNfZmqE 

)

Ben

[Kevin Squire]

Hi Ben, 

On Fri, Mar 7, 2014 at 11:02 AM, ben <benjamin...@gmail.com> wrote:

Thanks, John. 

I am still a little confused:

(1) Does the main module of a package have to be named like the package?

No, but when you type "using Pkg" or "import Pkg", Julia looks first in the the current directory for Pkg.jl, and if not there, it searches the package directory ($HOME/.julia/v0.3), and the LOAD_PATH for the following:

1. Pkg.jl

2. Pkg/src/Pkg.jl

3. Pkg.jl/src/Pkg.jl

(The relevant lines are at the beginning of https://github.com/JuliaLang/julia/blob/master/base/loading.jl.)

So if you rename the package and then try to use "using MyPkg" or "import MyPkg", Julia will be looking for the wrong package name.  You can get around this with

require("GalaxyBlue/src/Galaxy.jl")   

# or require("GalaxyBlue/src/Galaxy") -- the .jl is optional

using Galaxy  # Finds the Galaxy module you just required

I've opened up an issue to clarify this (https://github.com/JuliaLang/julia/issues/6082).

 

(2) What is the priority of modules/packages when, eg, calling `import Galaxy` if there is both a package and a module named Galaxy (see example below)?

This should hopefully be clarified by the answer to (1) above.

 

(3) What is the correct use of import/using/require? 

Some of this is described here:

http://julia.readthedocs.org/en/latest/manual/modules/

although it should probably be updated.

Briefly:

1. "using MyModule" imports all of the exported functions of MyModule into the current namespace

2. "import MyModule" imports the module name; functions can be called with "MyModule.fn(x,y)"

3. "import MyModule: fn1, fn2" imports the functions "fn1" and "fn2" into the current namespace

4. "require("somefile.jl")" loads a particular file.  This is useful when:

  a. you have a script or some bare function definitions you wish to load/execute from another file or at the REPL

  b. a file defines a module with a different name than the filename (as in my answer to #1 above)

 

(4) What is the point of the (documented) syntax `Pkg.clone("github/fork/url/Galaxy.jl.git", "GalaxyBlue")`? Is it deprecated? Is it exactly equivalent to a git clone from terminal?

Pkg.clone("http://github.com/myid/SomePackage.jl") is meant to clone a package that might not be in the official list.  It is equivalent to git clone in the Julia package directory, with the only difference that it strips ".jl" from the name (but Julia will find the package contents either way--see the answer to 1 above).

I'm not familiar with the second parameter, but I assume it does what you ask.

 

Here is an example which can happen in practice and which is very unclear to me:

[Package Galaxy]

module Galaxy

  (stuff)

  module Star

    using Galaxy

    (stuff)

  end

end

[Forked Package GalaxyBlue]

module Galaxy

  (stuff)

  module Star

    using Galaxy              <-------------------------------- what happens here? is it going to use the forked version or load the package Galaxy?

    (stuff)

  end

end

Testing:

kevin@IsiahThomas:~/.julia/v0.3$ find Galaxy

Galaxy

Galaxy/src

Galaxy/src/Galaxy.jl

kevin@IsiahThomas:~/.julia/v0.3$ cat Galaxy/src/Galaxy.jl

module Galaxy

println("Galaxy")

end

kevin@IsiahThomas:~/.julia/v0.3$ julia -e 'using Galaxy'

Galaxy

kevin@IsiahThomas:~/.julia/v0.3$ find GalaxyQuest

GalaxyQuest

GalaxyQuest/src

GalaxyQuest/src/Galaxy.jl

kevin@IsiahThomas:~/.julia/v0.3$ cat GalaxyQuest/src/Galaxy.jl

module Galaxy

export galaxy

galaxy() = "Milky Way"

module Star

using Galaxy

println(Galaxy.galaxy())

end

end

kevin@IsiahThomas:~/.julia/v0.3$ julia -e 'require("GalaxyQuest/src/Galaxy"); using Galaxy.Star'

Milky Way

This is what would be expected, actually, since the Galaxy definition is already loaded.  No magic.  If the module definition exists, it's used.

 

Hope this is helpful!

(If you wanted to take this answer and update the documentation, a pull request would be very much appreciated.)

Cheers,

   Kevin

[Mauro]

Thanks, Ben for asking, thanks, Kevin for clarifying.

One related thing which I'm a bit unclear on is the difference between
`require` and `include`. include pastes the code of the file into that
place whereas require loads it. this seems similar, probably equivalent
sometimes but not quite the same. So, what is the difference? Maybe
this should go into the manual too.

And a list like this (+include) in the manual would be helpful too:

--

[Jameson Nash]

The primary difference is the search path: `include` searches relative to the current file, whereas `require` searches relative to the user installation (also `require` is implemented with `include`)

[ben]

Thanks Kevin, that's a great answer! Along with Jameson's explanation of the difference between include() and require(), I now have a much better understanding of these things. 

The only outstanding question as far as I am concerned is: is there a good use case for Pkg.clone("github/fork/url/Galaxy.jl.git", "GalaxyBlue")? I find it misleading that you are given an option to change the package name in a way which will always (most of the time?) break ulterior "import" / "using", unless you manually change the name of the source file Galaxy.jl -> GalaxyBlue.jl. But maybe I am failing to imagine a situation where this is useful.

(I am continuing discussion about the pull request directly on github's corresponding issue.)