Using modules with private repos

241 views
Skip to first unread message

Dean Schulze

unread,
Mar 11, 2020, 4:07:18 PM3/11/20
to golang-nuts
The docs for modules assume that you'll be publishing your golang modules (actually a git push) to the few public repos that golang has built in support for.  I want to use a private repo, but there doesn't seem to be any docs for that.  This post has some suggestions, but it relies on entries in .gitcofig that will interfere with normal access to those public repos.  It doesn't mention using ssh keys for authentication.  I assume this is what go does for the public reops it has built in support for.

I hope there is a better way.  Is there?

Marcin Romaszewicz

unread,
Mar 11, 2020, 5:47:16 PM3/11/20
to Dean Schulze, golang-nuts
There is really no nicer way. You do not have to make those .insteadOf overrides in your global git config, though, you can do it in the local git config for your Go repository to contain the damage. You will also find that private repositories don't work nicely with things like the Athens module proxy, since you don't want your CI/CD systems hitting public servers.

-- Marcin

On Wed, Mar 11, 2020 at 1:07 PM Dean Schulze <dean.w....@gmail.com> wrote:
The docs for modules assume that you'll be publishing your golang modules (actually a git push) to the few public repos that golang has built in support for.  I want to use a private repo, but there doesn't seem to be any docs for that.  This post has some suggestions, but it relies on entries in .gitcofig that will interfere with normal access to those public repos.  It doesn't mention using ssh keys for authentication.  I assume this is what go does for the public reops it has built in support for.

I hope there is a better way.  Is there?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/c49ab83c-315c-4de1-9225-7f34d9cf395d%40googlegroups.com.

Bryan C. Mills

unread,
Mar 11, 2020, 7:59:24 PM3/11/20
to golang-nuts
The protocol described in https://golang.org/cmd/go/#hdr-Remote_import_paths allows you to serve any repo URL for any import path that begins with your server's domain name.

You can use basic auth with such a server via credentials stored in a .netrc file, which also works with git.

Dave Mazzoni

unread,
Apr 1, 2020, 7:58:28 AM4/1/20
to golang-nuts
I have become totally confused about using local modules. Please help -- I'll try to keep this short:
I have a package (dfs) and a main package. I tried laying things out an using _ for local packages from your excellent post on github, but I still cannot get things to build correctly.
I got the impression that there should only be 1 go.mod at the top level of a local project (please assume I don't have net access) with 2 sub-directories under it: one with the package dfs, the other with the main package that needs to include dfs.
Should I provide more info? A tree listing and error messages?

Dave Mazzoni

unread,
Apr 1, 2020, 11:47:41 AM4/1/20
to golang-nuts
With all the examples out there and a little experimentation, I found the solution I needed. Here's the tree from the 'top' directory:
├── dfs
│   ├── dfs.go
│   ├── dfs_test.go
│   └── go.mod
├── first.go
├── go.mod
└── ModuleDFS
Here are the file contents starting from the top of the directory structure on down:

go.mod:
======
module ModuleDFS
require _/dfs v0.0.0
replace _/dfs => ./dfs
go 1.14

first.go:
======
package main
import (
"bufio"
"fmt"
"log"
"os"
"strings"

"_/dfs"
)

func main() {
gr := make(map[string][]string)
//       vertex --^    ^-- slice of vertices adjacent to this vertex
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
in := scanner.Text()
// in: A B C etc.
// Ctrl-D at end!
flds := strings.Fields(in) // split line into fields
vs := make([]string, 0)
// set up vertices connected to first vertex at index 0
for f := 1; f < len(flds); f++ {
vs = append(vs, flds[f])
}
gr[flds[0]] = vs
}
if err := scanner.Err(); err != nil {
log.Fatalln("Error scanning:", err)
}
for k, v := range gr {
fmt.Printf("%s:%v\n", k, v)
}
dfs.SetupDfs(gr)
}
// check simply checks for an error w/o inline code everywhere
//  Also, since a fatal error occurred, this is no slowdown!
func check(why string, err error) {
if err != nil {
log.Fatalln(why, ":", err)
}
}

Directory dfs:
==========
go.mod:
======
module _/ModuleDFS/dfs
go 1.14

dfs.go:
=====
package dfs
import "fmt"
// setupDFS initialized dfs variables
func SetupDfs(gr map[string][]string) {
nT := 0 // Total nodes in whole graph
for n := range gr {
nT += len(gr[n])
}
visited := make(map[string]bool, nT)
fmt.Println("Total nodes:", nT, "Visited:", visited)
dfs(gr, "a", visited)
fmt.Println("Final:", visited)
}
// dfs peforms a Depth First Search of all nodes
func dfs(gr map[string][]string, at string,
vs map[string]bool) map[string]bool {
if vs[at] == true {
return vs
}
vs[at] = true
neighbors := gr[at]
fmt.Println("neighbors[", at, "]:", gr[at])
for _, next := range neighbors {
fmt.Println("next:", next)
dfs(gr, next, vs)
}
return vs
}
// Hello -- the most basic of testing
func Hello() string {
return "Hello World"
}

dfs_test.go:
=========
package dfs
import "testing"
func TestDfs(t *testing.T) {
want := "Hello World"
if got := Hello(); got != want {
t.Errorf("Hello() = %q, want %q", got, want)
}
}

Thanks for your help!

David Riley

unread,
Apr 1, 2020, 1:59:04 PM4/1/20
to Dave Mazzoni, golang-nuts
This is actually a fairly neat solution! I might look at using that from here on out.

How well does it work if something else (also with access to your private repository) tries to include your module?


- Dave

David Mazzoni

unread,
Apr 3, 2020, 4:22:51 PM4/3/20
to David Riley, golang-nuts
I’m glad it works for you too. I just added more subdirectory packages to the Module and everything is working well. I haven’t tried to access the Modules’s packages from another Module, although I plan to. I’ll let you all know what works.

Cucullus Non Facit Monachum : The cowl does not make the monk.

Reply all
Reply to author
Forward
0 new messages