goprox - dependency caching

[Daniel Theophanes]

Hi Aaron,

You mentioned you are developing goprox https://github.com/arschles/goprox  in another thread. I'm breaking it out here.

This is a general concept go should support and companies need. Google achieves this I hear by copying dependencies into their own source tree. Capital One also has strong controls on where source comes from, though I think they do it differently. I know heroku has to employ this type of proxies for other languages as well. So I agree that the concept of: "put dependencies under your control when you fetch them" is a good and needed one.

There are several places to place such a cache, having one doesn't always replace another:

• Theoretically you could configure a tool at the ENV level to always look at a proxy location for deps (possibly never go to end source).
• With today's tools you can configure a flexible origin to point to a local vcs repository.
• With today's tools you can store the current revision of the dep in your project's vendor directory.

Anecdotally, I don't really get affected by github downtime or the code.google.com transition because I copy dependencies into the local vendor directory.

I do think it is important for us to have a story for organizations, such as Capital One which disallow copying dependencies into the local repository but still need to control the origin. Obviously there are solutions available, but I'm not a part of that environment first hand so I'm unsure the best solution to recommend. There may be other options we also want to consider as well (I have a few Ideas, though I don't care to go into them right now).

I do want to let you know I'm at least partially aware of situations like yours, and would like to help design a first class solution. -Daniel

[Sam Boyer]

Agreed, this is important. Some Uber folks came into the glide queue a while back to raise this issue, or at least part of it: https://github.com/Masterminds/glide/issues/372

When I was thinking about this problem a while back, I split it into two problems:

• Reverse proxy-type situations: keeping a durable local mirror that replicates the upstream, excepting that it stays around if the upstream goes away.
• Literally anything that is fancier, in any way, at all.

Maybe this is obvious, but it still seems worth stating. This is a problem that, while it looks similar to a lot of other things we might want a dep mgmt system to do, can probably be solved within a narrow scope having few side effects, and so it's best to do so.

[Florin Pățan]

I'd like to point out that there are various other tools out there that can be used to achieve the same thing. Not to mention that a package manage could in fact remove the need for such a proxy if, for example, a manifest file would exist that specifies where the location of a certain package is.

[Matt Farina]

On Saturday, July 30, 2016 at 7:55:42 PM UTC-4, Florin Pățan wrote:

I'd like to point out that there are various other tools out there that can be used to achieve the same thing. Not to mention that a package manage could in fact remove the need for such a proxy if,  for example, a manifest file would exist that specifies where the location of a certain package is.

There are two types of aliases or other locations sometimes needed.

1. To specify an alternative location in a manifest file in a source repo. This is useful for things like forks.
2. To specify an alternate location that's not part of the source. Such as when you mirror a public repo. Sometimes you want to pull from a mirror without altering the source. The config for that is part of your toolchain.
   

The #2 option is where things like goprox come in handy.

Other programming language ecosystems (and even linux dists) have mirrors like this.