update lock file only from $GOPATH

[roger peppe]

Hi all,

Is there way to update the lock file (only) from the versions of the
currently checked out dependencies in $GOPATH?

I tried "dep ensure -no-vendor" but that spends ages (minutes)
going to fetch latest versions of all dependencies, and may
end up with something different, when I actually
just want to lock to what's already there.

I'd expect it to check that the currently checked out dependencies
are compatible with Gopkg.toml, but otherwise just update
Gopkg.lock.

Can some combination of "dep ensure" flags do this?

cheers,
rog.

[Sunny]

Hi,

I'm afraid dep can't do that right now because dep can consider
packages in GOPATH only when initialized with certain flag, and you
would lose your existing lock file if you do that. But I think that
should be fine because you want what's in your GOPATH. So, deleting
your manifest (Gopkg.toml) and lock (Gopkg.lock), and running `dep
init -gopath` would work for you. It will still download the packages
to dep's cache and that might take time, depending on your Internet
speed. The resulting manifest and lock would have projects with the
same version as in your GOPATH. Since dep performs static analysis and
ensures that all the dependencies are met, you would be good.

Hope this helps.

[roger peppe]

On 20 December 2017 at 11:43, Sunny <indias...@gmail.com> wrote:
> Hi,
>
> I'm afraid dep can't do that right now because dep can consider
> packages in GOPATH only when initialized with certain flag, and you
> would lose your existing lock file if you do that. But I think that
> should be fine because you want what's in your GOPATH. So, deleting
> your manifest (Gopkg.toml) and lock (Gopkg.lock), and running `dep
> init -gopath` would work for you.

Hi,

Thanks for your response.

Deleting Gopkg.toml doesn't sound like a great idea, as that's got
human stuff that we want to keep.

Or are you suggesting something like this?

rm Gopkg.toml Gopkg.lock
dep init -gopath
git checkout Gopkg.toml

> It will still download the packages
> to dep's cache and that might take time, depending on your Internet
> speed.

I thought the point about the -gopath flag was that it didn't
go to the network. Is that wrong?

In general, I'm finding it a bit frustrating that most dep operations
go to the network for the latest version of all repositories, as repositories
change rarely, I usually don't mind about having the absolute latest version,
and fetching all that data takes ages (on the order of minutes) and isn't
even possible when I'm not connected to the internet.
I would much prefer it if all dep operations worked disconnected by default,
only going to the network if explicitly directed or when absolutely necessary
(e.g. no version of the dependency can be found in $GOPATH).

cheers,
rog.

> The resulting manifest and lock would have projects with the
> same version as in your GOPATH. Since dep performs static analysis and
> ensures that all the dependencies are met, you would be good.
>
> Hope this helps.
>
>
> On Wednesday, December 20, 2017 at 3:58:49 PM UTC+5:30, roger peppe wrote:
>>
>> Hi all,
>>
>> Is there way to update the lock file (only) from the versions of the
>> currently checked out dependencies in $GOPATH?
>>
>> I tried "dep ensure -no-vendor" but that spends ages (minutes)
>> going to fetch latest versions of all dependencies, and may
>> end up with something different, when I actually
>> just want to lock to what's already there.
>>
>> I'd expect it to check that the currently checked out dependencies
>> are compatible with Gopkg.toml, but otherwise just update
>> Gopkg.lock.
>>
>> Can some combination of "dep ensure" flags do this?
>>
>> cheers,
>> rog.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-package-manag...@googlegroups.com.
> To post to this group, send email to go-package...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Sam Boyer]

On Wednesday, December 20, 2017 at 12:28:37 PM UTC-5, roger peppe wrote:

On 20 December 2017 at 11:43, Sunny <indias...@gmail.com> wrote:
> Hi,
>
> I'm afraid dep can't do that right now because dep can consider
> packages in GOPATH only when initialized with certain flag, and you
> would lose your existing lock file if you do that. But I think that
> should be fine because you want what's in your GOPATH. So, deleting
> your manifest (Gopkg.toml) and lock (Gopkg.lock), and running `dep
> init -gopath` would work for you.

Hi,

Thanks for your response.

Deleting Gopkg.toml doesn't sound like a great idea, as that's got
human stuff that we want to keep.

Or are you suggesting something like this?

    rm Gopkg.toml Gopkg.lock
    dep init -gopath
    git checkout Gopkg.toml

This is probably the fastest way to get to your goal of incorporating what's currently on GOPATH, though you'll need another `dep ensure` run afterwards to make sure that everything in the new `Gopkg.lock` is still OK with respect to `Gopkg.toml`.

 

> It will still download the packages
> to dep's cache and that might take time, depending on your Internet
> speed.

I thought the point about the -gopath flag was that it didn't
go to the network. Is that wrong?

it is, yes. i'm actually writing docs related to this literally right now, to be shipped with the soon-to-be-released v0.4.0 :) meanwhile, though, the updated spec doc https://docs.google.com/document/d/1Qwa3jDKDh45t5qAGjRpeu3ZsAbkukIYDDmYJdMxH2uY/edit explains the init mechanics a bit more in-depth.

 

In general, I'm finding it a bit frustrating that most dep operations
go to the network for the latest version of all repositories, as repositories
change rarely, I usually don't mind about having the absolute latest version,
and fetching all that data takes ages (on the order of minutes) and isn't
even possible when I'm not connected to the internet.
I would much prefer it if all dep operations worked disconnected by default,
only going to the network if explicitly directed or when absolutely necessary
(e.g. no version of the dependency can be found in $GOPATH).

  cheers,
    rog.

i quite agree :) this is also the sort of behavior we're aiming for with https://github.com/golang/dep/issues/431. unfortunately, it's pretty tricky work to implement effectively - we're basically building a layered cache there. it's conceivable that we could ship something that simply avoids the initial network operations without needing to do all the persistent caching so that we could see some benefits sooner, but i haven't really had my head in that work for about a month now, so i'm not sure how realistic it would be to separate skipping the network from the persistent local disk cache.

in any case, help would be very, very welcome on this front; this is one of two things that makes dep really quite absurdly slow.

 

> The resulting manifest and lock would have projects with the
> same version as in your GOPATH. Since dep performs static analysis and
> ensures that all the dependencies are met, you would be good.
>
> Hope this helps.
>
>
> On Wednesday, December 20, 2017 at 3:58:49 PM UTC+5:30, roger peppe wrote:
>>
>> Hi all,
>>
>> Is there way to update the lock file (only) from the versions of the
>> currently checked out dependencies in $GOPATH?
>>
>> I tried "dep ensure -no-vendor" but that spends ages (minutes)
>> going to fetch latest versions of all dependencies, and may
>> end up with something different, when I actually
>> just want to lock to what's already there.
>>
>> I'd expect it to check that the currently checked out dependencies
>> are compatible with Gopkg.toml, but otherwise just update
>> Gopkg.lock.
>>
>> Can some combination of "dep ensure" flags do this?
>>
>>   cheers,
>>     rog.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to go-package-management+unsub...@googlegroups.com.

[Sam Boyer]

On Wednesday, December 20, 2017 at 10:10:39 PM UTC-5, Sam Boyer wrote:

On Wednesday, December 20, 2017 at 12:28:37 PM UTC-5, roger peppe wrote:

On 20 December 2017 at 11:43, Sunny <indias...@gmail.com> wrote:
> Hi,
>
> I'm afraid dep can't do that right now because dep can consider
> packages in GOPATH only when initialized with certain flag, and you
> would lose your existing lock file if you do that. But I think that
> should be fine because you want what's in your GOPATH. So, deleting
> your manifest (Gopkg.toml) and lock (Gopkg.lock), and running `dep
> init -gopath` would work for you.

Hi,

Thanks for your response.

Deleting Gopkg.toml doesn't sound like a great idea, as that's got
human stuff that we want to keep.

Or are you suggesting something like this?

    rm Gopkg.toml Gopkg.lock
    dep init -gopath
    git checkout Gopkg.toml

This is probably the fastest way to get to your goal of incorporating what's currently on GOPATH, though you'll need another `dep ensure` run afterwards to make sure that everything in the new `Gopkg.lock` is still OK with respect to `Gopkg.toml`.

 

i meant to say a bit more here: regardless of whether -gopath or -skip-tools is passed, dep still has to run all the inferences it has made from tools/GOPATH through its solver in order to guarantee that the invariants dep maintains are satisfied. That does entail hitting the network, for the same reason that `dep ensure` needs to.

If we didn't do this check, we'd be generating potentially invalid Gopkg.toml and Gopkg.lock files. That doesn't really help (except for the case where you're trying to incrementally convert - https://github.com/golang/dep/issues/909), as you'll just face the same hurdles on subsequent `dep ensure` runs.

[roger peppe]

> That does entail hitting the network, for the same reason that `dep ensure` needs to.

I don't really understand this. Why *does* this need to hit the
network, given that
modern distributed VCS systems tend to cache all information locally?

Also, is there an underlying issue here that it's hard with git (or
other VCS's) to tell
what version(s) a given checked out commit corresponds to?

>>> > email to go-package-manag...@googlegroups.com.

>>> > To post to this group, send email to go-package...@googlegroups.com.
>>> > For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to go-package-manag...@googlegroups.com.