go.dev is live!

[Julie Qiu]

Hey Gophers, 

We are excited to share that go.dev, a new hub for Go developers, is now live!

At go.dev, you will find information on how to get started with the language, featured use cases, and other resources. It is a companion site to golang.org. You can read about it on the latest Go blog post.

You will also find a new place to discover Go packages and modules, pkg.go.dev, by clicking on Explore in the header or footer.

Pkg.go.dev serves Go documentation like godoc.org, but it also understands modules and has information about previous versions of a package (such as all releases of the Go standard library!). It also detects and displays licenses and has a better search algorithm.

You can follow Go issue 33654 for future developments on pkg.go.dev.

We are just starting to build out go.dev, so some bugs are to be expected. We want to work with all of you to make this site better for Go developers, so please share your feedback with us and file issues if you spot them! You can do that by clicking "Share Feedback" or "Report an Issue" at the footer of every page, or by emailing go-discove...@google.com. See go.dev/about for more information about the site.

We hope you enjoy the new site and look forward to hearing your feedback!

Warmly ☀️, 

Julie for the Go team

[Dan Kortschak]

Bringing this to golang-dev.

I've looked into the problem more and it's worse than this. In some
cases, where a package has a non-recognised license and is vendored
into another repository, the containing repository's license is
reported as the license of the vendored package. Potentially leading to
the incorrect understanding of the licensing of the vendored code and
possibly being interpreted as an assertion of rights over the code by
people who are not the authors.

This can be seen here: https://pkg.go.dev/search?q=k8s.io+gonum

On Thu, 2019-11-14 at 20:30 +1030, Dan Kortschak wrote:
> Yes. This is my exact concern.
>
> This has two impacts, one is the non-discoverability and the other is
> a
> possibility of misapprehension that the packages are in fact non-
> licensed which is an actual harm to the packages. The irony is that
> we
> have gone to excessive lengths to ensure that all our original
> sources
> have their licenses represented as well.
>
> On Thu, 2019-11-14 at 10:56 +0100, Jan Mercl wrote:
> > On Thu, Nov 14, 2019 at 5:55 AM Dan Kortschak <d...@kortschak.io>
> > wrote:
> >
> > > It looks like license detection needs work.
> > >
> > > See https://pkg.go.dev/gonum.org/v1/gonum?tab=overview and note
> > > it
> > > has
> > > a BSD 3 clause, as shown by GitHub's assessment (just above the
> > > "Clone
> > > or download" button) at https://github.com/gonum/gonum and the
> > > LICENSE
> > > file that it links to.
> >
> > Moreover, such packages seem to be, as a side effect, non-
> > discoverable
> > because their README and documentation are not indexed (nor
> > displayed). For example, searching for `pure go SQL database' has
> > zero
> > results.
> >
> > I don't care about automatic license recognizing, even though it's
> > admittedly a useful information for the site user. But I don't
> > understand why projects with unrecognized LICENSE file contents
> > don't
> > show the README and the documentation.
> >
> > I'll better leave the fact of hiding even the LICENSE file per se
> > from
> > the user with the message `“Licenses” hidden due to license
> > restrictions.` without comment because I guess it's just a legal
> > issue
> > I don't understand.
> >
> > Any yes, all my packages that I tried are victims to the above
> > issues.
> > Here's an example of a recognized LICENSE vs an unrecognized one:
> >
> > jnml@e5-1650:~/src/modernc.org/mathutil> diff -u ~/goroot/LICENSE
> > LICENSE
> > --- /home/jnml/goroot/LICENSE 2019-09-10 14:24:54.469859557 +0200
> > +++ LICENSE 2019-09-10 14:36:43.347068286 +0200
> > @@ -1,4 +1,4 @@
> > -Copyright (c) 2009 The Go Authors. All rights reserved.
> > +Copyright (c) 2014 The mathutil Authors. All rights reserved.
> >
> > Redistribution and use in source and binary forms, with or without
> > modification, are permitted provided that the following conditions
> > are
> > @@ -10,7 +10,7 @@
> > copyright notice, this list of conditions and the following
> > disclaimer
> > in the documentation and/or other materials provided with the
> > distribution.
> > - * Neither the name of Google Inc. nor the names of its
> > + * Neither the names of the authors nor the names of the
> > contributors may be used to endorse or promote products derived
> > from
> > this software without specific prior written permission.
> >
> > jnml@e5-1650:~/src/modernc.org/mathutil>
> >
>
>

[Ian Davis]

It appears the misleading licensing issue is still present on https://pkg.go.dev/

Is there any update from the go.dev team on this or an issue that tracks this?

-- Ian

> --
> You received this message because you are subscribed to the Google
> Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to golang-dev+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-dev/b81210af1f197b789f5b81d12002d15e0f929540.camel%40kortschak.io.
>

[Dan Kortschak]

Yes it is still there. I have sent queries to the (closed) contact list
go-discove...@google.com and they say they are working on it.
The last email I received from them was 18 December. I have seen no
evidence of any work to actually fix the problem other than to extend
the disclaimer here[0] to say that the correctness of the license
detection is not guaranteed.

As a demonstration of the incorrectness of using a machine to do a
lawyer's work, I have written this package[1] and it is now listed as a
BSD 3-clause licensed work. Read licenses carefully.

Dan

[0]https://pkg.go.dev/license-policy
[1]https://pkg.go.dev/github.com/kortschak/unlicensable

[Russ Cox]

On Mon, Jan 6, 2020 at 4:16 AM Dan Kortschak <d...@kortschak.io> wrote:

Yes it is still there. I have sent queries to the (closed) contact list
go-discove...@google.com and they say they are working on it.
The last email I received from them was 18 December. I have seen no
evidence of any work to actually fix the problem other than to extend
the disclaimer here[0] to say that the correctness of the license
detection is not guaranteed.

Hi Dan,

You probably didn't intend it, but your mail comes across as pretty critical

of the team's efforts. 

December 18 was just before what for many people was a two-week vacation. 

I can assure you that, even if you can't see "evidence of any work", the team

here is working quite hard on improving license detection on pkg.go.dev.

License detection is very difficult. Anyone who tells you otherwise has 

not tried to do it at scale (or not inspected the correctness of their results!).

On top of that, changes have legal implications, so quite often more care and

review is needed than for other kinds of bug fixes.

Thanks for your patience.

Best,

Russ

[Dan Kortschak]

Hi Russ,

Thanks for reaching out. It probably comes across that way because I
have seen little engagement with the issue because of the closed nature
of the development process and issue tracking and this is extremely
frustrating. The closed approach is at odds with the historical
approach to Go development. The issue that I raised initially were
brought up in an issue in mid November and closed where I was told to
take it to the feedback list which is not an open forum.

At the moment, Gonum (and moderncc) at least fail to have their license
detected when presented as their own modules/packages, but worse Gonum
packages, when vendored into a module with a detected licence have that
license presented as the license for the Gonum code (this presumably
happens for other vendored code in the same situation). This is a
misrepresentation of the licensing of the code and moral rights of the
code's authors.

I don't doubt that teaching a machine to practice law is difficult, and
doing so correctly is undoubtedly not possible for a human level of
correctness (see [1]). However, it is my view that while the machines
are in law school they should not be practicing where they have this
impact.

regards
Dan

[1]https://pkg.go.dev/github.com/kortschak/unlicensable?tab=licenses

[Dan Kortschak]

As an addendum to this, perhaps go.dev could attempt to defer license
recognition to the hosting service where possible (AFAICS bitbucket
doesn't support this, but GitHub — and maybe GitLab — do which covers
probably most of the packages in the wild). This would capture at least
what those hosts view the license to be which I would imagine goes a
long way to covering Google's position on presenting licensed work
through go.dev.

https://developer.github.com/v4/object/license/

Dan