explain functionality similar to "go mod why for" to help with message "build constraints exclude all Go files in ..." ?

[fge...@gmail.com]

Is there some functionality to list each build constraint that is not
satisfied when the result of go get is "build constraints exclude all
Go files in ..."?

go mod why is very helpful when module dependencies are to be explained.
A similar functionality would be helpful and maybe a message to refer
to this functionality when "build constraints exclude all Go files in
..." is the conclusion.

(For a random package I've found on pkg.go.dev it took a few confusing
minutes until I realized that cgo requirement is the build
constraint.)

On a tangential note: would anybody else be interested to help with
issue https://github.com/golang/go/issues/39195 ? (Probably all build
constraints should be searchable.)

[Axel Wagner]

In general, answering that question is NP-complete. It's the boolean satisfiability problem. It would be possible to implement *some* solution and maybe stop after a timeout or something. But even then, the answer will not be unique and might sometimes be less than helpful.

This doesn't mean we *can't* do it, but the problem is harder than it might seem. You could file an issue (after searching if someone else suggested it already, of course :) ).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2Bctqrq1%2BSVDQO3ecgyj7Kq4Qv6-tE3QWFTrVHtNqro8hUCwCw%40mail.gmail.com.

[fge...@gmail.com]

Thanks for thinking about the issue!
I should have asked for something more explicit, probably something like this:
; go build listconstraintexclusions
buildconstraint_1 excludes:
file1.go
file2.go
file3.go
buildconstraint_2 excludes:
file4.go
file5.go
No go files left to build.
;

Iiuc the complexity of creating this list is in P.
(I understand my original question was different.)

On 3/29/21, Axel Wagner <axel.wa...@googlemail.com> wrote:
> In general, answering that question is NP-complete. It's the boolean
> satisfiability

> problem <https://en.wikipedia.org/wiki/Boolean_satisfiability_problem>. It

[Axel Wagner]

I might not understand what you are intending.

My understanding is that you want, given a set of .go files, know "why" they are excluded. Which, to me, means finding a configuration of build tags that would allow at least one of them to be built. A file can be built with a set of tags, if the boolean formula expressed by the build constraints evaluates to true. Therefore, determining why a file is excluded by a set of build tags is the boolean assignability problem.

It is trivial, given a set of build tags, to find out if a given file builds. So, it is easy to say "this file does not build given the build tags I have". But it's NP-complete, to find out what build tags must be set for a file to build. So it's hard to say "this file does not build, but it *would* if I changed the build tags to that".

It's even harder to determine what the "intuitively right" answer is. For example, a package might require cgo when built on Windows/arm, but not when built on Linux/amd64. So both the answer "you need cgo" and the answer "you need to amd64" would be correct, if you are trying to build on Windows/arm and it's not obvious which is correct. Or, a more trivial example: What about a build tag like `// +build foo,!foo`? Should the report say `foo` has to be set, or that it has to not be set, or do we need additional logic to detect cases like this?

Again, none of this means we can't do *something*. But without a clear, efficient algorithm and without an obviously correct answer to ambiguities, we'd first have to decide on what we'd specifically like to do :)

[Axel Wagner]

FWIW, you might be assuming that a build tag always has the form `// +build linux` or `// +build !cgo` - in which case it's indeed clear what the "correct" answer is and how to determine it. But you can't assume that build tags only take that form, you can express *any* boolean formula, using any amount of variables with them.

[fge...@gmail.com]

Thanks, fully agree on everything.

Please consider this workflow:
1. find package on pkg.go.dev (currently there is no build constraint
specific information listed beside the result, cf. issue #39195)
2. get package (go get, git clone, cp etc.)
3. "build constraints exclude all Go files in ..."
4. wat? (pkg.go.dev did not "promise" this)
5. let's visit _each_file manually (or try to grep for some hints) to
try to identify all failing build constraints and think about how to
satisfy them or give up failing.

Please also note, if step 1 or 2 or 3 provided more information on
build constraints, the failing step 3 wouldn't be surprising. Now it
is and I believe some tooling support (either in step 1, 2 or 3) would
be helpful.

(I agree the answer to "what build tags must be set for a file to
build" is NP complete.)

An "explanation" like I described in my 2nd email would be very
helpful and that isn't more complex than P. (If it is, please provide
a hint.)

Or even just the identified and relevant build constraints in CNF is
most probably enough to take a look at and go like nah, I can see
buildconstraintX and buildconstraintY, I can give up _quickly_ and not
waste _more_ time trying to get the package to build.

If you agree the workflow above is not ideal, what do you think about
the "build constraint explanation" or the build constraints in CNF?

On 3/29/21, Axel Wagner <axel.wa...@googlemail.com> wrote:
> FWIW, you might be assuming that a build tag always has the form `// +build
> linux` or `// +build !cgo` - in which case it's indeed clear what the
> "correct" answer is and how to determine it. But you can't assume that
> build tags only take that form, you can express *any* boolean formula,
> using any amount of variables with them.
>
> On Mon, Mar 29, 2021 at 12:47 PM Axel Wagner
> <axel.wa...@googlemail.com>
> wrote:
>
>> I might not understand what you are intending.
>>
>> My understanding is that you want, given a set of .go files, know "why"
>> they are excluded. Which, to me, means finding a configuration of build
>> tags that would allow at least one of them to be built. A file can be
>> built
>> with a set of tags, if the boolean formula expressed by the build

>> constraints <https://golang.org/cmd/go/#hdr-Build_constraints> evaluates

[Axel Wagner]

On Mon, Mar 29, 2021 at 2:57 PM <fge...@gmail.com> wrote:

Thanks, fully agree on everything.

Please consider this workflow:
1. find package on pkg.go.dev (currently there is no build constraint
specific information listed beside the result, cf. issue #39195)
2. get package (go get, git clone, cp etc.)
3. "build constraints exclude all Go files in ..."
4. wat? (pkg.go.dev did not "promise" this)
5. let's visit _each_file manually (or try to grep for some hints) to
try to identify all failing build constraints and think about how to
satisfy them or give up failing.

Please also note, if step 1 or 2 or 3 provided more information on
build constraints, the failing step 3 wouldn't be surprising. Now it
is and I believe some tooling support (either in step 1, 2 or 3) would
be helpful.

I agree. I'm not saying the problem is not worth solving and I'm not saying we can't do *something* to try and address it. I just don't know what, because I don't know an answer that seems "obviously right".

An "explanation" like I described in my 2nd email would be very
helpful and that isn't more complex than P. (If it is, please provide
a hint.)

I still don't understand how you think we should provide this explanation without solving SAT.

For example, you write

buildconstraint_1 excludes:
file1.go
file2.go
file3.go

What is "builtconstraint_1" and how is it determined by the go tool? My first impression was that it is supposed to be a build tag (like "cgo"), which opens up the problems I asked - it's not clear which tag is "at fault" or how to figure that out.

Or even just the identified and relevant build constraints in CNF is
most probably enough to take a look at and go like nah, I can see
buildconstraintX and buildconstraintY, I can give up _quickly_ and not
waste _more_ time trying to get the package to build.

Sure. That's certainly easy enough to do. FWIW, currently you can get an approximation of that via

grep -R "// +build" .

After #41184 that would give an even clearer result, as the build constraints are actually guaranteed to be in DNF with at most one line per file (AIUI).

It is certainly possible to build a version of this into the Go tool for go 1.17 already. That is, we could build in a command that just recursively looks for and parses build constraints and prints them out per-file in CNF or DNF. Personally, I feel like I would rarely use it though, because I'd find just a plain grep easier to remember, given that I use it daily. But my personal feelings shouldn't deter you from proposing this :)

[fge...@gmail.com]

On 3/29/21, Axel Wagner <axel.wa...@googlemail.com> wrote:
> On Mon, Mar 29, 2021 at 2:57 PM <fge...@gmail.com> wrote:

...

> An "explanation" like I described in my 2nd email would be very
>> helpful and that isn't more complex than P. (If it is, please provide
>> a hint.)
>>
>
> I still don't understand how you think we should provide this explanation
> without solving SAT.
> For example, you write
>
> buildconstraint_1 excludes:
> file1.go
> file2.go
> file3.go
>
> What is "builtconstraint_1" and how is it determined by the go tool? My
> first impression was that it is supposed to be a build tag (like "cgo"),
> which opens up the problems I asked - it's not clear which tag is "at
> fault" or how to figure that out.

I assumed (ha!) buildconstraint_1 as a parsed predicate (e.g. "//
+build linux") would be helpful. In a more complex case it isn't.
Thanks again!

>
> Or even just the identified and relevant build constraints in CNF is
>> most probably enough to take a look at and go like nah, I can see
>> buildconstraintX and buildconstraintY, I can give up _quickly_ and not
>> waste _more_ time trying to get the package to build.
>>
>
> Sure. That's certainly easy enough to do. FWIW, currently you can get an
> approximation of that via
>
> grep -R "// +build" .
>

> After #41184 <https://github.com/golang/go/issues/41184> that would give an

> even clearer result, as the build constraints are actually guaranteed to be
> in DNF with at most one line per file (AIUI).
>
> It is certainly possible to build a version of this into the Go tool for go
> 1.17 already. That is, we could build in a command that just recursively
> looks for and parses build constraints and prints them out per-file in CNF
> or DNF. Personally, I feel like I would rarely use it though, because I'd
> find just a plain grep easier to remember, given that I use it daily. But
> my personal feelings shouldn't deter you from proposing this :)

agreed :)

Still the experience is very much non-go like, though with such a
power build constraint language it'll stay like that.

[Wojciech S. Czarnecki]

Dnia 2021-03-29, o godz. 15:17:42
"'Axel Wagner' via golang-nuts" <golan...@googlegroups.com> napisał(a):

> I still don't understand how you think we should provide this explanation without solving SAT.

...

> What is "builtconstraint_1" and how is it determined by the go tool? My
> first impression was that it is supposed to be a build tag (like "cgo"),
> which opens up the problems I asked - it's not clear which tag is "at
> fault" or how to figure that out.

As I understood the OP, it would be nice to have build -n and -v messaging us
with short info about the constraint that excluded given file from under build.

And I concur.

--
Wojciech S. Czarnecki
<< ^oo^ >> OHIR-RIPE

[fge...@gmail.com]

Since the build constraint language is equivalent to a first order logic language, no guaranteed useful information can be provided in the general case in a limited time.

I also thought that predicates would be useful, but only the terms in a normal form could be useful, but that would provide - most probably - "too much".

grep should be good enough :)

[Wojciech S. Czarnecki]

Dnia 2021-03-29, o godz. 20:41:02
fge...@gmail.com napisał(a):

> Since the build constraint language is equivalent to a first order logic language, no guaranteed useful information can be provided in the general case in a limited time.

The build tool decides about exclusion anyway. In the very moment it knows a file is to be skipped it could print (with -n or -v) that constraint line that gave the deciding predicate. There is enough information for our brains to solve a "why we got no valid sources here" puzzle.

> grep should be good enough :)

Why build a regexp later for something build tool knows in situ?
(https://play.golang.org/p/BeYjYFgLTuU)

export GOOS=linux
cd buildconstr

CURRENT:

go build -n -tags bsddebug
package fairbe.org/ohir/buildconstr: build constraints exclude all Go files in /path.../buildconstr


DESIRED:

go build -n -tags bsddebug
(command config lines elided)
./a.go excluded at line 4: // +build darwin freebsd netbsd openbsd
./b.go excluded at line 3: // +build lindebug,linux
./c.go excluded at line 3: // +build !lindebug,!bsddebug
./d.go excluded at line 3: // +build windows
./e_windows.go excluded by name constraint (GOOS != windows)
package github.com/ohir/buildconstr: build constraints exclude all Go files in /path.../buildconstr


TC,

[Axel Wagner]

Hi,

FWIW I believe printing every build constraint for every file skipped is far too verbose, even for -n or -v mode.

It is not uncommon to use build constraints and when you do, you often do it for many or even all files in a package. I don't even think it would be helpful, because it would be very hard to glean the relevant information from it.

That's notably very different from a distinct flag or command, which doesn't spam in the common case and can filter the information down to a more useful subset.

Why build a regexp later for something build tool knows in situ?

You don't need a regexp. You just need to grep for `// +build`. In the future (after go 1.18), you can grep for `//go:build` and get a per-file DNF form of all build constraints in a package.

Again, just to be on record: I'm not denying the usefulness of tooling being able to give you more insight into if and why files are excluded from the build.

(https://play.golang.org/p/BeYjYFgLTuU)

export GOOS=linux
cd buildconstr

CURRENT:

go build -n -tags bsddebug
package fairbe.org/ohir/buildconstr: build constraints exclude all Go files in /path.../buildconstr


DESIRED:

go build -n -tags bsddebug
   (command config lines elided)
./a.go excluded at line 4: // +build darwin freebsd netbsd openbsd
./b.go excluded at line 3: // +build lindebug,linux
./c.go excluded at line 3: // +build !lindebug,!bsddebug
./d.go excluded at line 3: // +build windows
./e_windows.go excluded by name constraint (GOOS != windows)
package github.com/ohir/buildconstr: build constraints exclude all Go files in /path.../buildconstr


TC,

--
Wojciech S. Czarnecki
 << ^oo^ >> OHIR-RIPE

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/20210330140808.20c13b4d%40xmint.

[Wojciech S. Czarnecki]

Dnia 2021-03-30, o godz. 14:20:29

"'Axel Wagner' via golang-nuts" <golan...@googlegroups.com> napisał(a):

> FWIW I believe printing every build constraint for every file skipped is
> far too verbose, even for -n or -v mode.

Both -n and -v modes already are very verbose, for purpose.

If I am giving either I expect to see a torrent of information. Now I can see exact
compile commands, what is missing and so on but the simple information that
a file was excluded isn't there. It might be.

> That's notably very different from a distinct flag or command, which
> doesn't spam in the common case and can filter the information down
> to a more useful subset.

Of course it would be better to have specialized flag, eg. "-excluded"

> You don't need a regexp. You just need to grep for `// +build`.

Heh, `// +build` is-a regexp :) Grep is fast, nonetheless `grep -rn "// +build"`
matches this regexp on every line of every file (you also can find -print| but that's
yet more to alias and run). Just to print what `build` can just log after reading
a few lines it must read anyway.


DESIRED:
go build -n -excludes -tags bsddebug

[Axel Wagner]

Again: I understand what you are asking for and I'm not saying there is no benefit.

Personally, I don't feel the benefit is very large and I don't think it's worth it. You might very well think it is and that's fine. As i said in my first message, if anyone feels this is important to have, my suggestion is to search github for an existing issue about this (I would not be surprised if there is one) and filing one if there are none.

I don't think there is anything else to discuss in this thread - IMO all the information is here. Either you think it's a good idea, or you don't.

[fge...@gmail.com]

Thank you both, for sharing your thoughts on this!

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEk4FQovmMhgYVY2cgrGRPJPPmDNqCMa33%3DNhW%2BTC_vnw%40mail.gmail.com.
>