Building go executables under tup that use .gitignore feature

[Peter Moore]

I've hit a problem with building go executables when `.gitignore` tup directive is active.

Specifically, it seems `go build` scans repository `.gitignore` files and thus tup considers them as inputs that should be declared.

Example

```
pmoore@Peters-MacBook-Pro:~/tmp $ ls
Tupfile        Tupfile.ini    go.mod        main.go
pmoore@Peters-MacBook-Pro:~/tmp $ cat Tupfile
.gitignore
: |> go build -o %o |> hello-world
pmoore@Peters-MacBook-Pro:~/tmp $ cat Tupfile.ini
pmoore@Peters-MacBook-Pro:~/tmp $ cat go.mod
module github.com/petemoore/tup-go-gitignore

go 1.15
pmoore@Peters-MacBook-Pro:~/tmp $ cat main.go
package main

import "fmt"

func main() {
    fmt.Println("Hello World")
}
pmoore@Peters-MacBook-Pro:~/tmp $
```

Without the `.gitignore` directive in Tupfile, all is ok. However, with it, we get:

```
* 0) go build -o hello-world                                                                                                                                                                                                                        
 *** tup messages ***
tup error: Missing input dependency - a file was read from, and was not specified as an input link for the command. This is an issue because the file was created from another command, and without the input link the commands may execute out of order. You should add this file as an input, since it is possible this could randomly break in the future.
 - [16] .gitignore
 *** Command ran successfully, but failed due to errors processing input dependencies.
 [ ] 100%
 *** tup: 1 job failed.
```

If I now add `.gitignore` as an order-only input, i.e.

```
pmoore@Peters-MacBook-Pro:~/tmp $ cat Tupfile
.gitignore
: | .gitignore |> go build -o %o |> hello-world
```

then instead I get:

```
* 0) [0.001s] .
tup error: Explicitly named file '.gitignore' in subdir '.' is scheduled to be deleted (possibly the command that created it has been removed).
tup error: Error parsing Tupfile line 2
  Line was: ': .gitignore |> go build -o %o |> hello-world'
 [ ] 100%
 *** tup: 1 job failed.
```

I'm not sure there is a way to prevent `go build` from scanning `.gitignore` files in the repo, so I'm not sure how to overcome this. Any ideas?

Many thanks!

Pete

P.S. I'm aware I could manage the .gitignore file myself, but as of go 1.17, it seems `go build` will scan all .gitignore files in the entire repo, and that then means I would have to manage the .gitignore file in all directories of my repo, which wouldn't be practical. So at the moment I'm stuck with go1.16 which only seems to only scans the .gitignore file of the directory the `go build` is issued in (or the go package(s) that are being built).

[Mike Shal]

On Thu, Jul 28, 2022 at 6:36 AM 'Peter Moore' via tup-users <tup-...@googlegroups.com> wrote:

I've hit a problem with building go executables when `.gitignore` tup directive is active.

Specifically, it seems `go build` scans repository `.gitignore` files and thus tup considers them as inputs that should be declared.

 

You can try using an output exclusion to ignore dependencies. It would look something like this:

$ cat Tupfile

.gitignore
: |> go build -o %o |> hello-world ^/.gitignore

(The ^ denotes a regex pattern to match against files. Any files read/written to that match the pattern are dropped and ignored).

I'm not familiar with building go programs in general. I'm guessing 'go build' is their own internal build system for creating a whole go program? That sort of situation (embedding a full build system run inside a tup rule) is unlikely to scale well, but for a small program you might not care.

While trying to test this locally I noticed an issue if your exclusion pattern matches the expected output file, so I suggest grabbing the latest if you hit an internal error about tup_entry_rm called when refcount=1.

Hope that helps!

-Mike

[Peter Moore]

On Sun, Jul 31, 2022 at 12:01 AM Mike Shal <mar...@gmail.com> wrote:

On Thu, Jul 28, 2022 at 6:36 AM 'Peter Moore' via tup-users <tup-...@googlegroups.com> wrote:

I've hit a problem with building go executables when `.gitignore` tup directive is active.

Specifically, it seems `go build` scans repository `.gitignore` files and thus tup considers them as inputs that should be declared.

 

You can try using an output exclusion to ignore dependencies. It would look something like this:

$ cat Tupfile

.gitignore
: |> go build -o %o |> hello-world ^/.gitignore

(The ^ denotes a regex pattern to match against files. Any files read/written to that match the pattern are dropped and ignored).

This worked perfectly, many thanks Mike! And apologies, this was clearly documented in the man page, I should have seen this.

 

I'm not familiar with building go programs in general. I'm guessing 'go build' is their own internal build system for creating a whole go program? That sort of situation (embedding a full build system run inside a tup rule) is unlikely to scale well, but for a small program you might not care.

Indeed, in my case these are simple go programs that are used by other parts of the build system, so scaling is not an issue. If you are curious, this is the project: https://github.com/spectrum4/spectrum4.

 

While trying to test this locally I noticed an issue if your exclusion pattern matches the expected output file, so I suggest grabbing the latest if you hit an internal error about tup_entry_rm called when refcount=1.

Thanks for the heads up!

 

Hope that helps!

Very much so! :-)

 

-Mike

--
--
tup-users mailing list
email: tup-...@googlegroups.com
unsubscribe: tup-users+...@googlegroups.com
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "tup-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tup-users/kEp_jZMqOfM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tup-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tup-users/CA%2B6x0LVooZe2GUAhdF4NpwgZqR%2BS5nce-_os6EXmv5eqsjpP6A%40mail.gmail.com.