Self-Inclusion

65 views
Skip to first unread message

pmenso57

unread,
Jun 6, 2020, 5:24:34 PM6/6/20
to tup-users
I have a Tupfile that does the equivalent of this:

ifeq ($(VAR),)
    VAR = 123
    include Tupfile
else
#... stuff
endif

This works perfectly fine on Linux, but on Windows (running inside or outside of MSYS) I get "Tupfile: Broken pipe".  Any ideas?

Andrew Jensen

unread,
Jun 7, 2020, 3:22:26 PM6/7/20
to tup-users
Logically, this is equivalent to the following, without including itself:

ifeq ($(VAR),)
    VAR = 123
endif
#... stuff

pmenso57

unread,
Jun 7, 2020, 3:40:45 PM6/7/20
to tup-users
Sure, it is a minimal example that demonstrates the error.

Andrew Jensen

unread,
Jun 7, 2020, 6:33:39 PM6/7/20
to tup-users
My point is that restructuring your Tupfile could eliminate the need to recursively include itself.

I have not used tup on Windows, but I suspect the problem may be because Windows locks a file when it is open, and prevents is from being opened a second time. The recursive open thus fails. Linux has no such restriction, so the recursive open works fine.

pmenso57

unread,
Jun 8, 2020, 12:32:24 AM6/8/20
to tup-users
A file does not have to be locked when opened on Windows, but what I am actually doing something like:

ifeq ($(MODE),)
    MODE = common
    include Tupfile
    MODE = debug
    include Tupfile
    MODE = profile
    include Tupfile
    MODE = release
    include Tupfile
else
    ifeq ($(VARIANT),)
        VARIANT = common
        include Tupfile
        VARIANT = shared
        include Tupfile
        VARIANT = static
        include Tupfile
        VARIANT =
    else
        # lots of stuff based on $(MODE) and $(VARIANT)
    endif
endif

The real thing has more layers than this--including building with multiple compilers, cross-compiling, etc..  The "variants" mechanism does not work as (AFAICT) there is no way for them to share build artifacts (e.g. object files used in both static and shared linking variants).

Sure, the above can be refactored into a separate Tupfile for each layer, but I would rather not spread out the Cartesian product generation over four or five files.

Mike Shal

unread,
Jun 9, 2020, 1:01:04 AM6/9/20
to tup-...@googlegroups.com
Can you try http://gittup.org/tup/win32/tup-v0.7.8-88-gaba879aa.zip and see if it works now? As Andrew suspected, the Tupfile was kept open, and trying to re-open it failed on Windows. It's pretty straightforward to close it after reading it though, so this should be fixed.

You may want to look into using a Tupfile.lua for more complicated things like this, though. You could use nested for-loops to iterate through all combinations of mode & variant, which would be more straight-forward to understand.

Eg:
modes = {
        "common", "debug", "profile", "release"
}      
variants = {
        "common", "shared", "static"
}      
for i, mode in ipairs(modes) do
        for j, variant in ipairs(variants) do
                tup.rule("echo Using mode " .. mode .. " with variant " .. variant)
        end    
end

-Mike

pmenso57

unread,
Jun 9, 2020, 4:33:20 AM6/9/20
to tup-users
Thanks.  I will give it a shot tomorrow, as well as investigate using Lua.

Mike Shal

unread,
Jun 9, 2020, 12:58:23 PM6/9/20
to tup-...@googlegroups.com
I should mention, you can access global variables set by lua in regular tup-syntax files, and vice versa. So if you don't want to rewrite your "lots of stuff based on $(MODE) and $(VARIANT)" in lua, you can do something like this:

# Tupfile.lua:
modes = {
        "common", "debug", "profile", "release"
}
variants = {
        "common", "shared", "static"
}
for i, mode in ipairs(modes) do
        MODE = mode

        for j, variant in ipairs(variants) do
                VARIANT = variant
                tup.include("rules.tup")
        end
end

# rules.tup:
: |> echo Using mode $(MODE) with variant $(VARIANT) |>

(The local variables i, mode, j, and variant aren't accessible in rules.tup, but the global variables modes, variants, MODE, and VARIANT are).

You may also want to "tup refactor" to help verify that refactoring your Tupfiles into lua doesn't produce any unwanted changes if you are expecting it to generate the exact same command-lines.

-Mike

pmenso57

unread,
Jun 11, 2020, 12:56:03 PM6/11/20
to tup-users
Okay, I tested the executable provided above, and the recursive include worked perfectly.

With that said, it is no longer relevant as I have switched over to Tupfile.lua as per your suggestion.

Related to that, what constitutes "change" with respect to Lua?  I would initially think it would only consist of "exported" environment variable changes or changes to the rule resulting from tup.rule(...) (etc.).  However, when I change something innocuous in one of the Lua files (e.g. add a space at the end of a line of Lua code), I sometimes get some outputs rebuilt (but not all outputs rebuilt or even all outputs that are derived from the Lua function in question--just "some outputs").

-Paul
Reply all
Reply to author
Forward
0 new messages