Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Behavior of %d doesn't match manual

27 views
Skip to first unread message

Sean Russell

unread,
Nov 9, 2024, 9:49:45 AM11/9/24
to tup-users
Hi,

%d is giving me the top-level directory containing the Tupfile, not the parent of the basename file, when used in the output.

Per the manual in the %-flags section,

   The name of the lowest level of the directory. For example, in foo/bar/Tupfile, this would be the string "bar". 

However, I'm getting the parent directory that contains the Tupfile.

   cd ~
   mkdir -p tmp/tup/a/b
   cd tmp/tup
   tup init
   touch a/b/c.xyz
   echo ': a/b/c.xyz |> touch %o |> %d.txt' > Tupfile
   tup

for me produces:

   $ tup
   ...
   $ ls -1
   a
   Tupfile
   tup.txt

I was expecting b.txt.

What's my misunderstanding the %d flag?

Guillaume @layus Maudoux

unread,
Nov 9, 2024, 11:09:05 AM11/9/24
to tup-...@googlegroups.com
It is not really clear, but I think you should indeed read it as

The name of the lowest level of the directory [of the Tupfile this flag appears in.]

It is not related to the inputs of the rule, but the Tupfile where the rule is instantiated. See the manual again.

More generally, Tup does not accommodate well performing actions in other directories than the current Tupfile. I you want to create a/b/c.out, then the rule should reside in a/b/Tupfile. Not in the top-level Tupfile.

Written in a/b/Tupfile, your rule would produce b.txt as expected.

Overall I think you expect Tupfiles to be more generic than they are. If you want to generate rules programmatically, you may have to consider lua and other scripting capabilities of Tup.

Sean Russell

unread,
Nov 13, 2024, 11:39:51 AM11/13/24
to tup-users
On Saturday, November 9, 2024 at 4:09:05 PM UTC layu... wrote:
It is not really clear, but I think you should indeed read it as

The name of the lowest level of the directory [of the Tupfile this flag appears in.]

That clarifies it, thank you.
 
More generally, Tup does not accommodate well performing actions in other directories than the current Tupfile. I you want to create a/b/c.out, then the rule should reside in a/b/Tupfile. Not in the top-level Tupfile.

All of these, including the directory question, are trying to build assets used by top-level rules.

I'm trying to replace GNU Make, and I'm likely hitting a perspective wall. What I'm trying to do is to have a bunch of "styles" or flavors of sources, separated into subdirectories:

styles/style1/sourcefile.suffix
styles/style2/sourcefile.suffix

and use them to build assets in a different subdirectory. The build tools need to only see one subdirectory at a time, or they get confused. I was trying to remove duplication with something like:

foreach styles/*/input.suf | sourcefile |> somecommand --style %g %i |> tmpDir/%d-%B
foreach tmpDir/* |> finalprocessor %f %o |> outDir/%g.pdf

or using %g in both places on the first line, whatever works. Since globbing on directories isn't supported, and since %d doesn't work like I thought, right now I'm doing this by repeating rules for each style; 6 styles, 6 nearly duplicate rules, differing only by the input (and outputs, since that also has to be hard-coded). I actually have multiple input file types in each style, and each needs its own command, so with 6 styles and two templates (the style sources) in each, I am writing 12 rules which I was hoping to get down to 2.

What I'm understanding is that if I want to do something like this, I'll need to create a Tupfile in the styles/ directory, and one in each of the styles with a rule that looks like:

*.suf | ../../sourcefile |> somecommand --style %d %i |> ../../tmpDir/%d-%B

I'll duplicate each Tupfile to each subdirectory (since the rules are identical), and hope I keep them in sync... although, I suppose I could have a top-level config file, and each Tupfile would just import that top-level config file. The Tupfiles would still be duplicate, but I wouldn't worry about forgetting to copy changes across.

Then my top-level Tupfile would contain the same, original rule like:

foreach tmpDir/* |> finalprocessor %f %o |> outDir/%g.pdf

Is that right? Or is there an easier way to go about this that I'm missing?

If you want to generate rules programmatically, you may have to consider lua and other scripting capabilities of Tup.

I'm not. I only did that in my examples for reproducability by readers. The Tupfile is hand-coded, and more complex than my minimal examples.

Guillaume @layus Maudoux

unread,
Nov 13, 2024, 1:55:04 PM11/13/24
to Sean Russell, tup-users
The example around %d in the manual is based on a macro. Using macros seems the way to go for you. While the Tupfile will be duplicated, it will be very short. Just a single macro invocation. The macro has to be defined in Tuprules.tup, and is defined only once. I think %d does not make much sense outside of a macro.

Sean Russell

unread,
Nov 13, 2024, 5:42:08 PM11/13/24
to tup-users
Thanks! I'd looked at macros earlier, but ran into a different issue I don't understand.  Per the manual, both inputs and outputs should be able to be specified in a macro:

!macro = [inputs] | [order-only inputs] |> command |> [outputs]

So, I'm thinking, at the top level in my Tuprules.tup:

!mmac = input.suf | ../../ooinp.suf |> compilecommand --style %d %i |> ../../tmp/%d-%B

and in my project/styles/style1/Tupfile:

include_rules
: |> !mmac |>

(because without the pipes bracketing the rule, tup complains about missing pipes) at which point tup tells me that "!-macros can't have normal inputs, only order-only inputs."

I also tried putting the input in the rule, and the order-only input in the macro, but even with various combinations of pipe placement in both the macro and the rule I never got anything other than the error message about normal inputs.

Then I tried putting all of the inputs in the rule, and only the outfile in the Tuprules.tup, which is explicitly allowed in the manual:

You will only want to specify the output parameter in either the !-macro or the :-rule that uses it

!mmac = |> compilecommand --style %d %i |> ../../tmp/%d-%B

and in the deep Tupfile:

include_rules
: input.suf | ../../ooinp.suf |> !mmac |>

which gives me a "Missing second '|>' marker" error; adding one to the end of the macro, so that everything (except the start of the rule) is now bracketed, still generates the missing second marker error. However, if I put both all inputs and outputs in the rule, it runs -- sort of. But now in complains that the top-level Tupfile rule that uses the output of the deep Tupfile is "Unable to use inputs from a generated directory that isn't written by this Tupfile."  Here's where I got stuck (I've trimmed it down to a bare minimum needed to generate the error):

./Tupfile:

: foreach tmp/* |> cmd1 %f %o |> out/%B-%t.pdf

./Tuprules.tup:

!mmac = |> cmd2 %f %o |>

./styles/style/Tupfile:

include_rules
: input.ext | ../../ooinput.ext |> !mmac |> ../../tmp/%d-%B

I get the impression that macros can only be used in the commands. If that's not correct, could you provide a minimal example of providing order-only inputs or outputs in a macro that demonstrates my error? And is there a way of getting around this directory error? Another rule in the top-level Tupfile to explicitly create temporary directories?

Thank you.

Guillaume (@layus) Maudoux

unread,
Nov 13, 2024, 7:46:56 PM11/13/24
to tup-...@googlegroups.com, Sean Russell

At this point it gets difficult to help without seeing the code, but let me try

On 13/11/24 23:42, Sean Russell wrote:
Thanks! I'd looked at macros earlier, but ran into a different issue I don't understand.  Per the manual, both inputs and outputs should be able to be specified in a macro:

!macro = [inputs] | [order-only inputs] |> command |> [outputs]

So, I'm thinking, at the top level in my Tuprules.tup:

!mmac = input.suf | ../../ooinp.suf |> compilecommand --style %d %i |> ../../tmp/%d-%B

I think your inputs in a rule should be relative to where the rule is used. Is that what this rule does ?

I am sure however that you are not supposed to output in another folder than the one where your Tupfile is.
A core assumption of Tup is that the rule producing a/b/c/foo.bar will be in a/b/c/Tupfile.

So ... |> ../../tmp/%d-%B will always be wrong.

From this, all the examples below are doomed to fail.

and in my project/styles/style1/Tupfile:

include_rules
: |> !mmac |>

(because without the pipes bracketing the rule, tup complains about missing pipes) at which point tup tells me that "!-macros can't have normal inputs, only order-only inputs."

I also tried putting the input in the rule, and the order-only input in the macro, but even with various combinations of pipe placement in both the macro and the rule I never got anything other than the error message about normal inputs.

Then I tried putting all of the inputs in the rule, and only the outfile in the Tuprules.tup, which is explicitly allowed in the manual:

You will only want to specify the output parameter in either the !-macro or the :-rule that uses it

!mmac = |> compilecommand --style %d %i |> ../../tmp/%d-%B

and in the deep Tupfile:

include_rules
: input.suf | ../../ooinp.suf |> !mmac |>

which gives me a "Missing second '|>' marker" error; adding one to the end of the macro, so that everything (except the start of the rule) is now bracketed, still generates the missing second marker error. However, if I put both all inputs and outputs in the rule, it runs -- sort of. But now in complains that the top-level Tupfile rule that uses the output of the deep Tupfile is "Unable to use inputs from a generated directory that isn't written by this Tupfile."  Here's where I got stuck (I've trimmed it down to a bare minimum needed to generate the error):

./Tupfile:

: foreach tmp/* |> cmd1 %f %o |> out/%B-%t.pdf

./Tuprules.tup:

!mmac = |> cmd2 %f %o |>

./styles/style/Tupfile:

include_rules
: input.ext | ../../ooinput.ext |> !mmac |> ../../tmp/%d-%B

I get the impression that macros can only be used in the commands. If that's not correct, could you provide a minimal example of providing order-only inputs or outputs in a macro that demonstrates my error? And is there a way of getting around this directory error? Another rule in the top-level Tupfile to explicitly create temporary directories?

Thank you.
--
--
tup-users mailing list
email: tup-...@googlegroups.com
unsubscribe: tup-users+...@googlegroups.com
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to the Google Groups "tup-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tup-users+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/tup-users/ba8db968-f957-40b0-bf38-c79f464e3a3fn%40googlegroups.com.

Guillaume (@layus) Maudoux

unread,
Nov 13, 2024, 7:49:22 PM11/13/24
to tup-...@googlegroups.com, Sean Russell

Indeed, this is the first line of the Tupfiles section in the manual.

Not sure there is a good place for this really, but maybe under rules would help.

TUPFILES

You must create a file called "Tupfile" anywhere in the tup hierarchy that you want to create an output file based on the input files. The input files can be anywhere else in the tup hierarchy, but the output file(s) must be written in the same directory as the Tupfile.

On 13/11/24 23:42, Sean Russell wrote:

Sean Russell

unread,
Nov 16, 2024, 12:06:54 PM11/16/24
to tup-users
Ok. I understand. Thanks for all of the answers!
Reply all
Reply to author
Forward
0 new messages