outputs in different dirs / global bins / resource files

[Mike Shal]

Hi all,

I just pushed a branch on github to test out some new features. Please give it a try if you are interested in them and let me know how it works for you.

1) You can now specify outputs in other directories:

: foreach *.c |> gcc -c %f -o %o |> subdir/%B.o

If 'subdir' doesn't exist, it will automatically be created when needed, and automatically removed when all generated files are also removed from that directory (unless you manually create a file in that directory, in which case it becomes a "normal" directory). So you can use this to install files to a different location, for example.

However, you can't use a wildcard to match outputs coming from other directories. So if I had:

subdir/Tupfile:
: *.o |> link... |> myprog

This won't match the .o files coming from the other example. If you need to do something like that, you can use the second feature in this branch...

2) When you use a <group> as an input, you can use '%r' to be expanded at runtime to a resource file listing all of the files in the group. This is effectively the "global bin" support, as well as resource file support. For example:

Tuprules.tup:
MY_ROOT = $(TUP_CWD)
!cc = |> gcc -c %f -o %o |> %B.o | $(MY_ROOT)/<objs>

This makes the !cc rule put all objects into a group at the top-level of the project called <objs>. If we list <objs> as an input, then %r behaves like this:

: $(MY_ROOT)/<objs> |> gcc `cat %r` -o %o |> myprog

%r becomes a string like '../.tup/tmp/res-1', which tup creates on the fly to list all of the objects in <objs>, relative to the directory of the command being executed. In a Visual Studio environment you can use it as a resource file:

: $(MY_ROOT)/<objs> |> cl @%r /Fe%o |> myprog.exe

The groups with %r can be used with or without outputs in other directories, but it does help if you need to "wildcard" those files. The resource file might look like:

foo/foo.o
bar/bar.o

Or if used in a subdirectory, it would become:

../foo/foo.o
../bar/bar.o

I realize it is potentially confusing, but due to the way tup parses Tupfiles, it is difficult to find a way to have wildcards work with outputs in other directories that is guaranteed to be correct.

Note this branch has a different database version from the master branch, so you'll want to try it out in a sandbox. (Clone your project to test, and build the branch version of tup in a separate area). I haven't fully committed to this implementation yet, so please let me know if it works or not for your use-cases.

Thanks,
-Mike

[Andrew Wagner]

Sweeet!

So if I understand correctly, feature 1. will allow rules that spew
out a bunch of files in subdirectories, i.e. unzipping an archive, or
generating a bunch of code in subdirectories while metaprograming?

> --
> --
> tup-users mailing list
> email: tup-...@googlegroups.com
> unsubscribe: tup-users+...@googlegroups.com
> options: http://groups.google.com/group/tup-users?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "tup-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to tup-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Mike Shal]

On Wed, Jun 5, 2013 at 1:45 PM, Andrew Wagner <drew...@gmail.com> wrote:

Sweeet!

So if I understand correctly, feature 1. will allow rules that spew
out a bunch of files in subdirectories, i.e. unzipping an archive, or
generating a bunch of code in subdirectories while metaprograming?

Hi Andrew,

You still have to list the files as outputs, so unzipping an archive is probably going to be difficult to do. It's still probably going to be easier if you just unzip the file before putting it in your version control. This also makes things like 'git log' actually useful on their contents. Outputs-in-other-directories is probably most useful for installing things in a final directory.

I'm not quite sure what you mean by the second part?

-Mike

[Andrew Wagner]

We are doing some code generation using code we don't always control.
Often it wants to put stuff in subdirectories. Right now I use
symlink based hacks to make things work; using this feature sounds
much cleaner. Thanks!

[Mike Shal]

On Wed, Jun 5, 2013 at 10:55 AM, Mike Shal <mar...@gmail.com> wrote:

2) When you use a <group> as an input, you can use '%r' to be expanded at runtime to a resource file listing all of the files in the group. This is effectively the "global bin" support, as well as resource file support. For example:

Tuprules.tup:
MY_ROOT = $(TUP_CWD)
!cc = |> gcc -c %f -o %o |> %B.o | $(MY_ROOT)/<objs>

This makes the !cc rule put all objects into a group at the top-level of the project called <objs>. If we list <objs> as an input, then %r behaves like this:

: $(MY_ROOT)/<objs> |> gcc `cat %r` -o %o |> myprog

%r becomes a string like '../.tup/tmp/res-1', which tup creates on the fly to list all of the objects in <objs>, relative to the directory of the command being executed. In a Visual Studio environment you can use it as a resource file:

: $(MY_ROOT)/<objs> |> cl @%r /Fe%o |> myprog.exe

I changed the %r slightly, so that it works better if you have multiple groups as inputs. Instead of %r, it is now %<groupname> (including the angle-brackets, but without any path), so the above examples would now be:

: $(MY_ROOT)/<objs> |> gcc `cat %<objs>` -o %o |> myprog

and:

: $(MY_ROOT)/<objs> |> cl @%<objs> /Fe%o |> myprog.exe

Multiple resource files will be generated if you use multiple %<group> flags in the command-line.

-Mike