groups

[Mike Shal]

Hey all,

I just pushed a new branch called 'groups' which can help writing
Tupfiles where commands should run in stages. For example, you may
want to build all auto-generated header files before compiling
anything, but it could be cumbersome/slow to declare the headers as
inputs to each compilation command. Instead of writing a Tupfile like
this:

: |> generate header... |> foo.h
: |> generate header... |> bar.h
... lots more headers generated
: foreach *.c | foo.h bar.h ... |> gcc -c ... |>

You can now do something like this:

: |> generate header... |> foo.h <myheaders>
: |> generate header... |> bar.h <myheaders>
...
: foreach *.c | <myheaders> |> gcc -c ... |>

Anything that lists <myheaders> as an output becomes fair game for use
by those commands that list <myheaders> as an input. The group is
actually inserted as a node in the graph in whatever directory you
specify, so <myheaders> in a/Tupfile is different from <myheaders> in
b/Tupfile. You could use ../b/<myheaders> to refer to a group in
another directory. It may be easiest to put any groups in the
top-level of your project using a Tuprules.tup file:

myproject/Tuprules.tup:
MYPROJECT_ROOT = $(TUP_CWD)
!generate_header = |> python blah... |> %B.h $(MYPROJECT_ROOT)/<autoheaders>
!cc = | $(MYPROJECT_ROOT)/<autoheaders> |> gcc -c %f -o %o |> %B.o

Then in any sub Tupfile you can do:
: foreach *.h.in |> !generate_header |>
: foreach *.c |> !cc |>

And the cc commands can use the generated headers without tup complaining.

Note that it is trivial to add circular dependencies by using groups,
so don't do that. You could also invert commands in a Tupfile by using
a group, but that is not the recommended use case.

Another example where it might help is with vala/vapi files (Simon or
Tal or other vala users, can you try it out and see if it helps?) -

myproject/Tuprules.tup:
MYPROJ_ROOT = $(TUP_CWD)
!fast-vapi = |> valac-0.16 --fast-vapi=%B.vapi %f |> %B.vapi
$(MYPROJ_ROOT)/<vapi>
!valac = | $(MYPROJ_ROOT)/<vapi> |> valac-0.16 $(VAPISPARAMS)
$(VALAPKGS) %f -C |> %B.c

This way you don't have to list *.vapi as inputs to the valac command.
However, you still need some way to construct the correct
--use-fast-vapi arguments to pass to valac, and the grouping feature
doesn't help with that.

Any feedback would be appreciated!

Thanks,
-Mike

[Freddie Chopin]

Hi!

Is there any way groups could be used as actual input to commands? Sth like when in a project with lots of subdirs with sources each subdir adds object files to top-folder's group and than this group is used to get objects for linking?

!cc = |> gcc $(C_FLAGS) -c %f -o %o |> %B.o $(TOPDIR)/<objects> 

...

: $(TOPDIR)/<objects>  |> !link |> $(PROJECT).elf

Something like this fails because "./<objects>" string (instead of actual objects) is inserted directly into the command, so I guess it's not the intended usage for this feature... It would be great if you woundn't have to actually list all objects directly (or even with wildcards and subdirectories) and groups seemed a very good option to just keep all these files listed.

4\/3!!

[Rendaw]

I'm not sure of my Tup history, but the only mention I see of "groups"
in the manual is "bins", which look like they have more or less the
same usage... that might be why you're having errors. Also, all
outputs have to be in the same directory as the Tupfile, so the bin
synax should probably be something like:
!cc = |> gcc $(C_FLAGS) -c %f -o %o |> %B.o {objects}
: foreach {objects} |> !link |> $(PROJECT).elf

I'm not sure how this works with objects generated in different
directories. I used the {bin} syntax to generate a "thin archive"
with ar to store the list of filenames, then used that archive in
other directories, although I had some ar issues on Windows with MinGW.

[Freddie Chopin]

W dniu piątek, 30 listopada 2012 07:36:05 UTC+1 użytkownik Rendaw napisał:

I'm not sure of my Tup history, but the only mention I see of "groups"
in the manual is "bins", which look like they have more or less the
same usage... that might be why you're having errors.

True, this feature is not documented in the manual, but it's there! You can check test cases - there are like 10 covering groups, most of them are related to autoheaders.

 

 Also, all
outputs have to be in the same directory as the Tupfile, so the bin
synax should probably be something like:
!cc = |> gcc $(C_FLAGS) -c %f -o %o |> %B.o {objects}
: foreach {objects}  |> !link |> $(PROJECT).elf

Well, the files are in the same dir as Tupfile, I'm only looking for some nice way to get the list of all objects without typing them explicitly (or like that: sub1/*.o sub2/*.o sub3/sub3A/*.o ...) and group seemed like a nice idea...

 

4\/3!!

[Rendaw]

On Nov 30, 4:06 pm, Freddie Chopin <freddie.cho...@gmail.com> wrote:
> W dniu piątek, 30 listopada 2012 07:36:05 UTC+1 użytkownik Rendaw napisał:
>
>
>
> > I'm not sure of my Tup history, but the only mention I see of "groups"
> > in the manual is "bins", which look like they have more or less the
> > same usage... that might be why you're having errors.
>
> True, this feature is not documented in the manual, but it's there! You can
> check test cases - there are like 10 covering groups, most of them are
> related to autoheaders.
>

Wow, well I guess that means I deleted them from my fork without any
idea of what they do! I had no idea. Sorry, I guess I can't help
here.

[Mike Shal]

(Going through some old emails...)

In case you haven't seen it, you can use %<groupname> in the command-line and it will be expanded properly now at run-time. I realize this is an old thread, but just in case anyone else comes across it...

-Mike