Any way to prepend/append a string to each item of a list variable?

[Eric Boren]

I have something like this:

'variables': {

  'my_awesome_list': [

    'a',

    'b',

    'c',

    'd',

  ],

},

And I want to use it elsewhere like this:

......

'dependencies': [

  'somefile.gyp:<@(my_awesome_list)',

],

......

Which does not work.  I've tried doing this with a list comprehension:

......

'dependencies': ['somefile.gyp:%s' % item for item in '<@(my_awesome_list)'.split(' ')],

......

But that evaluates to:

'dependencies': [

  'somefile.gyp:a b c d',

],

If I change it to:

......

'dependencies': ['%s' % item for item in '<@(my_awesome_list)'.split(' ')],

......

It evaluates to:

'dependencies': [

  'a',

  'b',

  'c',

  'd',

],

As I would expect.  Why does the above case not work?  Is there another way to do this?

Thanks,

Eric

[Marc-Antoine Ruel]

Create a meta-target in somefile.gyp that depends on a, b, c, d with 'type': 'none'.

Then in the other file depend only on the meta-target.

M-A

2012/10/5 Eric Boren <bor...@google.com>

--
 
 
 

[Ryan Sleevi]

A perhaps off-topic diversion, but are the Pythonic idioms from Eric's
example meant to be supported in canonical GYP syntax? I had thought
that their ability to be used was more of a "bug" than a "feature",
and may be lost at any time, but I wasn't sure if that was a shared
feeling.

> --
>
>
>

[Mark Mentovai]

It is a bug. It shouldn’t work. It might trip up gyp --check, and even if it doesn’t, it might just stop working in the future.

[Eric Boren]

Okay.  Then is there a way to do what I'm talking about?  Basically, I have a meta-target which has several dependencies in separate files.  In my top-level "all" target, I want to list those same targets (but with a different path, since that file is in the parent directory), in addition to the meta-target, so that I can build them all at once with the meta-target OR build them independently.  Right now I have to copy the same list around several times with slightly different prefixes.  It seems like using a list variable would be the most elegant way to handle it.

[Ryan Sleevi]

Like Marc-Antoine suggested, a 'none' target type is a great way to
bundle up dependencies, and is used to great success (eg: in
Chromium).

I believe the full syntax is
{
'target_name': 'meta',
'type': 'none',
'dependencies': [
'a.gyp:a',
'b.gyp:b',
'c.gyp:c',
],
'export_dependent_settings': [
'a.gyp:a',
'b.gyp:b',
'c.gyp:c',
],
}

Have I misunderstood what you're trying to accomplish here?

[Eric Boren]

This is particularly problematic, because my_awesome_list is constructed using some conditionals which I'd rather not have to repeat.

On Fri, Oct 5, 2012 at 3:29 PM, Eric Boren <bor...@google.com> wrote:

Okay, that works for this case.  The export_dependent_settings did the trick.  I still have a situation where the list-of-string manipulation would help a great deal. I want to use a list of target names for both the dependency list and for the list of inputs to an action.  So, given the same my_awesome_list, something equivalent to:

...

'dependencies': [

  '<(item).gyp:<(item)' for item in '<@(my_awesome_list)',

],

'actions': [{

  ...

  'inputs': [

    '<(PRODUCT_DIR)/lib<(item).a' for item in '<@my_awesome_list)',

  ],

}],

[Ryan Sleevi]

On Fri, Oct 5, 2012 at 12:29 PM, Eric Boren <bor...@google.com> wrote:
> Okay, that works for this case. The export_dependent_settings did the
> trick. I still have a situation where the list-of-string manipulation would
> help a great deal. I want to use a list of target names for both the
> dependency list and for the list of inputs to an action. So, given the same
> my_awesome_list, something equivalent to:
>
> ...
> 'dependencies': [
> '<(item).gyp:<(item)' for item in '<@(my_awesome_list)',
> ],
>
> 'actions': [{
> ...
> 'inputs': [
> '<(PRODUCT_DIR)/lib<(item).a' for item in '<@my_awesome_list)',
> ],
> }],

Maybe it helps to think about this differently.

What you want is a list of the link-time library dependencies for a
target, so that you can use them in an action, right? Do you want the
explicit list (eg: only the list from 'dependencies') or the
transitive list (eg: all of your dependencies, plus all of their
dependencies, etc)?
Do they need to match a particular order? (note: GYP generators don't
currently provide any ordering guarantees)

I think right now, you're looking at how to solve a particular
problem, but I think it might help to understand the problem you're
trying to solve, to see if it might not be solved in a different
manner.

> --
>
>
>

[Eric Boren]

Okay, I'll zoom all the way out.  We want to be able to create a single static library for the "core" Skia functionality, plus a few accessory static libs.  For our core library, however, we have one or two sets of source files which require a different set of cflags from the rest of the targets, so we can't just glob all of the sources into one target.  So, we end up with multiple static libraries we want linked together.  AR doesn't really do this, so I've written a little script to merge multiple static libs into one.  I'm doing this using an action to launch the script.

Essentially, there were two related problems I was trying to solve.  The first was to have a meta-target for all of the static libraries (core + accessories) so that we can run "make static_libs" but also be able to run "make some_accessory_lib".  That required propagating the dependencies from the static_libs meta-target to the "all" target, which I was attempting to do by creating a list of the static library targets and manipulating it to get the paths to be correct.  This is solved with direct_dependent_settings.

The second problem is to link multiple static libraries into one "core" library.  Right now I list the component parts in 'dependencies' and then list their outputs as inputs to the action which calls my merge script.  Again, I wanted to do this by manipulating a list, so that I could derive the target dependencies and action inputs from the same place.  It seems like you're suggesting that there's an easier way to pipe the output filepaths from the dependencies to the action inputs?

[Ryan Sleevi]

On Fri, Oct 5, 2012 at 12:46 PM, Eric Boren <bor...@google.com> wrote:
> Okay, I'll zoom all the way out. We want to be able to create a single
> static library for the "core" Skia functionality, plus a few accessory
> static libs. For our core library, however, we have one or two sets of
> source files which require a different set of cflags from the rest of the
> targets, so we can't just glob all of the sources into one target. So, we
> end up with multiple static libraries we want linked together. AR doesn't
> really do this, so I've written a little script to merge multiple static
> libs into one. I'm doing this using an action to launch the script.
>
> Essentially, there were two related problems I was trying to solve. The
> first was to have a meta-target for all of the static libraries (core +
> accessories) so that we can run "make static_libs" but also be able to run
> "make some_accessory_lib". That required propagating the dependencies from
> the static_libs meta-target to the "all" target, which I was attempting to
> do by creating a list of the static library targets and manipulating it to
> get the paths to be correct. This is solved with direct_dependent_settings.

The dependencies between your meta-target 'all' (which I presume is of
type: none) and your meta-target 'static_libs' (which I also presume
is of type: none) should already be propagating. The only reason for
'direct_dependent_settings' + 'export_dependent_settings' is you
expect other consumers to be depending on skia.gyp:all for
linking/compile information, which is generally a very uncommon use of
a meta-target.

That said, it does sound like your solution was solved.

>
> The second problem is to link multiple static libraries into one "core"
> library. Right now I list the component parts in 'dependencies' and then
> list their outputs as inputs to the action which calls my merge script.
> Again, I wanted to do this by manipulating a list, so that I could derive
> the target dependencies and action inputs from the same place. It seems
> like you're suggesting that there's an easier way to pipe the output
> filepaths from the dependencies to the action inputs?

So, teasing out the feature, the 'ideal' world would have creating a
single ("fat") static library from multiple input thin archives would
be a fine solution, right? Something essentially akin to the
"loadable_module" target type, but in a way that dependents can still
statically link to.

Such a feature would generally only be preferable when dealing with
'dist' type packaging - that is, where skia.gyp can be used to create
a re-distributable static library. In the case of say, using Chromium
with skia, it would actually be preferable *not* to use such a 'fat'
archive, and instead use 'thin' archives (quicker link times with less
RAM)

Perhaps that's something worth exploring separately.

Under current functionality of GYP, you can *probably* do some
creative cheating here. I suspect the example I'm about to provide
won't work, but hopefully it'll point you to the right functionality.

{
'target_name': 'example',
'type: 'static_library',
'variables': {
'my_dependencies': [

'a.gyp:a',
'b.gyp:b',
'c.gyp:c',
],

},
'dependencies': [
'<@(my_dependencies)',
],
'actions': [
{
'action_name': 'test',
'variables': {
'dependency_file': '<|(dependencies.txt <@(my_dependencies))',
},
'inputs': [ ... ],
'action': [
'myscript.py',
'<(PRODUCT_DIR)',
'<(dependency_file)'
],
},
],
}

That will invoke myscript.py with the path to the PRODUCT_DIR and the
path to a text file, whose contents will be something like

[path_relative_to_current_gyp_file_dir]/a.gyp:a
[path_relative_to_current_gyp_file_dir]/b.gyp:b
[path_relative_to_current_gyp_file_dir]/c.gyp:c

Which you can use to then normpath, cut, etc.

Altogether, it's a giant hack, but I think it should do what you want.
It just means you have to move your processing of a.gyp:a from GYP
(where, as Mark said, it's "not supported") into your helper script
(which you then end up coding against how GYP behaves, which is not
ideal)

[Eric Boren]

Okay, that works for this case.  The export_dependent_settings did the trick.  I still have a situation where the list-of-string manipulation would help a great deal. I want to use a list of target names for both the dependency list and for the list of inputs to an action.  So, given the same my_awesome_list, something equivalent to:

...

'dependencies': [

  '<(item).gyp:<(item)' for item in '<@(my_awesome_list)',

],

'actions': [{

  ...

  'inputs': [

    '<(PRODUCT_DIR)/lib<(item).a' for item in '<@my_awesome_list)',

  ],

}],

[Eric Boren]

I was mistaken about this; they do propagate without using any export_dependent_settings, and I'm not sure why I thought they didn't.  And yes, both meta-targets are of type "none."

 

>
> The second problem is to link multiple static libraries into one "core"
> library.  Right now I list the component parts in 'dependencies' and then
> list their outputs as inputs to the action which calls my merge script.
> Again, I wanted to do this by manipulating a list, so that I could derive
> the target dependencies and action inputs from the same place.  It seems
> like you're suggesting that there's an easier way to pipe the output
> filepaths from the dependencies to the action inputs?

So, teasing out the feature, the 'ideal' world would have creating a
single ("fat") static library from multiple input thin archives would
be a fine solution, right? Something essentially akin to the
"loadable_module" target type, but in a way that dependents can still
statically link to.

Exactly.

 

Such a feature would generally only be preferable when dealing with
'dist' type packaging - that is, where skia.gyp can be used to create
a re-distributable static library. In the case of say, using Chromium
with skia, it would actually be preferable *not* to use such a 'fat'
archive, and instead use 'thin' archives (quicker link times with less
RAM)

Yep.  Chromium uses a separate set of gyp files from ours, which just share a .gypi or two, so it's okay if our target types are slightly different. 

Yikes.  I think I'd rather just copy the list a couple of times and run my script to merge the libraries.