Lookup libraries linked to a target

13 views
Skip to first unread message

Grant

unread,
Mar 11, 2010, 5:49:29 PM3/11/10
to swtoolkit
I've added another function in component_builders.py to help
accommodate our workflow. We have a repository of external libraries
on Windows and Mac that we ship with our product (boost, wxWidgets,
etc.). As we don't have them installed in /usr/local/lib on our
machines, we need to copy them along side our binaries in scons-out/
build-type/staging, scons-out/build-type/tests, etc. The function
below does just that using the GatherInputs() method from
gather_inputs.py.

The problem, as noted in the documentation for GatherInputs(), is that
it can be rather slow on large projects. On Mac, we've gone from
about 2 seconds of scanning our sconscripts before building, to 15-20
seconds. Windows is even worse with scanning the sconscripts running
upwards of 1.5 minutes. Does anyone know of a better method of
accomplishing this task?


Here's the functions.


def ComponentProgram(env, prog_name, *args, **kwargs):
..... original function ....

# Set up deferred call to replicate resources
env.Defer(ComponentProgramDeferred)
env.Defer(CopyExternalLibraryDependenciesForProgram)

# Return the output nodes
return out_nodes


def CopyExternalLibraryDependenciesForProgram(env):
CopyExternalLibraryDependencies(env,
'COMPONENT_PROGRAM_RESOURCES')

def CopyExternalLibraryDependencies( env, envVar ):
if env.Bit('linux'):
# don't copy libs on linux
return 0

if not os.getenv('EXTERNAL_SDKS',None):
print "EXTERNAL_SDKS environment variable is not defined.
Please update and run the configure script from the External_SDKs
repo"
print "You may need to log out and log back in after running
the script"
SCons.Script.Exit(1)

prog_name = env['PROGRAM_BASENAME']

suffixes = []
for l in env['LIBSUFFIXES']:
suffixes.append(".*\\"+env.subst(l))

all_outputs = []
externalLibs = env.Flatten(env.GatherInputs(env.Alias(prog_name),
suffixes))

if env.Bit('windows'):
# On windows, the library dependency is listed as a .lib
file. We need to copy the corresponding dll file, though
externalLibs1 = [i.replace('.lib','.dll') for i in
externalLibs if i.startswith(os.getenv('EXTERNAL_SDKS')) and not
i.startswith(os.getenv(env.get('WX_ENV_VAR')))]
# Special case for our build of wxWidgets whose DLLs are all
postfixed with "_vcCedrus"
externalLibs2 = [i.replace('.lib','_vcCedrus.dll') for i in
externalLibs if i.startswith(os.getenv(env.get('WX_ENV_VAR')))]
externalLibs = externalLibs1
externalLibs.extend(externalLibs2)
else:
externalLibs = [i for i in externalLibs if
i.startswith(os.getenv('EXTERNAL_SDKS'))]

for i in externalLibs:
if os.path.islink(i):
externalLibs.append( os.path.realpath(i) )

env.Publish(prog_name,'run',externalLibs)

for resource, dest_dir in env.get(envVar).items():
all_outputs += env.ReplicatePublished(dest_dir, externalLibs,
resource)

env.Alias(prog_name, all_outputs)

Randall Spangler

unread,
Mar 15, 2010, 12:54:48 PM3/15/10
to swto...@googlegroups.com
Hi Grant,

Rather than scanning your programs for the libs they need at the end, publish all of the external SDK DLLs ahead of time.  Since your program depends on those via $LIBS, ReplicatePublished() on your program will also copy in the DLLs.

Here's a snippet of what we do on one of our mac projects; something similar should work on Windows.

thirdparty_libs = []
for dir in env.SubstList2('$THIRDPARTY_LIB_DIRS'):
  thirdparty_libs += env.Glob(dir + '/lib*.dylib')
  thirdparty_libs += env.Glob(dir + '/lib*.a')

import os
for lib in thirdparty_libs:
  name_parts = os.path.splitext(lib.name)
  if name_parts[1] == '.dylib':
    # TODO: Need to publish 'libfoo.dylib' or 'libfoo.a' as both
    # 'libfoo' and 'foo'.  Need to clean up sconscripts to remove 'lib' prefix
    # from all libs for mac, linux.
    lib_basename = name_parts[0]
    env.Publish(lib_basename, 'run', lib)
    env.Publish(lib_basename[3:], 'run', lib)

- Randall


--
You received this message because you are subscribed to the Google Groups "swtoolkit" group.
To post to this group, send email to swto...@googlegroups.com.
To unsubscribe from this group, send email to swtoolkit+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/swtoolkit?hl=en.


Grant

unread,
Mar 16, 2010, 1:04:57 PM3/16/10
to swtoolkit
Awesome. This is exactly what I was looking for. Thank you so much!

On Mar 15, 9:54 am, Randall Spangler <rspang...@google.com> wrote:
> Hi Grant,
>
> Rather than scanning your programs for the libs they need at the end,
> publish all of the external SDK DLLs ahead of time.  Since your program
> depends on those via $LIBS, ReplicatePublished() on your program will also
> copy in the DLLs.
>
> Here's a snippet of what we do on one of our mac projects; something similar
> should work on Windows.
>
> thirdparty_libs = []
> for dir in env.SubstList2('$THIRDPARTY_LIB_DIRS'):
>   thirdparty_libs += env.Glob(dir + '/lib*.dylib')
>   thirdparty_libs += env.Glob(dir + '/lib*.a')
>
> import os
> for lib in thirdparty_libs:
>   name_parts = os.path.splitext(lib.name)
>   if name_parts[1] == '.dylib':
>     # TODO: Need to publish 'libfoo.dylib' or 'libfoo.a' as both
>     # 'libfoo' and 'foo'.  Need to clean up sconscripts to remove 'lib'
> prefix
>     # from all libs for mac, linux.
>     lib_basename = name_parts[0]
>     env.Publish(lib_basename, 'run', lib)
>     env.Publish(lib_basename[3:], 'run', lib)
>
> - Randall
>

> > swtoolkit+...@googlegroups.com<swtoolkit%2Bunsu...@googlegroups.com>

Grant

unread,
Mar 31, 2010, 1:57:56 PM3/31/10
to swtoolkit
Another question for you on this topic.

Any tips for mac on running install_name_tool on a library after it's
been copied? (We've turned off the default hard linking behavior in
swtoolkit so that changes in install_name don't also get applied to
the original binary)

On Mar 15, 9:54 am, Randall Spangler <rspang...@google.com> wrote:

> Hi Grant,
>
> Rather than scanning your programs for the libs they need at the end,
> publish all of the external SDK DLLs ahead of time.  Since your program
> depends on those via $LIBS, ReplicatePublished() on your program will also
> copy in the DLLs.
>
> Here's a snippet of what we do on one of our mac projects; something similar
> should work on Windows.
>
> thirdparty_libs = []
> for dir in env.SubstList2('$THIRDPARTY_LIB_DIRS'):
>   thirdparty_libs += env.Glob(dir + '/lib*.dylib')
>   thirdparty_libs += env.Glob(dir + '/lib*.a')
>
> import os
> for lib in thirdparty_libs:
>   name_parts = os.path.splitext(lib.name)
>   if name_parts[1] == '.dylib':
>     # TODO: Need to publish 'libfoo.dylib' or 'libfoo.a' as both
>     # 'libfoo' and 'foo'.  Need to clean up sconscripts to remove 'lib'
> prefix
>     # from all libs for mac, linux.
>     lib_basename = name_parts[0]
>     env.Publish(lib_basename, 'run', lib)
>     env.Publish(lib_basename[3:], 'run', lib)
>
> - Randall
>

> > swtoolkit+...@googlegroups.com<swtoolkit%2Bunsu...@googlegroups.com>

Reply all
Reply to author
Forward
0 new messages