Linking Shared Library from Static Libraries

[Eric Boren]

Gypmasters,

Skia is interested in a shared library build.  Ideally, we'd like a single shared library target which depends on a set of static libraries to contain all of the symbols from its dependencies.  On Linux (Make) this seems to work, but it doesn't work on Mac.  On Mac, nothing is built unless I include a dummy source file, and then none of the symbols from the dependencies are included.  Is there an easy way to do this?  If not, it seems like broken behavior to me.  Using gypi's for all sources will not work, since we need separate targets to define cflags on different files.

Thanks,

Eric

[Mark Mentovai]

1. Not Mac-specific, not Xcode-specific, nothing to do with GYP: linkers don’t pull members (.o files) out of static libraries (.a files) specified to the linker with -l unless they’re needed to satisfy an unresolved reference. If your target doesn’t have any code of its own, it has no unresolved references, and thus there’s no need to bring anything in from any static library. There are, however, linker options to change this behavior (-all_load on Mac).

2. Xcode-specific, nothing to do with GYP: Xcode requires at least one source file (it can be empty) in a target in order to produce any output at all. Without this, it doesn’t even know what language runtimes (C++ or Objective-C) are required. This is something between an Xcode limitation and an Xcode bug.

You found a solution to (2) that gets Xcode to produce output (and it’s in fact the canonical solution, we use it in Chrome), but you’re still up against (1).

--
 
---
You received this message because you are subscribed to the Google Groups "gyp-developer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gyp-develope...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Eric Boren]

Here's a CL I've been hacking on: https://codereview.chromium.org/14582008/

Mark, for #1, if I can find the right linker options, I should be able to get this working?  According to: http://stackoverflow.com/questions/16082470/osx-how-do-i-convert-a-static-library-to-a-dynamic-one it looks like I need to provide the flag for each static library I'm linking.  I'm not really sure how to express that in gyp terms.

[Ami Fischman]

Mark: isn't your #1 why the ninja & make generators use --whole-archive?  Does Xcode not have a way to say the same?

[Mark Mentovai]

stackoverflow is the worst…

-all_load or -noall_load is effective for the entire linker invocation, it’s not something that can be turned on and off one library at a time like --whole-archive and --no-whole-archive.

The Mac linker also has a -force_load option to load all members (.o files) from a single archive (static library) without turning on the -all_load option on for the entire linker invocation.

You can give arbitrary options to the Mac linker from GYP with

  'xcode_settings': {

    'OTHER_LDFLAGS': [

      '-Wl,-your_linker_option_here',

     ],

  },

As an alternative, if there’s an easy way to reference all of the symbols you need to export from your shared library, you can do that.

[Mark Mentovai]

make started using --whole-archive for a different reason: to overcome circular dependencies among static libraries (which may no longer be a problem in Chrome). The GNU linker, when told “-lfoo -lbar”, will look in libbar.a to satisfy unresolved references in libfoo.a, but will not look in libfoo.a to satisfy unresolved references in libbar.a. Using --whole-archive brings all members (.o files) of all archives (static libraries) in without asking whether they’re needed to satisfy unresolved references. The Mac linker doesn’t behave this way: it will look in libfoo.a to satisfy libbar.a’s unresolved references even though -lfoo did not appear after -lbar.

The ninja GYP generator inherited this behavior from the make one. Chrome’s dependency relationships are generally in much better shape now than when we first switched to GYP, but Mac is still the only platform where we enforce no circular dependencies between any targets. It would be nice to enforce this for other platforms, too. This is http://crbug.com/35878. If that were fixed, it would actually be possible to eliminate --whole-archive.

--whole-archive isn’t “free,” it results in some behavior changes. For example, if one of the .o members has static initializers but that .o does not otherwise satisfy anything else’s unresolved references, the static initializer will not show up in final linked output without --whole-archive, but it will show up with it.

The Mac equivalent to --whole-archive is -all_load. It has different semantics. Unlike --whole-archive, it’s effective for the entire linker invocation. --whole-archive can be turned on and off as you progress through the command line, so “--whole-archive -lfoo --no-whole-archive -lbar” will load all of libfoo.a but will only load members (.o files) from libbar.a to satisfy unresolved references. On the Mac, “-lfoo -lbar -all_load” will load all of both libfoo.a and libbar.a. There is an alternative, -force_load, which lets you specify a single archive (static library) from which to load all members (.o files).

[Sam Clegg]

On Fri, May 3, 2013 at 9:02 AM, Mark Mentovai <ma...@chromium.org> wrote:
> make started using --whole-archive for a different reason: to overcome
> circular dependencies among static libraries (which may no longer be a
> problem in Chrome). The GNU linker, when told “-lfoo -lbar”, will look in
> libbar.a to satisfy unresolved references in libfoo.a, but will not look in
> libfoo.a to satisfy unresolved references in libbar.a. Using --whole-archive
> brings all members (.o files) of all archives (static libraries) in without
> asking whether they’re needed to satisfy unresolved references. The Mac
> linker doesn’t behave this way: it will look in libfoo.a to satisfy
> libbar.a’s unresolved references even though -lfoo did not appear after
> -lbar.

Isn't --start-group/--end-group a better solution that problem?

[Evan Martin]

On Fri, May 3, 2013 at 10:02 AM, Sam Clegg <s...@google.com> wrote:

On Fri, May 3, 2013 at 9:02 AM, Mark Mentovai <ma...@chromium.org> wrote:
> make started using --whole-archive for a different reason: to overcome
> circular dependencies among static libraries (which may no longer be a
> problem in Chrome). The GNU linker, when told “-lfoo -lbar”, will look in
> libbar.a to satisfy unresolved references in libfoo.a, but will not look in
> libfoo.a to satisfy unresolved references in libbar.a. Using --whole-archive
> brings all members (.o files) of all archives (static libraries) in without
> asking whether they’re needed to satisfy unresolved references. The Mac
> linker doesn’t behave this way: it will look in libfoo.a to satisfy
> libbar.a’s unresolved references even though -lfoo did not appear after
> -lbar.

Isn't --start-group/--end-group a better solution that problem?

I am sorry, this mess is all my fault.

In the early early days, Chrome had a hack where every "library" target used a variable ("<(library)") instead of specifying "static_library".

This was an attempt at letting you globally switch all static libraries into shared libraries for linking speed.  I seem to recall I added whole-archive to make that work.

See the comment here:

https://code.google.com/p/chromium/codesearch#chromium/src/Makefile&q=whole-archive&sq=package:chromium&type=cs&l=170

Note that due to arguably a bug, you can't use both start-group and whole-archive at the same time, but it's fine because (as Mark mentions) whole-archive effectively solves the problem start-group is used to address.

As for what to do on Mac, I guess it's up to Mark's sense of aesthetics as to whether "refactoring" code into a static_library ought to make a dependent shared_library work (which is where the Linux behavior came from), or whether it's up to the .gyp author to adjust linker flags to do that.

[Dirk Pranke]

I believe there is a way to cause the linker to generate a shared library from just an archive on mac, win, and linux, but each involves hacks. The most notable downside is that you don't get the benefit of dead code stripping that you do with the normal pull-symbols-in-from-an-archive approach, especially if you're trying to build a single shared library out of multiple static libs (as opposed to a single static lib). This can lead to really weird issues down the road, and is the main reason why we don't do this all over the place (see, e.g., the content libraries in chromium, where we jump through gyp hoops to pull the files we need directly instead of just using a set of libraries).

[Eric Boren]

I've added the following to my shared library target:

  'xcode_settings': {

    'OTHER_LDFLAGS': [

      '-Wl,-all_load',

     ],

  },

The command line looks right, but I'm getting a bunch of "undefined symbols for arch i386".   I wonder if this is some problem with circular dependencies and library ordering?

 Ld ../xcodebuild/Debug/libskia.dylib normal i386

    cd /usr/local/google/skia/trunk/gyp

    setenv MACOSX_DEPLOYMENT_TARGET 10.7

    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -arch i386 -dynamiclib -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk -L/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks -F/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug -filelist /usr/local/google/skia/trunk/gyp/../xcodebuild/skia_base_libs.build/Debug/skia_base_libs.build/Objects-normal/i386/skia.LinkFileList -install_name /usr/local/lib/libskia.dylib -mmacosx-version-min=10.7 -Wl,-all_load -dynamiclib /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_core.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_opts.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_ports.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_utils.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_opts_ssse3.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_gr.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_skgr.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_sfnt.a /usr/local/google/skia/trunk/xcodebuild/Debug/libcityhash.a -framework ApplicationServices -framework AGL -framework OpenGL -single_module -compatibility_version 1 -current_version 1 -o /usr/local/google/skia/trunk/gyp/../xcodebuild/Debug/libskia.dylib

Undefined symbols for architecture i386:

  "CreateARGBImageEncoder()", referenced from:

      SkBitmapHasher::ComputeDigestInternal(SkBitmap const&, unsigned long long*) in libskia_utils.a(SkBitmapHasher.o)

  "SkColorFilter::InitializeFlattenables()", referenced from:

      SkFlattenable::InitializeFlattenables() in libskia_ports.a(SkGlobalInitialization_default.o)

..........

  "SkImages::InitializeFlattenables()", referenced from:

      SkFlattenable::InitializeFlattenables() in libskia_ports.a(SkGlobalInitialization_default.o)

  "vtable for SkErodeImageFilter", referenced from:

      SkErodeImageFilter::SkErodeImageFilter(SkFlattenableReadBuffer&) in libskia_ports.a(SkGlobalInitialization_default.o)

  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.

  "vtable for SkDilateImageFilter", referenced from:

      SkDilateImageFilter::SkDilateImageFilter(SkFlattenableReadBuffer&) in libskia_ports.a(SkGlobalInitialization_default.o)

  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.

  "vtable for SkStippleMaskFilter", referenced from:

      SkStippleMaskFilter::SkStippleMaskFilter(SkFlattenableReadBuffer&) in libskia_ports.a(SkGlobalInitialization_default.o)

  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.

ld: symbol(s) not found for architecture i386

clang: error: linker command failed with exit code 1 (use -v to see invocation)

[Mark Mentovai]

Take one of the undefined symbols, for example

  "CreateARGBImageEncoder()", referenced from:

      SkBitmapHasher::ComputeDigestInternal(SkBitmap const&, unsigned long long*) in libskia_utils.a(SkBitmapHasher.o)

Who’s supposed to provide that symbol? What implementation file ⇒ what object file ⇒ what static library?

[Eric Boren]

That particular symbol should be defined in SkBitmapHasher.o which is in libskia_utils.a:

$ nm xcodebuild/Debug/libskia_utils.a | grep CreateARGBImageEncoder

         U __Z22CreateARGBImageEncoderv

Most of the others are in libskia_ports.a and they show up when I run nm as well.

[Mark Mentovai]

U stands for Undefined. That tells you which library wants the symbol, but not which library provides it.

For a C++ function named CreateARGBImageEncoder(), you’d be looking for a T (text section) (or lowercase t) entry.

[Eric Boren]

Whoops!  Thanks Mark.  I'm able to build the shared library now, but I'm having trouble linking an executable against it; there are undefined symbols which *should* be contained in the shared library.  Interestingly, the executable seems to want to link against both the shared library and the static libraries contained within the shared library.

Ld ../xcodebuild/Debug/tests normal i386

    cd /usr/local/google/skia/trunk/gyp

    setenv MACOSX_DEPLOYMENT_TARGET 10.7

    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -arch i386 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk -L/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks -F/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug -filelist /usr/local/google/skia/trunk/gyp/../xcodebuild/tests.build/Debug/tests.build/Objects-normal/i386/tests.LinkFileList -mmacosx-version-min=10.7 /usr/local/google/skia/trunk/xcodebuild/Debug/libskia.dylib /usr/local/google/skia/trunk/xcodebuild/Debug/libflags.a /usr/local/google/skia/trunk/xcodebuild/Debug/libexperimental.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_pdf.a /usr/local/google/skia/trunk/xcodebuild/Debug/libpicture_utils.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_core.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_opts.a /usr/local/google/skia/trunk/xcodebuild/Debug/libskia_opts_ssse3.a /usr/local/google/skia/trunk/xcodebuild/Debug/libzlib.a -framework ApplicationServices -lz -o /usr/local/google/skia/trunk/gyp/../xcodebuild/Debug/tests

Undefined symbols for architecture i386:

  "sk_frewind(SkFILE*)", referenced from:

      SkFILEStream::rewind() in libskia_core.a(SkStream.o)

....

ld: symbol(s) not found for architecture i386

clang: error: linker command failed with exit code 1 (use -v to see invocation)

$ nm xcodebuild/Debug/libskia.dylib | grep sk_frewind

00225670 t __Z10sk_frewindP6SkFILE

[Mark Mentovai]

What’s your executable depend on in the GYP files?

[Eric Boren]

The shared library along with a handful of other things not included
in the shared lib (skia_base_libs is the shared library, which
contains core, ports, utils, opts...):

'dependencies': [
'skia_base_libs.gyp:skia_base_libs',
'flags.gyp:flags',
'experimental.gyp:experimental',
'pdf.gyp:pdf',
'tools.gyp:picture_utils',
],

>>>>> Who's supposed to provide that symbol? What implementation file => what
>>>>> object file => what static library?

[Mark Mentovai]

Are there any other dependencies being pushed to it from a 'target_defaults' section in its gyp file (or some nearby gypi that it includes?)

Is this checked in or can I look at a code review to see a more complete view of the files?

[Eric Boren]

It doesn't look that way, after scanning through all of the
target_defaults in our gyp files. At least some of the missing
symbols come from the "ports" target, which is linked into
skia_base_libs but is not being linked by the executable. My guess is
that the executable is ignoring the shared library and is failing
because it's missing the ports library.

Our gyp files are checked in:
https://code.google.com/p/skia/source/browse/#svn%2Ftrunk%2Fgyp

And the change I've been hacking on is here:
https://codereview.chromium.org/14582008/

[Mark Mentovai]

1. It’s totally weird for gyp/tests.gyp to include pathops_unittest.gypi in the context it includes it, within a target.

2. Although you’re making skia_base_libs.gyp:skia_base_libs into a shared_library, your tests.gyp:tests target still depends on flags.gyp:flags, a static_library, which in turn depends on core.gyp:core, also a static_library, and one that’s included in the skia_base_libs.gyp:skia_base_libs shared_library. Now you have a path from tests to core both as a static library and as part of a shared library. There are probably other relationships like this one, I stopped hunting as soon as I found the first one. In order for this project to work, you need to prevent executables from reaching around your new shared library into the static libraries that it’s made up of.

In GYP-land, declaring that an executable depends on a shared library only causes the executable to link against that shared library, it doesn’t cause the executable to link against the static libraries that the shared library depends on. This is how you want it to work, now you just need to get rid of the relationships to the static libraries that bypass the shared library like the one I’ve pointed out.

[Eric Boren]

Thanks, Mark.

On Tue, May 7, 2013 at 12:44 PM, Mark Mentovai <ma...@chromium.org> wrote:
> 1. It’s totally weird for gyp/tests.gyp to include pathops_unittest.gypi in
> the context it includes it, within a target.
>

What's the preferred method? Defining the source list as a variable
in the gypi, including it at the top of the file and expanding the
variable in the source list?

> 2. Although you’re making skia_base_libs.gyp:skia_base_libs into a
> shared_library, your tests.gyp:tests target still depends on
> flags.gyp:flags, a static_library, which in turn depends on core.gyp:core,
> also a static_library, and one that’s included in the
> skia_base_libs.gyp:skia_base_libs shared_library. Now you have a path from
> tests to core both as a static library and as part of a shared library.
> There are probably other relationships like this one, I stopped hunting as
> soon as I found the first one. In order for this project to work, you need
> to prevent executables from reaching around your new shared library into the
> static libraries that it’s made up of.
>

Indeed, I didn't realize that those dependency chains still existed.
I removed them all, including dependencies of the skia_base_libs on
each other, which was probably overkill. Patch set 7 of
https://codereview.chromium.org/14582008/. Now the skia_base_libs
static libs aren't included in the linker input, but I'm still getting
lots of undefined symbols, all coming from the shared library. I must
be missing some flag to LD?

Ld ../xcodebuild/Debug/tests normal i386
cd /usr/local/google/skia/trunk/gyp
setenv MACOSX_DEPLOYMENT_TARGET 10.7
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
-arch i386 -isysroot
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk
-L/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug

-F/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug -filelist
/usr/local/google/skia/trunk/gyp/../xcodebuild/tests.build/Debug/tests.build/Objects-normal/i386/tests.LinkFileList
-mmacosx-version-min=10.7
/usr/local/google/skia/trunk/xcodebuild/Debug/libskia.dylib
/usr/local/google/skia/trunk/xcodebuild/Debug/libflags.a
/usr/local/google/skia/trunk/xcodebuild/Debug/libexperimental.a
/usr/local/google/skia/trunk/xcodebuild/Debug/libskia_pdf.a
/usr/local/google/skia/trunk/xcodebuild/Debug/libpicture_utils.a

/usr/local/google/skia/trunk/xcodebuild/Debug/libzlib.a -lz -o

/usr/local/google/skia/trunk/gyp/../xcodebuild/Debug/tests
Undefined symbols for architecture i386:

"_SetCurveBounds", referenced from:
PathOpsBoundsTest(skiatest::Reporter*) in PathOpsBoundsTest.o
"SkFixedMod(int, int)", referenced from:
TestMath(skiatest::Reporter*) in MathTest.o
......

$ nm xcodebuild/Debug/libskia.dylib | grep _SetCurveBounds
004badf0 d _SetCurveBounds

$ nm xcodebuild/Debug/libskia.dylib | grep SkFixedMod
000aebf0 t __Z10SkFixedModii

[Mark Mentovai]

Eric Boren wrote:

Thanks, Mark.

On Tue, May 7, 2013 at 12:44 PM, Mark Mentovai <ma...@chromium.org> wrote:
> 1. It’s totally weird for gyp/tests.gyp to include pathops_unittest.gypi in
> the context it includes it, within a target.

What's the preferred method?  Defining the source list as a variable
in the gypi, including it at the top of the file and expanding the
variable in the source list?

Never mind, this is fine. I looked at the content of pathops_unittest.gyp, which would be weird to include into tests.gyp.

Lowercase d and t mean nonpublic (local) symbols in the data and text sections, respectively. You can’t access nonpublic symbols from outside of the dylib. If they’re intended to be API usable from outside of the library, you need to decorate them with the appropriate __attribute__((visibility)). It looks like you already have an SK_API macro that exists for this purpose.

[Eric Boren]

My inability to parse nm output strikes again. So it seems that we
use a bunch of stuff in our unit tests which Chrome's shared library
build does not, which makes sense. We'll have to figure out what to
do about that.

Thanks for all of your help.

[Evan Martin]

This problem (that unit tests want to reach into API that isn't really "public" and how to manage the symbol visibility) was the subject of a number of discussions and different approaches.  At one point there was a separate annotation for "exported for tests" vs "normally exported", but looking now I don't see it anymore.

[Scott Graham]

https://code.google.com/p/chromium/codesearch#search/&q=export_private&sq=package:chromium&type=cs ?