Review: Cleanup library symbol visibility issues

[Larry Gritz]

https://github.com/imageworks/OpenShadingLanguage/pull/138


* On gcc-based systems, use attribute((visibility("hidden|default"))) (in export.h) and -fvisibility=hidden on the compile & link lines to control symbol visibility so that only symbols that are part of the public APIs are visible in the resulting library. (But also must include functions that need to be found by dlsym() in order for LLVM to correctly generate code to call them from shaders.)

* Add Makefile/CMake option LLVM_STATIC to use static LLVM libs rather than dynamic library.

* On Linux, when LLVM_STATIC is used, also use linker option --version-script and a custom version-script for liboslexec.so in order to force LLVM's symbols to also remain hidden outside the OSL library. This is extremely helpful to apps that want to embed OSL but also embed LLVM for other reasons (and not necessarily the same version of LLVM) -- this makes sure the symbols don't crash when the program runs, leading to strange crashes.

* (SPI only - make it compile this way by default)

--
Larry Gritz
l...@larrygritz.com

[Larry Gritz]

A follow-up on this, I've updated the patch: now I hide non-public symbols optionally (HIDE_SYMBOLS).

Now the overall logic is:

* If HIDE_SYMBOLS is on, hide OSL library symbols that aren't part of the
public API for release builds (DEBUG builds still include all symbols).
It's probably good to do this, though the default is off and you turn
it on, any fancy stack traces will probably not reflect where a crash
really happened.

* If LLVM_STATIC is on, link against static libraries rather than dynamic.

* Furthermore, if LLVM_STATIC is on and it's Linux, hide the LLVM symbols
in release builds so they can't conflict with other apps or plugins.
(Sorry, I only know how to make it do this for Linux.)

(For SPI folks: this means we can compile with HIDE_SYMBOLS off, so Arnold crash stack traces work as they always did, but with LLVM_STATIC on, so that we don't conflict when OSL is used in Houdini plugins. I think that lets us have one internal build that will make everybody happy. External people have an orthogonal choice on both the symbol hiding as well as the static/hidden llvm.)

> --
> You received this message because you are subscribed to the Google Groups "OSL Developers" group.
> To post to this group, send email to osl...@googlegroups.com.
> To unsubscribe from this group, send email to osl-dev+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/osl-dev?hl=en.
>

--
Larry Gritz
l...@larrygritz.com