Branch OSL RB-1.5!

[Larry Gritz]

I have tagged the current top-of-tree as "Release-1.5.8RC1", and created branch "RB-1.5".

This is a release candidate. Henceforth, we will try really, really hard for nothing in RB-1.5 to break API or link compatibility or change behavior (except for bug fixes or minor enhancements that have very low risk of breaking anything). In a week or so, if nobody finds anything drastically broken, we will then formally "release" 1.5 as the branch of OSL that is recommended for production and commercial products.

I'm calling this a "release candidate" rather than a "beta" because at SPI, we build for our productions straight from the OSL master branch. So it does tend to be very stable and has seen extensive production use. Since we branched 1.4 in November, this working code was used to render our work in Edge of Tomorrow, The Amazing Spider-Man 2, Cloudy With a Chance of Meatballs 2, Blended, 22 Jump Street, and Guardians of the Galaxy. So it ought to be fairly production-hardened (modulo some of the very last-minute changes, which hopefully you will all test right away).

New development will continue in the "master" branch, including possibly breaks in compatibility.

You can read the CHANGES file for a terse summary of all that's happened since 1.4 became the release branch. When we get closer to a final release, I will post the whole thing here, as well as highlights about what's changed.


--
Larry Gritz
l...@larrygritz.com

[Larry Gritz]

A few minor fixes, now tagged as Release-1.5.9RC2, at the (current) tip of the RB-1.5 branch.

Speak up now, or expect this to become the official 1.5 release later this week. (With future bug fixes, of course.)

The 'master' branch has already had its version renamed to 1.6.0dev and is now allowed to diverge and incorporate new features or compatibility breaks.

-- lg

[Larry Gritz]

Release candidate 3 tagged as: Release-1.5.10RC3

As I said in another message, this one does unfortunately break compatibility in the RendererServices API.

This is the last intended compatibility break before we declare 1.5 final, which I'm hoping to do early next week, unless critical bugs are found.

-- lg

--
Larry Gritz
l...@larrygritz.com

[Thomas Dinges]

Hi Larry,
thanks for the great work!

I tested the new OSL release and OIIO 1.4.11 on Windows 7 x64, with Visual Studio 2013 now, I was happy to see that it almost compiles out of the box.
Attached 2 diff files here (sorry for not going via github, this is quicker for me now), with simple fixes.

1) OIIO fix for ptex, when linking statically. (patch against RB-1.4)
2) OSL fixes, mostly math includes. (patch against RB-1.5)

Hope this helps.

Best regards,
Thomas

[Larry Gritz]

A week or so had gone by without comment on the last release candidate, so with no more code patches, the head of the RB-1.5 branch has been re-tagged as "Release-1.5.10" (no more "release candidate" designation).

People incorporating OSL into commercial products or production use are encouraged to use the 1.5 branch.

Although we will continue to patch it with bug fixes (and occasionally "safe" enhancements), we will try very hard not to change anything considered to be correct behavior, nor to break public API, ABI, or link back-compatibility within the 1.5 branch.

We now consider 1.4 and older releases to be no longer supported, and we will not patch them any more unless specifically requested to fix a particular bug (and even then, it will depend on how easily it can be done).

Meanwhile, the "master" branch is where new innovation will occur. Be warned -- we may change public APIs, break link compatibility, or alter behavior at any time. But that said, at Sony Pictures Imageworks, we build our proprietary production renderer directly from the public OSL master branch, so it does tend at nearly all times to be stable, production-hardened, and if not bug-free, then at at least there is a dire need to have bugs fixed right away when they are discovered to impact production in any way.


Below are the full release notes. Separate emails in the coming days will highlight a few of the big changes in 1.5 and discuss the implications for renderer developers (some of the big ideas are not always done justice by the terse release notes).



Release 1.5 -- July 30, 2014 (compared to 1.4)
----------------------------------------------
Language, standard libary, and compiler changes (for shader writers):
* New closure function for microfacet BSDFs:
closure color microfacet (string distribution, normal N,
float alpha, float eta, int refract)
closure color microfacet (string distribution, normal N, vector U,
float xalpha, float yalpha, float eta, int refract)
These replace the growing zoo of microfacet_blah functions (using
different distribution values such as "beckmann" or "ggx", as well as
selecting refraction when refract!=0). The old varieties are now considered
deprecated and will eventually be removed (but will alow back-compatibility
of compiled oso files).
* Remove documentation about unimplemented hash() function -- it was never
implemented, and we can't figure out what it's good for. (#359) (1.5.7)
* Documented the specific names that should be used with getattribute() to
retrieve common camera parameters. We should expect these to be the same
for all renderers (though individual renderers may also support additional
attribute queries). #393 (1.5.9)

API changes, new options, new ShadingSystem features (for renderer writers):
* ShadingSystem API changes:
* ShadingSystem::Parameter() has a new variety that can set lockgeom=0
for a particular parameter at runtime, even if it was not declared
this way in the shader's metadata. This allows an app to effectively
say "I intend to vary this shader's parameter value, don't optimize as
if it were a a constant." (1.5.1)
* New ShadingSystem::ReParameter() lets you change the instance value of
a shader parameter after the shader group has already been declared
(but only if the shader has not yet been optimized, OR if the
parameter was declared with lockgeom=0). This, in combination with the
new SS::Parameter() variety, lets an app select a parameter for
variation, compile the shader network, and then continuously modify
the value and re-render without needing to redeclare and recompile the
shader group (though at a slightly higher execution cost than if the
parameter were fully optimized. (1.5.1)
* ShadingAttribState has been simlified and replaced with ShaderGroup.
And ShadingAttribStateRef replaced by ShaderGroupRef. (1.5.1)
* ShaderGroupBegin returns the ShaderGroupRef for the group, there is
no need to call state() to get the group reference. (1.5.1)
* New ShadingSystem::getattribute() and attribute() varieties that takes
a ShaderGroup* as a first argument, allows setting and retrieval of
attributes pertaining to a specific shader group. (#362, #368) (1.5.8)
SEE SECTION BELOW ON GROUP-SPECIFIC ATTRIBUTES
* New ShaderGroupBegin() variant takes a text description of a shader
group. (3.5.8) (#379) For example:
ShaderGroupBegin ("groupname", "surface",
"param float fin 3.14; " /*concatenate string*/
"shader a alayer;"
"shader b blayer;"
"connect alayer.f_out blayer.f_in;");
* ShadingSystem methods renderer() and texturesys() retrieve the
RendererServices and TextureSystem, respectively. (1.5.8)
* ShadingSystem statistics output now prints the version and major
runtime options. (1.5.8) (#369)
* New/changed ShadingSystem global attributes:
* Attributes "archive_groupname" and "archive_filename", when set, will
cause the named group to be fully archived into a .tar, .tar.gz, or
.zip file containing the serialized description of the group, and all
.oso files needed to reconstruct it. This is great for shipping
debugging or performance benchmark cases to the developers! It's also
a good way to "extract" a shader network from the midst of a complex
renderer and then be able to reconstitute it inside a debugging test
harness like testshade. (1.5.8) (#381) For more detail and explanation,
please see https://github.com/imageworks/OpenShadingLanguage/pull/381
* New per-ShaderGroup attributes:
* ShadingSystem::getattribute(group,...) allows queries that retrieve
all the names and types of userdata the compiled group may need
("num_userdata", "userdata_names", "userdata_types"), as well as the
names of all textures it might access and whether it's possible that
it will try to acccess textures whose names are not known without
executing the shader ("num_textures_needed", "textures_needed",
"unknown_textures_needed"). (#362) (1.5.8)
* Retrieving the group attribute "pickle" (as a string) gives you the
serialized description of the shader group in the same format that is
accepted by ShaderGroupBegin(). (1.5.8) (#381)
* Group-wise attribute "renderer_outputs" (array of strings) lets you
specify different renderer outputs for each shader group in addition
to the ShadingSystem-wide "renderer_outputs" that apply to all groups.
Remember that naming the renderer outputs prevents those from being
optimized away if they are not connected to downstream
shaders. (1.5.8) (#386)
* New group attribute queries: "groupname", "num_layers", "layer_names"
allow you to ask for the name of the group, how many layers there are,
and the names of all the layers.
* ShaderGlobals changes:
* Added a RendererServices pointer to ShaderGlobals, to eliminate a
bunch of pointless indirection during shader execution. (1.5.2)
* Split the definition of ShaderGlobals into its own header file,
shaderglobals.h. (1.5.1)
* RendererServices changes:
* RendererServices::get_texture_info method parameters have changed, now
includes an additional ShaderGlobals* parameter (as did all the other
texture functions already). (1.5.5)
* Split RendererServices definition into separate rendererservices.h. (1.5.6)
* Add RendererServices::texturesys() that returns a pointer to the
RS's TextureSystem. (1.5.6)
* Add a ShaderGlobals* parameter to the various get_matrix and
get_inverse_matrix methods. This makes it easier for renderers that
might need the ShaderGlobals (or access to the void* renderstate
contained therein) to look up matrices, especially if the renderer
supports different per-primitive transformation name bindings.
#396 (1.5.10)
* Change the last 4 RenderServices methods that took a void* renderstate
to instead take a ShaderGlobals *. It's more general, since the SG
does itself contain the renderstate pointer, and some implementations
or features might need the other contents of the SG. #396 (1.5.10)
* OSLQuery changes:
* OSLQuery::Parameter has a new 'structname' field, that for struct
parameters reveals the name of the structure type. The oslinfo utility
has also been updated to print this information. (1.5.4)
* OSLQuery now may be initialized by a ShaderGroup*, which allows an
app with a live shading system to query information about shaders
much more efficiently than reloading and parsing the .oso file from
disk. #387 (1.5.8)
* OSLQuery API has replaced most of its std::string parameters and
members with string_view and ustring, to speed it up and eliminate
lots of small allocations. This may introduce some small source
incompatibilities with the old OSLQuery, but they should be very
easy to fix in your code. #387 (1.5.8)
* Miscellaneous/general:
* Use OIIO's new string_view for many public API input parameters that
previously took a std::string& or a const char*. (1.5.7) (#361)
* De-virtualized OSLCompiler and ShadingSystem classes -- it is now safe to
construct them in the usual way, the create() and destroy() functions for
them are optional/deprecated. (#363) (1.5.8)

Performance improvements:
* Dramatically speed up LLVM optimization & JIT of small shaders --
reducing the per-shader-network compile overhead of small shaders
from 0.1-0.2s to less than 0.01s per shader group. (1.5.1)
* Runtime constant folding of substr(). (1.5.6)
* Improved code generation for texture calls, removing redundant setting
of optional texture parameters when they don't change from the default
values. (1.5.6)
* Remove lock from RendererServices texture calls that would create a big
thread botteneck (but only for renderers that used the RS base class
texture calls; those that overrode them would not notice). (1.5.6)
* Constant folding of certain assignments to individual components or
array elements. (#356) (1.5.7)
* Much more complete constant-folding of format(). (1.5.8) (#366)
* Enhanced simplification of component and array assignment, when
multiple assignments actually fill out the entire array or aggregate.
For example: 'color C; C[0] = 3.14; C[1] = 0; C[2] = 42.0;'
Now it recognizes that all components are assigned and will replace
the whole thing (and subsequent uses of C) with a constant
color. (1.5.8) (#367)
* Eliminate redundant retrieval of userdata within a shading
network. (1.5.8) (#373)

Bug fixes:
* testshade/testrender timer didn't properly break out execution time
from setup time. (1.4.1/1.5.1)
* Guard matrix*point transformations against possible division by zero.
(1.4.1/1.5.1)
* Fix subtle bug with splines taking arrays, where the number of knots
passed was less than the full length of the array, and the knots had
derivatives -- the derivatives would be looked up from the wrong spot
in the array (and could read uninitialized portions of the array).
(1.4.1/1.5.3)
* Fix incorrect function declarations for the no-bitcode case (only
affected Windows or if a site purposely built without precompiled
bitcode for llvm_ops.cpp). (1.4.1/1.5.3)
* When using Boost Wave for oslc's preprocessor, fixed it so that
variadic macros work. (1.5.4)
* Ensure that isconnected() works for struct parameters. (1.5.4)
* Fix incorrect data layout in heap for certain array parameters.
(1.5.5/1.4.2)
* oslc -D and -U did not work properly when OSL was built against
Boost Wave as the C preprocessor. (1.5.5/1.4.2)
* oslc bug fix for innitializing array elements inside a parameter that's a
struct. (1.5.6)
* Implementation of the luminance() function has been improved to
guarantee that all the components sum to exactly 1.0. (1.5.6)
* OSLQuery: for array parameters whose source code only specify default
values for partially initializing the array, nonetheless make the
default vectors be the correct length for the array (with zeroes for
array elements without explicit initialization). (1.5.6)
* oslc: negating closures resulted in hitting an assertion. (1.5.6)
* ShadingSystem - bug fix when Parameter() sets the same parameter twice
for the same instance, the second time back to the default
value. (1.5.8) (#370)
* Fix debug printing crash when group name was not set. (1.5.7)
* Bug fix with runtime optimization instance merging -- differing
"lockgeom" status of parameters should invalidate the merge. (1.5.8) (#378)
* Runtime optimizer bug fix: output parameter assignment elision was
overly aggressive when the output parameter in question was a renderer
output (and therefore should not be elided). (1.5.8) (#378)
* Fix printf("%f",whole_float_array) to work properly. (1.5.8) (#382)
* oslc improperly allowed ONE too many initializers for arrays. (1.5.8) (#383)

Under the hood:
* Internals refactor that separates the runtime optimization from the
LLVM generation and JIT. (1.5.1)
* Removed legacy built-in closures that weren't actually used. (1.5.1)
* Split all the direct calls to LLVM API into a separate LLVM_Util
class, which has no other OSL dependencies and may be used from other
apps to simpilfy use of LLVM to JIT code dynamically. (1.5.1)
* Stop using deprecated TypeDesc #defines. (1.5.4)
* A shader's printf, warning, and error output is now buffered so that
all the "console" output from a shader invocation will appear
contiguous in a log, rather than possibly line-by-line interspersed
with the output of another shader invocation running concurrently in a
different thread (1.5.5).
* By de-virtualizing OSLCompiler and ShadingSystem classes, we make it
much easier in the future to change both these classes (at least, adding
new methods) and changing the Impl internals, without breaking link
compatibility within release branches. (#363) (1.5.8)
* Eliminate the need for a special dlopen when using OSL from within
a DSO/DLL plugin of another app. (1.5.8) (#374, #384)

Build & test system improvements and developer goodies:
* Make OSL work with LLVM 3.4 (1.5.4)
* New llvm_util.h contains an LLVM_Util standalone utility class that can
be used from any app as a simplified wrapper for LLVM code generation
and JIT. (1.5.1)
* testshade consolidates --iparam, --fparam, --vparam, --sparam into
just --param that figures out the types (or could be explicitly
told. (1.5.1)
* More robust detection of clang, for older cmake versions.
(1.4.1/1.5.1)
* Handle 3-part version numbers for LLVM installations. (1.4.1/1.5.1)
* Fix USE_EXTERNAL_PUGIXML versus USING_OIIO_PUGI issue. (1.4.1/1.5.3)
* Remove all ehader include guards, switch to '#pragma once' (1.5.2)
* Adjust build to account for new library names in OpenEXR 2.1. (1.5.3)
* Make sure we include all OpenEXR and Ilmbase headers with <OpenEXR/foo.h>
and remove the addition of OpenEXR sub-folder from system search paths.
(1.5.1)
* Better detection of LLVM version number from the LLVM installation.
(1.5.1)
* Support for using LLVM's "MCJIT" instead of "old JIT" (controlled by
using the optional 'make USE_MCJIT=1'). This is mostly experimental,
and not recommended since performance of old JIT is still much better
than MCJIT. (1.5.6) Fixes in (1.5.8) (#385)
* Moved the source of all public header files to src/include/OSL, to more
closely mimic the install file layout. (1.5.6)
* Fixes for compilation warnings with newer gcc. (1.5.7) (#351)
* Fixes for IlmBase 2.1 build breakages. (#354) (1.5.7)
* Make testshade multithreaded, so it's always exercising the thread
safety of the shading system. (1.5.8) (#365)
* Compatibility with cmake 3.0. (1.5.8) (#377)
* Fix warnings with some gcc versions. (1.5.8)
* Make testrender accept the new shadergroup syntax. (1.5.9)
* Make testrender & testshade support all the standard camera getattribute
queries. (1.5.9)
* testshade/testrender: have get_attribute fall back on get_userdata,
and have get_userdata bind "s" and "t" to make it easy to test
shader that rely on userdata. #395 (1.5.10)
* Fixes for Boost 1.55 + Linux combination, where it needed -lrt on the
link line. #394 (1.5.10)

Documentation:
* Clarified docs about floor(). (1.5.8)
* Clarified that oren_nayar() is the name of the function, not
orennayar (it was not consistent). (1.5.8) (#375)
* backfacing() was previously left out of the docs. (1.5.8) (#376)
* Documented the specific names that should be used with getattribute() to
retrieve common camera parameters. We should expect these to be the same
for all renderers (though individual renderers may also support additional
attribute queries). #393 (1.5.9)
* Clarify that we intend for getattribute() to be able to retrieve
"userdata" (aka primitive variables) if no attribute is found with the
matching name. #395 (1.5.10)



--
Larry Gritz
l...@larrygritz.com

[Larry Gritz]

Sorry for the delay.  Committed your fixes.

--
You received this message because you are subscribed to the Google Groups "OSL Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osl-dev+u...@googlegroups.com.
To post to this group, send email to osl...@googlegroups.com.
Visit this group at http://groups.google.com/group/osl-dev.
For more options, visit https://groups.google.com/d/optout.
<oiio_ptex_fix.diff><osl_fixes_msvc2013.diff>

--
Larry Gritz
l...@larrygritz.com