OpenSceneGraph-3.6.5 release candidate 2 tagged, please test

[Robert Osfield]

Hi All,

I have just tagged the OpenSceneGraph-3.6.5-rc2:

    https://github.com/openscenegraph/OpenSceneGraph/tree/OpenSceneGraph-3.6.5-rc2

Please test across as many platforms and applications as you have available, and report success or failures here on this thread so we can track convergence.  All going well we'll be able to make the 3.6.5 stable release this week,

Thanks in advance for your help in testing/build and bug fixes :-)

Cheers,

Robert.

-- Changes since 3.6.5-rc1

Tue, 21 Jan 2020 09:43:08 +0000
Author : Robert Osfield
Removed stray space

Tue, 21 Jan 2020 09:32:57 +0000
Author : OpenSceneGraph git repository
Merge pull request #902 from mp3butcher/oqn3.6 OQN API convergence

Tue, 21 Jan 2020 09:16:51 +0000
Author : OpenSceneGraph git repository
Merge pull request #903 from dedowsdi/renderstageAdd getPreRenderList, getPostRenderList to RenderStage.

Fri, 17 Jan 2020 18:47:49 +0800
Author : dedowsdi
Add getPreRenderList getPostRenderList to RenderStage.

Fri, 23 Aug 2019 09:59:54 +0200
Author : Daniel Trstenjak
OcclusionQueryNode: make all usages of 'updateDefaultQueryGeometry' thread safe

Fri, 23 Aug 2019 09:46:02 +0200
Author : Daniel Trstenjak
OcclusionQueryNode: fix resetting to default query geometryWhen the query geometry gets reset to its default then its
vertices have to be updated by the bounding box dimensions of
the current children of the OcclusionQueryNode.


Wed, 14 Aug 2019 11:27:40 +0200
Author : Daniel Trstenjak
OcclusionQueryNode: fix use case of user defined query geometryThe user defined query geometry handling has been broken in several ways.

The previous way of defining a query geometry was using the non const
`getQueryGeometry` method and overriding its members. But then
`OcclusionQueryNode` wasn't aware of the geometry change and couldn't
internally handle it correctly.

The `computeBound` method never considered a user defined query geometry and
always just overrode the vertices of the geometry.

The member `_validQueryGeometry` wasn't correctly set.

This change should fix all this issues by introducing a small backward
compatibility break. The non const `getQueryGeometry` method is removed
forcing the user to use the `setQueryGeometry` method. But then `OcclusionQueryNode`
is aware of the user defined query geometry and can handle it correctly.


Tue, 29 Jan 2019 14:40:16 +0100
Author : Daniel Trstenjak
OcclusionQueryNode: reset the test result of the invalid geometryThere're cases that the occlusion test result has been retrieved
after the query geometry has been changed, it's the result of the
geometry before the change.


Tue, 29 Jan 2019 11:37:28 +0100
Author : Daniel Trstenjak
OcclusionQueryNode: ensure a valid query geometryIf the query geometry is invalid then don't do any occlusion queries and
never traverse the subgraphs.


Fri, 25 Jan 2019 15:02:45 +0100
Author : Daniel Trstenjak
OcclusionQueryNode: ensure a consistent value for '_passed'

Sat, 26 Jan 2019 16:33:23 +0000
Author : Robert Osfield
Introduced a QueryGeometry::getQueryResult(const osg::Camera*) method as a more informative replacedment for QueryGeometry::getNumPixels().

Mon, 20 Jan 2020 10:37:12 +0000
Author : OpenSceneGraph git repository
Merge pull request #900 from dedowsdi/fix_particle_rotationFix particle rotation.

Fri, 17 Jan 2020 11:18:30 +0800
Author : dedowsdi
Fix particle rotation.

Fri, 17 Jan 2020 09:28:09 +0000
Author : Robert Osfield
Updated ChangeLog

Fri, 17 Jan 2020 09:07:58 +0000
Author : Robert Osfield
Moved setting of isftream locale to Model::readOBJ(..) and Model::readMTL(..).

Fri, 17 Jan 2020 08:54:52 +0000
Author : Robert Osfield
Added explict setting of local to classic to avoid local platform settings affecting parsing

[Robert Osfield]

HI All,

Still waiting on feedback on how well 3.6.5-rc2 is working OK.  I'm ready to tag 3.6.5 at my end as there are no Issue reported yet that I can look into resolving.

If there are no Issue's raised by Monday I'll go ahead and tag 3.6.5 stable release.

Cheers,

Robert.

[Stuart Mentzer]

Hi Robert,

OSG 3.6.5-rc2 built fine on Windows with VC2019 (so VC2017 is probably OK) and appears to be running properly in our application. If there are more RCs I can try to test those and I should be able to get 3.6.5 binaries posted soon after it is released.

Other than the usual mods I make to build with VC using GNU make the FBX plugin support has some issues that I worked around, wrt CMake usage and support for the newer VC and FBX versions. I can share my fixed versions if you would like and you tell me the best way to do that now.

Cheers,

Stuart

[Robert Osfield]

Hi Stuart,

On Sun, 26 Jan 2020 at 13:46, Stuart Mentzer <obj...@objexx.com> wrote:

OSG 3.6.5-rc2 built fine on Windows with VC2019 (so VC2017 is probably OK) and appears to be running properly in our application. If there are more RCs I can try to test those and I should be able to get 3.6.5 binaries posted soon after it is released.

Great thanks.

 

Other than the usual mods I make to build with VC using GNU make the FBX plugin support has some issues that I worked around, wrt CMake usage and support for the newer VC and FBX versions. I can share my fixed versions if you would like and you tell me the best way to do that now.

It would be best to have 3.6.5 go out with support for recent VC and FBX versions so would appreciate if you could generate a PR for them.  I can merge them and make 3.6.5-rc3

Cheers,

Robert

 

[Stuart Mentzer]

It would be best to have 3.6.5 go out with support for recent VC and FBX versions so would appreciate if you could generate a PR for them.  I can merge them and make 3.6.5-rc3

Cheers,

Robert

 

OK, done. https://github.com/openscenegraph/OpenSceneGraph/pull/907 

These changes fix my Windows VC builds but I'm no expert in CMake or the requirements of all the different builds so I suggest that this is reviewed by experienced OSG builders.

Cheers,

Stuart

[Fabian Roth]

Hi,

I am currently testing this RC with openmw.

If i have the fps display or profiler open while exiting the application i get a crash on exit.

I am not sure if this is due to a bug in my build, a bug in openmw or a real issue with osg.

The issue seems to be related to the destruction of the default font, but i was not able to investigate further.

Attached is a Backtrace of the issue i am currently observing.

Greetings,

Fabian

*** Fatal Error ***
Invalid permissions for mapped object (signal 11)
Address: 0x5612597e50c0

* Backtrace

#2  0x00007f13a65b7f20 in <signal handler called> () at /lib/x86_64-linux-gnu/libc.so.6
#3  0x00005612597e50c0 in  ()
#4  0x000056125898fdc4 in OpenThreads::ScopedPointerLock<OpenThreads::Mutex>::ScopedPointerLock(OpenThreads::Mutex*) () at ./openmw/extern-git/OpenSceneGraph/include/OpenThreads/ScopedLock:54
        m = 0x5612597e5250
        this = <synthetischer Zeiger>
        lock = {m_lock = 0x5612597e5250}
        pitr = <optimized out>
#5  0x000056125898fdc4 in osg::StateAttribute::removeParent(osg::StateSet*) (this=0x5612647f58e0, object=<optimized out>, object@entry=0x5612647d48f0) at ./openmw/extern-git/OpenSceneGraph/src/osg/StateAttribute.cpp:38
        lock = {m_lock = 0x5612597e5250}
        pitr = <optimized out>
#6  0x0000561258991d2c in osg::StateSet::clear() (this=this@entry=0x5612647d48f0) at ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:734
        itr = {_M_node = 0x5612647f5a80}
#7  0x0000561258991f96 in __base_dtor  (this=0x5612647d48f0) at ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:285
#8  0x0000561258992129 in __deleting_dtor  (this=0x5612647d48f0) at ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:286
#9  0x0000561258bad988 in signalObserversAndDelete (signalDelete=true, doDelete=true, this=0x5612647d48f0) at /usr/include/c++/7/bits/stl_construct.h:107
        newRef = <optimized out>
        needDelete = <optimized out>
#10 0x0000561258bad988 in osg::Referenced::unref() const (this=0x5612647d48f0) at ./openmw/extern-git/OpenSceneGraph/include/osg/Referenced:201
        newRef = <optimized out>
        needDelete = <optimized out>
#11 0x0000561258bad988 in osg::ref_ptr<osg::StateSet>::~ref_ptr() () at ./openmw/extern-git/OpenSceneGraph/include/osg/ref_ptr:41
#12 0x0000561258bad988 in std::_Destroy<osg::ref_ptr<osg::StateSet> >(osg::ref_ptr<osg::StateSet>*) () at /usr/include/c++/7/bits/stl_construct.h:98
#13 0x0000561258bad988 in std::_Destroy_aux<false>::__destroy<osg::ref_ptr<osg::StateSet>*>(osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet>*) () at /usr/include/c++/7/bits/stl_construct.h:108
#14 0x0000561258bad988 in std::_Destroy<osg::ref_ptr<osg::StateSet>*>(osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet>*) () at /usr/include/c++/7/bits/stl_construct.h:137
#15 0x0000561258bad988 in std::_Destroy<osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet> >(osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet>*, std::allocator<osg::ref_ptr<osg::StateSet> >&) () at /usr/include/c++/7/bits/stl_construct.h:206
#16 0x0000561258bad988 in std::vector<osg::ref_ptr<osg::StateSet>, std::allocator<osg::ref_ptr<osg::StateSet> > >::~vector() [clone .lto_priv.5148] (this=0x561264714620) at /usr/include/c++/7/bits/stl_vector.h:434
#17 0x0000561258bad988 in __base_dtor  (this=0x5612647145c0) at ./openmw/extern-git/OpenSceneGraph/src/osgText/Font.cpp:295
#18 0x0000561258bada13 in __base_dtor  () at ./openmw/extern-git/OpenSceneGraph/src/osgText/DefaultFont.cpp:35
        this = 0x5612647145c0
#19 0x0000561258bada13 in __deleting_dtor  (this=0x5612647145c0) at ./openmw/extern-git/OpenSceneGraph/src/osgText/DefaultFont.cpp:37
#20 0x0000561258a1b6b4 in signalObserversAndDelete (signalDelete=true, doDelete=true, this=0x5612647145c0) at ./openmw/extern-git/OpenSceneGraph/src/osg/Referenced.cpp:281
        newRef = <optimized out>
        needDelete = <optimized out>
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#21 0x0000561258a1b6b4 in osg::Referenced::unref() const (this=0x5612647145c0) at ./openmw/extern-git/OpenSceneGraph/include/osg/Referenced:201
        newRef = <optimized out>
        needDelete = <optimized out>
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#22 0x0000561258a1b6b4 in osg::ref_ptr<osg::Object>::~ref_ptr() () at ./openmw/extern-git/OpenSceneGraph/include/osg/ref_ptr:41
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#23 0x0000561258a1b6b4 in std::pair<osg::ref_ptr<osg::Object>, double>::~pair() () at /usr/include/c++/7/bits/stl_pair.h:208
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#24 0x0000561258a1b6b4 in __base_dtor  (this=0x56126472ce40) at /usr/include/c++/7/bits/stl_pair.h:208
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#25 0x0000561258a1b6b4 in __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::destroy<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >(std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >*) () at /usr/include/c++/7/ext/new_allocator.h:140
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#26 0x0000561258a1b6b4 in std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > > >::destroy<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >(std::allocator<std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >&, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >*) () at /usr/include/c++/7/bits/alloc_traits.h:487
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#27 0x0000561258a1b6b4 in std::_Rb_tree<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >, std::_Select1st<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::_M_destroy_node(std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >*) () at /usr/include/c++/7/bits/stl_tree.h:650
        this = 0x561259805c38
        __p = 0x56126472ce20
        this = 0x561259805c38
        __p = 0x56126472ce20
#28 0x0000561258a1b6b4 in std::_Rb_tree<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >, std::_Select1st<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::_M_drop_node(std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >*) () at /usr/include/c++/7/bits/stl_tree.h:658
        this = 0x561259805c38
        __p = 0x56126472ce20
#29 0x0000561258a1b6b4 in _M_erase (this=this@entry=0x561259805c38, __x=0x56126472ce20) at /usr/include/c++/7/bits/stl_tree.h:1858
#30 0x0000561258a26262 in std::_Rb_tree<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >, std::_Select1st<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::clear() () at /usr/include/c++/7/bits/stl_tree.h:1171
        this = 0x561259805c38
        this = 0x561259805c38
        lock = {m_lock = @0x561259805c68}
#31 0x0000561258a26262 in std::map<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<osg::ref_ptr<osg::Object>, double>, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::clear() () at /usr/include/c++/7/bits/stl_map.h:1127
        this = 0x561259805c38
        lock = {m_lock = @0x561259805c68}
#32 0x0000561258a26262 in osgDB::ObjectCache::clear() (this=0x561259805c20) at ./openmw/extern-git/OpenSceneGraph/src/osgDB/ObjectCache.cpp:189
        lock = {m_lock = @0x561259805c68}
#33 0x0000561258a5bfa3 in clearObjectCache () at ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:1654
        this = 0x5612598055e0
#34 0x0000561258a5bfa3 in osgDB::Registry::destruct() (this=0x5612598055e0) at ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:507
#35 0x0000561258a5c10c in __base_dtor  (this=0x5612598055e0) at ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:486
#36 0x0000561258a5c499 in __deleting_dtor  (this=0x5612598055e0) at ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:487
#37 0x000056125863e367 in signalObserversAndDelete (signalDelete=true, doDelete=true, this=0x5612598055e0) at ./openmw/extern-git/OpenSceneGraph/src/osg/Referenced.cpp:281
        newRef = 0
        needDelete = true
#38 0x000056125863e367 in osg::Referenced::unref() const (this=0x5612598055e0) at ./openmw/extern-git/OpenSceneGraph/include/osg/Referenced:201
        newRef = 0
        needDelete = true
#39 0x00007f13a65bc041 in __run_exit_handlers (status=0, listp=0x7f13a6964718 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at exit.c:108
        atfct = <optimized out>
        onfct = <optimized out>
        cxafct = <optimized out>
        f = <optimized out>
        new_exitfn_called = 2523
        cur = 0x561259804ce0
#40 0x00007f13a65bc13a in __GI_exit (status=<optimized out>) at exit.c:139
#41 0x00007f13a659ab9e in __libc_start_main (main=0x56125860aae0 <main>, argc=2, argv=0x7ffd03613bb8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd03613ba8) at ../csu/libc-start.c:344
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -6745562391612795713, 94636792266176, 140724660157360, 0, 0, -1027869779201928001, -1125601318232042305}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x7f13a697a733 <_dl_init+259>, 0x7f13a6372a38}, data = {prev = 0x0, cleanup = 0x0, canceltype = -1500010701}}}
        not_first_call = <optimized out>
#42 0x000056125862bdea in _start ()

[Robert Osfield]

Hi Fiabian,

On Monday, 27 January 2020 09:41:43 UTC, Fabian Roth wrote:

Hi,

I am currently testing this RC with openmw.

If i have the fps display or profiler open while exiting the application i get a crash on exit.

I am not sure if this is due to a bug in my build, a bug in openmw or a real issue with osg.

The issue seems to be related to the destruction of the default font, but i was not able to investigate further.

Attached is a Backtrace of the issue i am currently observing.

That stack trace looks like the automatic clean up of the ObjectCache with the DefaultFont within it is related somehow to the crash.  How the DefaultFont is managed has changed, to address bugs ironically, and in a general sense the clean up the stack trace looks just fine to me, it's roughly what I'd expect to happen.  However, I don't have any clear idea why in this instance the crash has occurred.

Given there isn't any obvious amiss we are unfortunately are left to widen out the search for what is amiss.

Does running an OSG example like osgtext fail?

Do others in the OpenMMW community seen this same crash?

Is it possible to run OpenMMW single threaded to see if there might be some thread interaction?

What OS/compile and OpenMMW version combination are you using?

One possible cause of crash like this is memory corruption during the run of the application that is only revealed on clean up.  Using a memory tools like valgrind might be spot this type of issue.

Perhaps others might have seen something similar and can help shed some light on the nature of the crash.

If it's possible to recreate the crash with an standard OSG example, or a small modification of one, then this would be really helpful for me to jump in a start investigating the issue.

Cheers.

Robert.

 

 

[Chris Djali / AnyOldName3]

Hi Robert,

As I've mentioned in the past, I'm an OpenMW (note the single M) developer. It was actually me who reported the issues with the default font, and only a subset were resolved before you went on hiatus. I've lost the thread where we were discussing it as I'd bookmarked the forum thread, which is now dead. As I recall, though, my most recent minimal things-don't-get-relased-when-they-should example was resolvable by a couple of potential changes I suggested, but you said they'd have to wait until you could investigate more thoroughly, as it was in code no one had touched for a while, so could maybe upset other applications, or not exhaustively remove all such bugs.

I pretty much stopped trying to make OpenMW play nicely with 3.6.x when you disappeared as there wasn't much point if nothing would be mergeable until you had more time, and now you do, I don't have much.

Anyway, on to the matter at hand, I don't get the crash, but I'm missing some commits in the 3.6 branch, and also I still have one of my proposed fixes. I'm making things more upstream-like, and I'll add another sentence once everything's rebuilt.

So after waiting an age for everything to rebuild, I got reminded that the occlusion query API got changed by Julien Valentine's recent PR. He made a PR for OpenMW that was supposed to resolve that, but obviously it didn't work until I tweaked it. Once I'd built everything, I tested it, and I'm not seeing the crash here. This is with a Debug build of OpenMW and OSG, and on Windows, and I don't think anyone else is using debug builds of both, especially not on Windows.

I guess that means that I've not really given any more information beyond this not being something that happens for everyone. It might be dependent on other factors, so a more detailed description of how reliable the crash is and whether it's dependent on anything is needed before anyone can do anything on OpenMW's end.

One thing of note is that the OpenMW profiler doesn't use the default font (at least on my machine). It uses a truetype one. I seem to remember seeing it use the default font in the past, and it's not impossible that this is toggled via a setting I've forgotten about, but I've had a good look and can't find one.

So that's what I know.

Chris

[Fabian Roth]

Hi,

Thank you for the fast reply.

My build is using static osg, static osg-plugins and link time optimization.

I created an address sanitizer enabled build.

It exhibits a heap-use-after-free.

I will try to further investigate this week.

Greetings,

Fabian

=================================================================
==11872==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030000082c0 at pc 0x55b4b9659551 bp 0x7ffdf8a9c190 sp 0x7ffdf8a9c180
READ of size 8 at 0x6030000082c0 thread T0
    #0 0x55b4b9659550 in OpenThreads::ScopedPointerLock<OpenThreads::Mutex>::ScopedPointerLock(OpenThreads::Mutex*) ./openmw/extern-git/OpenSceneGraph/include/OpenThreads/ScopedLock:54
    #1 0x55b4b9659550 in osg::StateAttribute::removeParent(osg::StateSet*) ./openmw/extern-git/OpenSceneGraph/src/osg/StateAttribute.cpp:38
    #2 0x55b4b965a033 in osg::StateSet::clear() ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:734
    #3 0x55b4b965a5ef in __base_dtor  ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:285
    #4 0x55b4b965a9f8 in __deleting_dtor  ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:286
    #5 0x55b4b9c98246 in osg::Referenced::unref() const ./openmw/extern-git/OpenSceneGraph/include/osg/Referenced:201
    #6 0x55b4b9c98246 in osg::ref_ptr<osg::StateSet>::~ref_ptr() ./openmw/extern-git/OpenSceneGraph/include/osg/ref_ptr:41
    #7 0x55b4b9c98246 in void std::_Destroy<osg::ref_ptr<osg::StateSet> >(osg::ref_ptr<osg::StateSet>*) /usr/include/c++/7/bits/stl_construct.h:98
    #8 0x55b4b9c98246 in void std::_Destroy_aux<false>::__destroy<osg::ref_ptr<osg::StateSet>*>(osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet>*) /usr/include/c++/7/bits/stl_construct.h:108
    #9 0x55b4b9c98246 in void std::_Destroy<osg::ref_ptr<osg::StateSet>*>(osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet>*) /usr/include/c++/7/bits/stl_construct.h:137
    #10 0x55b4b9c98246 in void std::_Destroy<osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet> >(osg::ref_ptr<osg::StateSet>*, osg::ref_ptr<osg::StateSet>*, std::allocator<osg::ref_ptr<osg::StateSet> >&) /usr/include/c++/7/bits/stl_construct.h:206
    #11 0x55b4b9c98246 in std::vector<osg::ref_ptr<osg::StateSet>, std::allocator<osg::ref_ptr<osg::StateSet> > >::~vector() [clone .lto_priv.5218] /usr/include/c++/7/bits/stl_vector.h:434
    #12 0x55b4b9da20dd in osgText::Font::~Font() ./openmw/extern-git/OpenSceneGraph/src/osgText/Font.cpp:295
    #13 0x55b4b9e59ed2 in __base_dtor  ./openmw/extern-git/OpenSceneGraph/src/osgText/DefaultFont.cpp:35
    #14 0x55b4b9e59ed2 in __deleting_dtor  ./openmw/extern-git/OpenSceneGraph/src/osgText/DefaultFont.cpp:37
    #15 0x55b4b8df93d6 in osg::Referenced::unref() const ./openmw/extern-git/OpenSceneGraph/include/osg/Referenced:201
    #16 0x55b4b98e7518 in osg::ref_ptr<osg::Object>::~ref_ptr() ./openmw/extern-git/OpenSceneGraph/include/osg/ref_ptr:41
    #17 0x55b4b98e7518 in std::pair<osg::ref_ptr<osg::Object>, double>::~pair() /usr/include/c++/7/bits/stl_pair.h:208
    #18 0x55b4b98e7518 in __base_dtor  /usr/include/c++/7/bits/stl_pair.h:208
    #19 0x55b4b98e7518 in void __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::destroy<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >(std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >*) /usr/include/c++/7/ext/new_allocator.h:140
    #20 0x55b4b98e7518 in void std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > > >::destroy<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >(std::allocator<std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >&, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >*) /usr/include/c++/7/bits/alloc_traits.h:487
    #21 0x55b4b98e7518 in std::_Rb_tree<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >, std::_Select1st<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::_M_destroy_node(std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >*) /usr/include/c++/7/bits/stl_tree.h:650
    #22 0x55b4b98e7518 in std::_Rb_tree<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >, std::_Select1st<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::_M_drop_node(std::_Rb_tree_node<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >*) /usr/include/c++/7/bits/stl_tree.h:658
    #23 0x55b4b98e7518 in _M_erase /usr/include/c++/7/bits/stl_tree.h:1858
    #24 0x55b4b990fe4f in std::_Rb_tree<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> >, std::_Select1st<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > >, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::clear() /usr/include/c++/7/bits/stl_tree.h:1171
    #25 0x55b4b990fe4f in std::map<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> >, std::pair<osg::ref_ptr<osg::Object>, double>, osgDB::ObjectCache::ClassComp, std::allocator<std::pair<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, osg::ref_ptr<osgDB::Options const> > const, std::pair<osg::ref_ptr<osg::Object>, double> > > >::clear() /usr/include/c++/7/bits/stl_map.h:1127
    #26 0x55b4b990fe4f in osgDB::ObjectCache::clear() ./openmw/extern-git/OpenSceneGraph/src/osgDB/ObjectCache.cpp:189
    #27 0x55b4b9914418 in clearObjectCache ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:1654
    #28 0x55b4b9914418 in osgDB::Registry::destruct() ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:507
    #29 0x55b4b9917574 in __base_dtor  ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:486
    #30 0x55b4b9918268 in __deleting_dtor  ./openmw/extern-git/OpenSceneGraph/src/osgDB/Registry.cpp:487
    #31 0x55b4b8df93d6 in osg::Referenced::unref() const ./openmw/extern-git/OpenSceneGraph/include/osg/Referenced:201
    #32 0x7fde57502040  (/lib/x86_64-linux-gnu/libc.so.6+0x43040)
    #33 0x7fde57502139 in exit (/lib/x86_64-linux-gnu/libc.so.6+0x43139)
    #34 0x7fde574e0b9d in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b9d)
    #35 0x55b4b872bf79 in _start (.//openmw-build/openmw+0x3e9f79)

0x6030000082c0 is located 0 bytes inside of 24-byte region [0x6030000082c0,0x6030000082d8)
freed by thread T0 here:
    #0 0x7fde579919d8 in operator delete(void*, unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe19d8)
    #1 0x7fde57502040  (/lib/x86_64-linux-gnu/libc.so.6+0x43040)

previously allocated by thread T0 here:
    #0 0x7fde57990458 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xe0458)
    #1 0x55b4b9658949 in osg::Referenced::getGlobalReferencedMutex() ./openmw/extern-git/OpenSceneGraph/src/osg/Referenced.cpp:86

SUMMARY: AddressSanitizer: heap-use-after-free ./openmw/extern-git/OpenSceneGraph/include/OpenThreads/ScopedLock:54 in OpenThreads::ScopedPointerLock<OpenThreads::Mutex>::ScopedPointerLock(OpenThreads::Mutex*)
Shadow bytes around the buggy address:
  0x0c067fff9000: fd fd fa fa fa fa fa fa fa fa fd fd fd fd fa fa
  0x0c067fff9010: fa fa fa fa fa fa fd fd fd fd fa fa fa fa fa fa
  0x0c067fff9020: fa fa fd fd fd fd fa fa fa fa fa fa fa fa fd fd
  0x0c067fff9030: fd fd fa fa fa fa fa fa fa fa fd fd fd fa fa fa
  0x0c067fff9040: fa fa fa fa fa fa fd fd fd fa fa fa fa fa fa fa
=>0x0c067fff9050: fa fa fd fd fd fd fa fa[fd]fd fd fa fa fa fd fd
  0x0c067fff9060: fd fa fa fa fa fa fa fa fa fa fd fd fd fa fa fa
  0x0c067fff9070: fa fa fa fa fa fa fd fd fd fa fa fa fa fa fa fa
  0x0c067fff9080: fa fa fd fd fd fa fa fa fa fa fa fa fa fa 00 00
  0x0c067fff9090: 05 fa fa fa 00 00 05 fa fa fa fa fa fa fa fa fa
  0x0c067fff90a0: 00 00 07 fa fa fa 00 00 07 fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==11872==ABORTING

[OpenSceneGraph Users]

Hi Chris,

On Mon, 27 Jan 2020 at 23:51, OpenSceneGraph Users <osg-...@lists.openscenegraph.org> wrote:

As I've mentioned in the past, I'm an OpenMW (note the single M) developer. It was actually me who reported the issues with the default font, and only a subset were resolved before you went on hiatus. I've lost the thread where we were discussing it as I'd bookmarked the forum thread, which is now dead. As I recall, though, my most recent minimal things-don't-get-relased-when-they-should example was resolvable by a couple of potential changes I suggested, but you said they'd have to wait until you could investigate more thoroughly, as it was in code no one had touched for a while, so could maybe upset other applications, or not exhaustively remove all such bugs.

The googlegroup has search options, here's what I get if I search for OpenMW. it comes up with several threads with you contributing:

https://groups.google.com/forum/#!searchin/osg-users/OpenMW%7Csort:date

 

I pretty much stopped trying to make OpenMW play nicely with 3.6.x when you disappeared as there wasn't much point if nothing would be mergeable until you had more time, and now you do, I don't have much.

I had to step back from the OSG support side as it consumes so much time and mind-share, it's not really possible to tackle other complex work at the same time so I'm having to block the OSG support side and not tackle any complex other work during this period. I don't disappear completely, I just step back.

Anyway, on to the matter at hand, I don't get the crash, but I'm missing some commits in the 3.6 branch, and also I still have one of my proposed fixes. I'm making things more upstream-like, and I'll add another sentence once everything's rebuilt.

If you have things you feel would be suitable to merge with the 3.6 branch please make them. I can't review and provide feedback.  To properly judge changes I do need to understand the motivation behind them, it may be that the changes are workaround issues that are based solved in other ways.

 

So after waiting an age for everything to rebuild, I got reminded that the occlusion query API got changed by Julien Valentine's recent PR. He made a PR for OpenMW that was supposed to resolve that, but obviously it didn't work until I tweaked it. Once I'd built everything, I tested it, and I'm not seeing the crash here. This is with a Debug build of OpenMW and OSG, and on Windows, and I don't think anyone else is using debug builds of both, especially not on Windows.

When you say not seeing the crash, I we talking about a different issue to what Fabian was referring to?  Is Fabian working on the same versions of OpenME and OSG as yourself?

I guess that means that I've not really given any more information beyond this not being something that happens for everyone. It might be dependent on other factors, so a more detailed description of how reliable the crash is and whether it's dependent on anything is needed before anyone can do anything on OpenMW's end.

One thing of note is that the OpenMW profiler doesn't use the default font (at least on my machine). It uses a truetype one. I seem to remember seeing it use the default font in the past, and it's not impossible that this is toggled via a setting I've forgotten about, but I've had a good look and can't find one.

The OSG falls back to using the DefaultFont when the requested font can't be found, so any chance this might be happening?  Something like font files missing, case of the font being different - Windows isn't case sensitive so when you move to a case sensitive OS you can see issues if there are errors in the filename.

Cheers,

Robert.

[OpenSceneGraph Users]

Hi Fabian,

 

My build is using static osg, static osg-plugins and link time optimization.

I created an address sanitizer enabled build.

It exhibits a heap-use-after-free.

I will try to further investigate this week.

=================================================================
==11872==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030000082c0 at pc 0x55b4b9659551 bp 0x7ffdf8a9c190 sp 0x7ffdf8a9c180
READ of size 8 at 0x6030000082c0 thread T0
    #0 0x55b4b9659550 in OpenThreads::ScopedPointerLock<OpenThreads::Mutex>::ScopedPointerLock(OpenThreads::Mutex*) ./openmw/extern-git/OpenSceneGraph/include/OpenThreads/ScopedLock:54
    #1 0x55b4b9659550 in osg::StateAttribute::removeParent(osg::StateSet*) ./openmw/extern-git/OpenSceneGraph/src/osg/StateAttribute.cpp:38
    #2 0x55b4b965a033 in osg::StateSet::clear() ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:734

Given the stack trace it kinda looks like the getRefMutex() call in StateAttribute.cpp is the where things might be going astray (note the comment I've added below):

void StateAttribute::removeParent(osg::StateSet* object)
{
    OpenThreads::ScopedPointerLock<OpenThreads::Mutex> lock(getRefMutex()); // calls the base classes Referenced::getRefMutex() method that will map to Referenced::getGlobalReferencedMutex

    ParentList::iterator pitr = std::find(_parents.begin(),_parents.end(),object);
    if (pitr!=_parents.end()) _parents.erase(pitr);
}

The Referenced::getGlobalReferencedMutex() implementation in Referenced.cpp is:

OpenThreads::Mutex* Referenced::getGlobalReferencedMutex()
{
    static GlobalMutexPointer s_ReferencedGlobalMutext = new OpenThreads::Mutex;
    return s_ReferencedGlobalMutext.get();
}

// helper class for forcing the global mutex to be constructed when the library is loaded.
struct InitGlobalMutexes
{
    InitGlobalMutexes()
    {
        Referenced::getGlobalReferencedMutex();
    }
};
static InitGlobalMutexes s_initGlobalMutexes;

Which is all a bit hacky way of trying to get a singleton's _ReferencedGlobalMutext to construct before any other code calling getGlobalReferencedMutex() gets called.

I don't really know why a pointer is even being used here, it's not how I'd write the code these days, but off the top of my head don't recall the derivation and motivations between all this code as it dates back to the earliest days of the OSG project, so almost two decades :-)

What I'd write today would simply be:

static OpenThreads::Mutex s_ReferencedGlobalMutex;
OpenThreads::Mutex* Referenced::getGlobalReferencedMutex()
{
    return &s_ReferencedGlobalMutex;
}

You could try substituting this in.  I will try a build here just to make sure the above works fine for standard OSG work.  I don't expect this change to have any affect on your own code, if it does it suggest there is some issue with order of clean up of statics.

Robert.

[OpenSceneGraph Users]

Hi Fabian & Chris,

I was curious about the clean up of the global getGlobalReferencedMutex() so I added some debug messages to OpenThreads and to relevant calls in the OSG to track the creation and clean up of mutexes.  I tried an alternative means of static initialization of the static getGlobalReferencedMutex() and got the same behavior for before and after results so I don't think the version in the 3.6 branch is likley to be be a source of problem, even if it's not particular clean.  Below is diff of the changes I made.

Robert

$ git diff
diff --git a/src/OpenThreads/pthreads/PThreadMutex.cpp b/src/OpenThreads/pthreads/PThreadMutex.cpp
index 3a3d1c338..d122fc67c 100644
--- a/src/OpenThreads/pthreads/PThreadMutex.cpp
+++ b/src/OpenThreads/pthreads/PThreadMutex.cpp
@@ -22,6 +22,8 @@
 #include <OpenThreads/Mutex>
 #include "PThreadMutexPrivateData.h"
 
+#include <stdio.h>
+
 using namespace OpenThreads;
 
 //----------------------------------------------------------------------------
@@ -33,7 +35,7 @@ using namespace OpenThreads;
 Mutex::Mutex(MutexType type):
     _mutexType(type)
 {
-
+    printf("Mutex::Mutex(%d) %p\n", type, this);
     pthread_mutexattr_t mutex_attr;
     pthread_mutexattr_init( &mutex_attr );
     
@@ -107,6 +109,8 @@ Mutex::Mutex(MutexType type):
 //
 Mutex::~Mutex() {
 
+    printf("Mutex::~Mutex() %p\n", this);
+
     PThreadMutexPrivateData *pd =
         static_cast<PThreadMutexPrivateData*>(_prvData);
 
diff --git a/src/osg/Referenced.cpp b/src/osg/Referenced.cpp
index 95b665c57..267bda310 100644
--- a/src/osg/Referenced.cpp
+++ b/src/osg/Referenced.cpp
@@ -79,11 +79,16 @@ struct ResetPointer
 };
 
 typedef ResetPointer<DeleteHandler> DeleteHandlerPointer;
+
+#if 1
+#include <stdio.h>
+
 typedef ResetPointer<OpenThreads::Mutex> GlobalMutexPointer;

 
 OpenThreads::Mutex* Referenced::getGlobalReferencedMutex()
 {
     static GlobalMutexPointer s_ReferencedGlobalMutext = new OpenThreads::Mutex;

+    printf("Referenced::getGlobalReferencedMutex() %p\n", (s_ReferencedGlobalMutext.get()));
     return s_ReferencedGlobalMutext.get();
 }
 
@@ -96,6 +101,17 @@ struct InitGlobalMutexes
     }
 };
 static InitGlobalMutexes s_initGlobalMutexes;
+#else
+
+#include <stdio.h>
+
+static OpenThreads::Mutex s_ReferencedGlobalMutex;
+OpenThreads::Mutex* Referenced::getGlobalReferencedMutex()
+{
+    printf("Referenced::getGlobalReferencedMutex() %p\n", (&s_ReferencedGlobalMutex));
+    return &s_ReferencedGlobalMutex;
+}
+#endif
 
 // static std::auto_ptr<DeleteHandler> s_deleteHandler(0);
 static DeleteHandlerPointer s_deleteHandler(0);
diff --git a/src/osg/StateAttribute.cpp b/src/osg/StateAttribute.cpp
index e239fb3aa..c7df40894 100644
--- a/src/osg/StateAttribute.cpp
+++ b/src/osg/StateAttribute.cpp
@@ -39,6 +39,8 @@ void StateAttribute::removeParent(osg::StateSet* object)

 
     ParentList::iterator pitr = std::find(_parents.begin(),_parents.end(),object);
     if (pitr!=_parents.end()) _parents.erase(pitr);

+
+    printf("StateAttribute::removeParent()\n");
 }
 
 

[Chris Djali / AnyOldName3]

Hello again.

The googlegroup has search options, here's what I get if I search for OpenMW. it comes up with several threads with you contributing:

https://groups.google.com/forum/#!searchin/osg-users/OpenMW%7Csort:date

I can find chunks of the right thread (this is some: https://groups.google.com/forum/#!searchin/osg-users/OpenMW|sort:date/osg-users/4lwB0MZdPqM/-ohFXtp-CQAJ) but some seems to be missing. For example, this newer chunk shows up as a separate thread: https://groups.google.com/forum/#!searchin/osg-users/anyoldname3|sort:date/osg-users/lbFxUItJ_qc/KEkYK_1dAgAJ. It's reasonably likely that that's the end of the thread.

I don't disappear completely, I just step back.

I'm aware of the reasoning and that it's not total. It still has a pretty big impact on how much OSG work you actually get done, so I feel my word choice isn't that extreme.

If you have things you feel would be suitable to merge with the 3.6 branch please make them. I can't review and provide feedback.  To properly judge changes I do need to understand the motivation behind them, it may be that the changes are workaround issues that are based solved in other ways.

https://groups.google.com/d/msg/osg-users/lbFxUItJ_qc/c54xoN8oAAAJ has a diff that resolved my problem and I suggested went into 3.6. A few posts lower, there's what we both agreed was the source for a simple application that didn't work. You said you'd look into it in more detail later in the hope of finding other potential solutions. I can't do that for you. I put days into this and couldn't find any better solutions than the one I posted. That might have been because I didn't write OSG and am unaware of a nifty feature that will fix everything, because I'm not actually clever enough to conceive of the right approach, or because there's genuinely no other option. It's certainly not because I didn't do a thorough enough investigation.

When you say not seeing the crash, I we talking about a different issue to what Fabian was referring to?  Is Fabian working on the same versions of OpenME and OSG as yourself?

I was referring to the crash he was seeing with the profiler. It's still called OpenMW with one O and one W, by the way. My testing was with the 3.6 branch of OSG as of 9b41f260 and master branch OpenMW from a few weeks ago. I've also got minor local changes to adapt to the Julien Valentine occlusion query PR you merged recently. Nothing that touches anything relevant to Fabian's crash has been merged since then, so latest master branch OpenMW is a good analogue for what I'm using. I suspect Fabian isn't using the same OSG and OpenMW as me exactly, as he needs to either merge this https://github.com/OpenMW/openmw/pull/2676 (or something like it) or use an OSG revision from before this https://github.com/openscenegraph/OpenSceneGraph/pull/902 got merged.

The OSG falls back to using the DefaultFont when the requested font can't be found, so any chance this might be happening?  Something like font files missing, case of the font being different - Windows isn't case sensitive so when you move to a case sensitive OS you can see issues if there are errors in the filename.

I've checked, and we switched to using a bundled TTF font in the middle of 2018. Most development happens on Linux (and the person who switched the font over uses it exclusively) so the only way it wouldn't be finding the font would be if Fabian's doing something weird.

My build is using static osg, static osg-plugins and link time optimization.

For example, this could be the weird thing. We don't test statically linked OSG, although our semi-official Android port does things that way (it's not maintained by the core team, just one guy on his own, and it's got plenty of shortcomings). One possibility is that the Freetype plugin just got left out when building. Renaming the DLL so OpenMW couldn't see it did make it use the OSG default font in the profiler, but didn't allow me to reproduce Fabian's crash.

In summary:

• Fabian has done something weird with either OSG or OpenMW that hasn't been specified yet.
• It's beginning to feel like you're misspelling OpenMW deliberately.
• Regarding the as-yet unresolved default font/object cache not being released issue I reported in March, the ball was left in your court with nothing more I could do. Hopefully enough has been linked above that we can move forward with that again if you've got more time now.
• Without knowing what source code Fabian has built, I can't reproduce or identify the issue he's seeing.

Cheers,

Chris

[Fabian Roth]

Hi Robert,

Using your suggested changes i get a crash on start.

I forgot to mention i also link OpenThreads statically.

I am starting to suspect the static linking and optimization surfaces undefined behavior.

Greetings,

Fabian

ASAN:DEADLYSIGNAL
=================================================================
==19668==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x5597ebadb5ac bp 0x60c000000b80 sp 0x7ffce8efbba0 T0)
==19668==The signal is caused by a READ memory access.
==19668==Hint: address points to the zero page.
    #0 0x5597ebadb5ab in OpenThreads::ScopedPointerLock<OpenThreads::Mutex>::ScopedPointerLock(OpenThreads::Mutex*) ./openmw/extern-git/OpenSceneGraph/include/OpenThreads/ScopedLock:54
    #1 0x5597ebadb5ab in addParent ./openmw/extern-git/OpenSceneGraph/src/osg/StateAttribute.cpp:31
    #2 0x5597ebadbc84 in setAttribute ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:1784
    #3 0x5597ebadc737 in osg::StateSet::setAttributeAndModes(osg::StateAttribute*, unsigned int) [clone .part.309] ./openmw/extern-git/OpenSceneGraph/src/osg/StateSet.cpp:1076
    #4 0x5597ebcb7241 in __base_ctor  ./openmw/extern-git/OpenSceneGraph/src/osgUtil/RenderBin.cpp:174
    #5 0x5597ebcb7a37 in __base_ctor  ./openmw/extern-git/OpenSceneGraph/src/osgUtil/RenderBin.cpp:37
    #6 0x5597ebcb7a37 in renderBinPrototypeList ./openmw/extern-git/OpenSceneGraph/src/osgUtil/RenderBin.cpp:53
    #7 0x5597eab5bacb in RenderBinSingletonProxy::RenderBinSingletonProxy() ./openmw/extern-git/OpenSceneGraph/src/osgUtil/RenderBin.cpp:58
    #8 0x5597eab5bacb in __static_initialization_and_destruction_0 ./openmw/extern-git/OpenSceneGraph/src/osgUtil/RenderBin.cpp:58
    #9 0x5597eab5bacb in _GLOBAL__sub_I__ZN7osgUtil9RenderBin21getRenderBinPrototypeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE.lto_priv.3181 ./openmw/extern-git/OpenSceneGraph/src/osgUtil/RenderBin.cpp:665
    #10 0x5597eabb1163 in global constructors keyed to 65535_0_objects.cpp.o.6481610 (./openmw-build/openmw+0x3b5163)
    #11 0x5597ec775bcc in __libc_csu_init (./openmw-build/openmw+0x1f79bcc)
    #12 0x7f213df67b27 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b27)
    #13 0x5597eabe6039 in _start (./openmw-build/openmw+0x3ea039)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ./openmw/extern-git/OpenSceneGraph/include/OpenThreads/ScopedLock:54 in OpenThreads::ScopedPointerLock<OpenThreads::Mutex>::ScopedPointerLock(OpenThreads::Mutex*)
==19668==ABORTING

[Chris Djali / AnyOldName3]

Hi Fabian,

Link-time optimisation should be fine - we do it on release builds with no problems. It's either something you've changed, or it's the static linking.

We still don't know exactly which version of OpenMW and OSG you've built, though. It's pretty obviously not the RC this thread is discussing if you're using an existing OpenMW release or something you got from the master branch with no local changes, as OpenMW literally won't build without unofficial changes yet. Can you get back to us with a specific answer?

Cheers,

Chris

[Fabian Roth]

Hi Chris,

I am using the latest openmw master with the compatibility patch from the pull request cherry picked, my build changes and minor other tweaks.

I use the osg rc with a only a cmake version change.

The branches are here:

https://github.com/Eli2/openmw/tree/eli2-openmw-static

https://github.com/Eli2/OpenSceneGraph/tree/eli2-openmw-static

By now i strongly suspect i run into the "Static Initialization Order Fiasco", as described here:

https://cryptopp.com/wiki/Static_Initialization_Order_Fiasco

I am currently looking into why my build uses the default font.

Greetings,

Fabian

[Robert Osfield]

Hi Fabian,

On Wednesday, 29 January 2020 00:24:35 UTC, Fabian Roth wrote:

Hi Chris,

I am using the latest openmw master with the compatibility patch from the pull request cherry picked, my build changes and minor other tweaks.

I use the osg rc with a only a cmake version change.

The branches are here:

https://github.com/Eli2/openmw/tree/eli2-openmw-static

https://github.com/Eli2/OpenSceneGraph/tree/eli2-openmw-static

By now i strongly suspect i run into the "Static Initialization Order Fiasco", as described here:

https://cryptopp.com/wiki/Static_Initialization_Order_Fiasco

I am currently looking into why my build uses the default font.

The way that the singleton methods are done is an attempt to resolve the construction order issue, but l don't recall a focus on the destruction side.  It could well be an issue.

One workaround might be to add a mutex directly to the Font objects that are being destructed rather than using the global one.  Or, to explicitly clear the ObjectCache on destruction rather than leaving it to static clean up.  The later would be my preferred approach.

A call to:

   osgDB::Registry::instance()->getObjectCache()->clear();

In application clean up should be sufficient, or force the destruction of the Registry singleton:

   osgDB::Registry::instance(true);   // passing in true forces the destruction, yes a bit hacky... the OSG has a long history so can't recall the circumstances of that addition...

I will hold the 3.6.5 release back till we have some conclusion on this issue, in case we need to make changes to the core OSG.

Cheers,

Robert.

 

[OpenSceneGraph Users]

Hi Chris,

Thanks the links.  I've tracked down the example you created and re-run it on my system and on the scene graph creation of the second window/view I get text without textures. 

In summary:

• Fabian has done something weird with either OSG or OpenMW that hasn't been specified yet.

If the codebase is the same perhaps it comes down to a sensitivity to compiler version or dependencies?

• It's beginning to feel like you're misspelling OpenMW deliberately.

Sigh.  I likely have dyslexia, while I've never been formally diagnosed I have the traits, the downside is that I regularily get letters wrong, don't spot mispellings.  This isn't personal, it's just an issue I have to deal with, and unfortunately as I read/wrote code and read/write email the community also have to accept that I don't get everything right every time.  In other ways my brain functions pretty well so overall I can still be productive.

• Regarding the as-yet unresolved default font/object cache not being released issue I reported in March, the ball was left in your court with nothing more I could do. Hopefully enough has been linked above that we can move forward with that again if you've got more time now.

If this is the one that the attached example recreates then I will be looking into this today.

 

[Chris Djali / AnyOldName3]

Hi Robert,

I'm reasonably sure that Fabian's crash isn't the same issue as that example exposes.

• Fabian has done something weird with either OSG or OpenMW that hasn't been specified yet.

If the codebase is the same perhaps it comes down to a sensitivity to compiler version or dependencies?

We have official builds for a variety of Linux distros, Windows and MacOS (and semi-official builds for Android that are generally less reliable) so we should have pretty good coverage with known-good dependency and tooling versions. As I've mentioned before, the most likely culprit for this suddenly appearing for Fabian is that we pretty much never link to OSG statically. It's a nightmare on Windows (I don't think anyone's even attempted it in the last five years) but is more feasible on Linux, where the majority of our contributors are, so I've put out a call for other testers to try and reproduce the problem.

Also, Robert, I'm assuming you don't have a copy of Morrowind to test OpenMW with. Right now, Steam, GreenManGaming and Fanatical are all offering it for less than £4, but at least one of those sales ends in less than twenty hours. If you're not keen, £4 is a reasonable investment for me to make to increase cooperation between our projects, but it would help if you got back to me quickly.

Thanks,

Chris

[OpenSceneGraph Users]

Hi Robert,

In relation to the DefaultFont crash issue, I noticed that my code would occasionally crash on creation of osgText::Text.

Most of my osgText::Text is not created on the main thread.

In order to avoid the DefaultFont crash, I created a

osg::ref_ptr<osgText::Font> necessaryFont = osgText::Font::getDefaultFont();
which sticks around from the beginning of the program to the end and doesn't get used by anything.
After I did this, my code no longer crashed on osgText::Text creation.
The OpenSceneGraph version used is 3.6.4 and on both Windows (7 64-bit) and Linux (Ubuntu 16.04).
Also when I was previously using OpenSceneGraph version 3.6.2, osgText::Text creation never crashed.

Regards,
Anna

On Wed, Jan 29, 2020 at 5:15 PM OpenSceneGraph Users <osg-...@lists.openscenegraph.org> wrote:

Hi Robert,

I'm reasonably sure that Fabian's crash isn't the same issue as that example exposes.

• Fabian has done something weird with either OSG or OpenMW that hasn't been specified yet.

If the codebase is the same perhaps it comes down to a sensitivity to compiler version or dependencies?

We have official builds for a variety of Linux distros, Windows and MacOS (and semi-official builds for Android that are generally less reliable) so we should have pretty good coverage with known-good dependency and tooling versions. As I've mentioned before, the most likely culprit for this suddenly appearing for Fabian is that we pretty much never link to OSG statically. It's a nightmare on Windows (I don't think anyone's even attempted it in the last five years) but is more feasible on Linux, where the majority of our contributors are, so I've put out a call for other testers to try and reproduce the problem.

Also, Robert, I'm assuming you don't have a copy of Morrowind to test OpenMW with. Right now, Steam, GreenManGaming and Fanatical are all offering it for less than £4, but at least one of those sales ends in less than twenty hours. If you're not keen, £4 is a reasonable investment for me to make to increase cooperation between our projects, but it would help if you got back to me quickly.

Thanks,

Chris

On Wednesday, 29 January 2020 14:46:02 UTC, OpenSceneGraph Users wrote:

--
You received this message because you are subscribed to the Google Groups "OpenSceneGraph Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osg-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/73cd3f2d-7a89-4ded-b247-e3586cea02ca%40googlegroups.com.
_______________________________________________
osg-users mailing list
osg-...@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[Chris Djali / AnyOldName3]

Hi all,

I have new information about what Fabian has done that's weird. When he's been saying he's doing static linking, he's not doing what most people would associate that phrase with (i.e. building OSG to a library file such that all code is contained within and consuming that when linking OpenMW). Instead, it's more like what CMake calls an object library - he's building dependencies to intermediate-representation object files, but only doing translation to native code and linking once when everything's an object file. I wouldn't be surprised if this puts static initialisers and destructors in orders they've never been in before and that's what's causing the problem.

Hopefully this will point the investigation in the right direction.

Cheers,

Chris

To unsubscribe from this group and stop receiving emails from it, send an email to osg-...@googlegroups.com.

[OpenSceneGraph Users]

Hi Anna,

On Wed, 29 Jan 2020 at 22:38, OpenSceneGraph Users <osg-...@lists.openscenegraph.org> wrote:

In relation to the DefaultFont crash issue, I noticed that my code would occasionally crash on creation of osgText::Text.

Most of my osgText::Text is not created on the main thread.

In order to avoid the DefaultFont crash, I created a

osg::ref_ptr<osgText::Font> necessaryFont = osgText::Font::getDefaultFont();
which sticks around from the beginning of the program to the end and doesn't get used by anything.
After I did this, my code no longer crashed on osgText::Text creation.
The OpenSceneGraph version used is 3.6.4 and on both Windows (7 64-bit) and Linux (Ubuntu 16.04).
Also when I was previously using OpenSceneGraph version 3.6.2, osgText::Text creation never crashed.

This sounds like a bug somewhere in the initialization of the Font, to investigate I'll need to reproduce the problem on my system.

Does the multi-theaded test code path in osgtext fail for you as well:

       osgtext --mt

I have just tried this on my Kubunutu 18.04 system with the 3.6 branch and it works fine.

Could you create a small test program that illustrates what you are doing?

Cheers,

Robert.

[OpenSceneGraph Users]

I too have seen thread-related issues when creating Text from multiple threads. I was never able to find the time to debug it to conclusion; but I suspect that a shared Font object that needs to create new Glyph textures isn't doing so in a thread-safe manner. For example, Font::assignGlyphToGlyphTexture() modifies some structures without Mutex protection. Just something to check.

Glenn Waldron / osgEarth

[OpenSceneGraph Users]

Hi Chris,

I slowly closing in on the cause of the Font issue, currently it looks like the removeView() is behaving differently form the CompositeViewer destructor and not handling clean up of contexts correctly.  I need to refactor how things are done internally, but expect to have a solution checked in this afternoon.

This fix might remove the issue with the static initialization by cleaning up Font GL objects before the viewers are entirely destroyed and prior to static clean up.

On Wed, 29 Jan 2020 at 22:21, OpenSceneGraph Users <osg-...@lists.openscenegraph.org> wrote:

Also, Robert, I'm assuming you don't have a copy of Morrowind to test OpenMW with. Right now, Steam, GreenManGaming and Fanatical are all offering it for less than £4, but at least one of those sales ends in less than twenty hours. If you're not keen, £4 is a reasonable investment for me to make to increase cooperation between our projects, but it would help if you got back to me quickly.

Thanks for the tip.  I am not a gamer these days, way too many other things competing with my time.  I also have enough dev work that I really don't want to get sucked into another open source project, there really are too many projects that use the OSG for me to start providing direct support for these projects beyond what I can provide through the OSG forum/mailing list.  Please remember there is one of me vs many OSG users/projects.

Cheers,

Robert.

[Robert Osfield]

Hi Chris et. al,

On Thursday, 30 January 2020 14:39:08 UTC, OpenSceneGraph Users wrote:

I slowly closing in on the cause of the Font issue, currently it looks like the removeView() is behaving differently form the CompositeViewer destructor and not handling clean up of contexts correctly.  I need to refactor how things are done internally, but expect to have a solution checked in this afternoon.

I have now checked a fix to the CompositeViewer::removeView() bug where GL objects, including the Font, were not being cleaned up correctly.

     https://github.com/openscenegraph/OpenSceneGraph/commit/25868955d2214139722806d20d2c4b60e32d4a61

Cheers,

Robert.

[OpenSceneGraph Users]

Hi Robert,

Compiling and a few simple runs worked fine, using

windows 10 Enterprise 1909 18363,592

Visual Studio 15.9.19 

CMake 3.15.5

Regards, Laurens.

On Thu, Jan 30, 2020 at 5:30 PM OpenSceneGraph Users <osg-...@lists.openscenegraph.org> wrote:

Hi Chris et. al,

On Thursday, 30 January 2020 14:39:08 UTC, OpenSceneGraph Users wrote:

I slowly closing in on the cause of the Font issue, currently it looks like the removeView() is behaving differently form the CompositeViewer destructor and not handling clean up of contexts correctly.  I need to refactor how things are done internally, but expect to have a solution checked in this afternoon.

I have now checked a fix to the CompositeViewer::removeView() bug where GL objects, including the Font, were not being cleaned up correctly.

     https://github.com/openscenegraph/OpenSceneGraph/commit/25868955d2214139722806d20d2c4b60e32d4a61

Cheers,

Robert.

--

You received this message because you are subscribed to the Google Groups "OpenSceneGraph Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to osg-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/33fb547f-d61a-4c75-a3ec-e717ee1a6454%40googlegroups.com.

[Chris Djali / AnyOldName3]

Hi Robert,

That commit does indeed seem to have fixed my minimal reproducible example, and, unlike with my other minimal reproducible examples, it's fixed the issue with the OpenMW-CS, too. Thanks for getting that sorted.

Now I just have to remember what my favourite workaround for 3.4.1 was...

Cheers,

Chris

[Fabian Roth]

Hi Robert,

Thank you for the help.

Manually clearing the registry fixed my crash on exit.

I will continue testing the next rc, so far so good.

Greetings,

Fabian

[OpenSceneGraph Users]

Hi Anna & Glenm,

On Thu, 30 Jan 2020 at 13:00, OpenSceneGraph Users <osg-...@lists.openscenegraph.org> wrote:

I too have seen thread-related issues when creating Text from multiple threads. I was never able to find the time to debug it to conclusion; but I suspect that a shared Font object that needs to create new Glyph textures isn't doing so in a thread-safe manner. For example, Font::assignGlyphToGlyphTexture() modifies some structures without Mutex protection. Just something to check.

Thanks for the suggestion.  I've done a code review of the Glyph/GlyphTexture/Font code and have added a mutex lock to the Font::assignGlyphToGlyphTexture().  There are already mutex locks elsewhere, but there may be a need to add more.  The change is checked into the 3.6 branch and master:

    https://github.com/openscenegraph/OpenSceneGraph/commit/2e0472ba7e05e680a9a7e0bc7676d4e5a12eefa5

I have tested by running all the OSG examples and running osgtext --mt, and don't see any regressions.  I can't confirm that the change has fixed the crashes that you both have seen as I haven't got any example that reproduces it.  Could you test the 3.6 branch and let me know how you get on. 

This fix isn't in 3.6.5-rc3. but will be part of the final 3.6.5 stable release.

Cheers,

Robert.