Help with crash bugs - Freelance gig for Haxe / OpenFL programmers

[Jiri Kupiainen]

Nicolas suggested I post here about a dilemma we're facing, so here goes.

We're preparing for the full launch of our game The Masterplan soon, after a bit over half a year spent in Steam Early Access. We're doing our best to get good visibility for the launch, and hopefully it will also bring some more positive attention for Haxe and the community.

Everything looks good for the launch, except for the fact that our early access customers are experiencing an uncomfortably high number of crash bugs - which we'll need to eliminate before the full launch.

In short, we are looking for people interested in helping us get rid of these crash bugs. We're dealing with two categories of bugs: ones that leave a stack trace, and ones that don't. The latter category is especially what we need help with.

If you are (or someone you know might be) interested in a paid Haxe programming gig, please reply here or email me at ji...@sharkpunch.com!

The game is built on top of OpenFL and Haxe. CPP target on Windows, OSX and Linux.

Any help in getting rid of these bugs is very much appreciated!

[David Elahee]

Hi jiri

When do you expect/want to release ?

David.

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/d/optout.

[Lars Doucet]

Hey all,

Afraid I can't volunteer for this myself, but I would like to mention that I'm interested in improving crashdumper with any intelligence you uncover in this process. If whoever ends up working on this initiative is able to figure out why certain kinds of crashes don't generate a stack trace or result in "hard" crashes (the kind that don't send a proper signal that crashdumper can deetect), please do send me a ping. In a perfect world crashdumper would be able to catch absolutely everything that goes wrong with a program.

[Lars Doucet]

Incidentally -- one possible tip. I'm not sure if you do a lot of direct BitmapData manipulations or not, but those make up more than 50% of my hard crash (no stack trace, no crashdumper catch) events in Defender's Quest HD. Usually these are simple out-of-bounds errors on the bitmap data bounds, or using a disposed or otherwise invalid bitmap data.

[TiagoLr]

Hey, i might be available to help, what kind of bugs is it? Openfl/lime bugs, platform specific bugs or is it game engine bugs? Send me a pm, thanks Lars for tweeting this, really good looking game.

[Jiri Kupiainen]

We're aiming for early June. Not announced yet, so shhh... :)

[Leo Lännenmäki]

One of the more frequent crashes that we have seems to be related to the playback of sounds. We are running -Dlegacy OpenFL (currently testing on OpenFL 3.0.3 and lime 2.3.3 and hxcpp 3.2.94).

Whenever one of these sound related crashes happen on Mac we get these kinds of lines to stdout.

OpenAL error was raised: 40965
OpenAL error was raised: 40965
OpenAL error was raised: 40965
OpenAL error was raised: 40961
OpenAL error was raised: 40961
OpenAL error was raised: 40965
OpenAL error was raised: 40961
OpenAL error was raised: 40961

These map to #define AL_OUT_OF_MEMORY 0xA005 and #define AL_INVALID_NAME 0xA001. The stack traces usually look somewhat like the following:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread

0 lime.ndll 0x00000001059192c9 SDL_DYNAPI_entry + 619449

1 lime.ndll 0x0000000105919717 SDL_DYNAPI_entry + 620551

2 lime.ndll 0x0000000105616d42 nme_gl_get_tex_parameter__2 + 13554

3 lime.ndll 0x0000000105616c54 nme_gl_get_tex_parameter__2 + 13316

4 lime.ndll 0x0000000105614b7b nme_gl_get_tex_parameter__2 + 4907

5 lime.ndll 0x0000000105616408 nme_gl_get_tex_parameter__2 + 11192

6 lime.ndll 0x000000010566a6f5 nme_sound_channel_set_pitch__2 + 277

7 com.sharkpunch.themasterplan 0x0000000101e7ab18 ExternalPrimitive::__run(Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&) + 584 (Lib.cpp:155)

8 com.sharkpunch.themasterplan 0x0000000100754fd0 Dynamic::operator()(Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&) + 128 (Dynamic.h:187)

9 com.sharkpunch.themasterplan 0x0000000100f40b0c openfl::_v2::media::SoundChannel_obj::__construct(Dynamic, hx::Null<double>, hx::Null<int>, hx::ObjectPtr<openfl::media::SoundTransform_obj>) + 812 (SoundChannel.cpp:89)

10 com.sharkpunch.themasterplan 0x0000000100f40de8 openfl::_v2::media::SoundChannel_obj::__new(Dynamic, hx::Null<double>, hx::Null<int>, hx::ObjectPtr<openfl::media::SoundTransform_obj>) + 200 (SoundChannel.cpp:109)

11 com.sharkpunch.themasterplan 0x0000000100f4b8eb openfl::_v2::media::Sound_obj::play(hx::Null<double>, hx::Null<int>, hx::ObjectPtr<openfl::media::SoundTransform_obj>) + 1931 (Sound.cpp:413)

12 com.sharkpunch.themasterplan 0x0000000100d9e050 sharkpunch::engine::audio::AudioSourceComponent_obj::playActually() + 736 (AudioSourceComponent.cpp:118)

13 com.sharkpunch.themasterplan 0x0000000100d92f3b sharkpunch::engine::audio::AudioSystem_obj::updateNode(Dynamic, double) + 3067 (AudioSystem.cpp:300)

14 com.sharkpunch.themasterplan 0x0000000101d365df ash::tools::__ListIteratingSystem_objupdateNode(hx::Object*, Dynamic const&, Dynamic const&) + 95 (ListIteratingSystem.cpp:113)

15 com.sharkpunch.themasterplan 0x0000000101eb335f hx::CMemberFunction2::__run(Dynamic const&, Dynamic const&) + 79 (DynamicImpl.h:226)

16 com.sharkpunch.themasterplan 0x000000010000e190 Dynamic::operator()(Dynamic const&, Dynamic const&) + 112 (Dynamic.h:187)

17 com.sharkpunch.themasterplan 0x0000000101d374f1 ash::tools::ListIteratingSystem_obj::update(double) + 497 (ListIteratingSystem.cpp:248)

18 com.sharkpunch.themasterplan 0x0000000100d922bf sharkpunch::engine::audio::AudioSystem_obj::update(double) + 607 (AudioSystem.cpp:230)

19 com.sharkpunch.themasterplan 0x0000000101d6b994 ash::core::Engine_obj::update(double) + 1188 (Engine.cpp:928)

20 com.sharkpunch.themasterplan 0x0000000101dd61c9 sharkpunch::engine::Game_obj::update(double) + 153 (Game.cpp:1430)

21 com.sharkpunch.themasterplan 0x0000000101dd636a sharkpunch::engine::__Game_objupdate(hx::Object*, Dynamic const&) + 74 (Game.cpp:1443)

22 com.sharkpunch.themasterplan 0x0000000101eb38c7 hx::CMemberFunction1::__run(Dynamic const&) + 71 (DynamicImpl.h:138)

23 com.sharkpunch.themasterplan 0x000000010000e6fd Dynamic::operator()(Dynamic const&) + 93 (Dynamic.h:187)

24 com.sharkpunch.themasterplan 0x0000000101d3a3c9 ash::signals::Signal1_obj::dispatch(Dynamic) + 345 (Signal1.cpp:57)

25 com.sharkpunch.themasterplan 0x0000000100e3c60a sharkpunch::engine::RenderTickProvider_obj::dispatchTick(hx::ObjectPtr<openfl::_v2::geom::Rectangle_obj>) + 410 (RenderTickProvider.cpp:170)

26 com.sharkpunch.themasterplan 0x0000000100e3c6a5 sharkpunch::engine::__RenderTickProvider_objdispatchTick(hx::Object*, Dynamic const&) + 85 (RenderTickProvider.cpp:176)

27 com.sharkpunch.themasterplan 0x0000000101eb38c7 hx::CMemberFunction1::__run(Dynamic const&) + 71 (DynamicImpl.h:138)

28 com.sharkpunch.themasterplan 0x000000010000e6fd Dynamic::operator()(Dynamic const&) + 93 (Dynamic.h:187)

29 com.sharkpunch.themasterplan 0x0000000100ff62ed openfl::_v2::display::DirectRenderer_obj::__onRender(Dynamic) + 861 (DirectRenderer.cpp:101)

30 com.sharkpunch.themasterplan 0x0000000100ff63a5 openfl::_v2::display::__DirectRenderer_obj__onRender(hx::Object*, Dynamic const&) + 85 (DirectRenderer.cpp:108)

31 com.sharkpunch.themasterplan 0x0000000101eb38c7 hx::CMemberFunction1::__run(Dynamic const&) + 71 (DynamicImpl.h:138)

32 com.sharkpunch.themasterplan 0x0000000101e36186 val_call1 + 134 (CFFI.cpp:607)

33 lime.ndll 0x000000010563ebeb nme_font_iterate_device_fonts__1 + 15403

34 lime.ndll 0x0000000105641467 nme_font_iterate_device_fonts__1 + 25767

35 lime.ndll 0x00000001056435b4 nme_font_iterate_device_fonts__1 + 34292

36 lime.ndll 0x000000010565a569 nme_stage_begin_render__2 + 105

37 com.sharkpunch.themasterplan 0x0000000101e7a32d ExternalPrimitive::__run(Dynamic const&) + 461 (Lib.cpp:134)

38 com.sharkpunch.themasterplan 0x000000010000e6fd Dynamic::operator()(Dynamic const&) + 93 (Dynamic.h:187)

39 com.sharkpunch.themasterplan 0x0000000100fc0561 openfl::_v2::display::Stage_obj::__render(bool) + 817 (Stage.cpp:1559)

40 com.sharkpunch.themasterplan 0x0000000100fb5ea1 openfl::_v2::display::Stage_obj::__checkRender() + 529 (Stage.cpp:521)

41 com.sharkpunch.themasterplan 0x0000000100fc013c openfl::_v2::display::Stage_obj::__pollTimers() + 268 (Stage.cpp:1524)

42 com.sharkpunch.themasterplan 0x0000000100fb6950 openfl::_v2::display::Stage_obj::__doProcessStageEvent(Dynamic) + 2544 (Stage.cpp:599)

43 com.sharkpunch.themasterplan 0x0000000100fb7f4f openfl::_v2::display::Stage_obj::__processStageEvent(Dynamic) + 207 (Stage.cpp:802)

44 com.sharkpunch.themasterplan 0x0000000100fb8039 openfl::_v2::display::__Stage_obj__processStageEvent(hx::Object*, Dynamic const&) + 89 (Stage.cpp:808)

45 com.sharkpunch.themasterplan 0x0000000101eb38c7 hx::CMemberFunction1::__run(Dynamic const&) + 71 (DynamicImpl.h:138)

46 com.sharkpunch.themasterplan 0x0000000101e36186 val_call1 + 134 (CFFI.cpp:607)

47 lime.ndll 0x0000000105659f6f nme_stage_set_next_wake__2 + 559

48 lime.ndll 0x0000000105642ed8 nme_font_iterate_device_fonts__1 + 32536

49 lime.ndll 0x0000000105605f14 0x105600000 + 24340

50 lime.ndll 0x0000000105605d78 0x105600000 + 23928

51 lime.ndll 0x0000000105659b78 nme_set_package__4 + 264

52 com.sharkpunch.themasterplan 0x0000000101e79e0d ExternalPrimitive::__Run(Array<Dynamic> const&) + 589 (Lib.cpp:171)

53 com.sharkpunch.themasterplan 0x0000000101eaccb2 Dynamic::operator()(Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&) + 226 (DynamicImpl.h:542)

54 com.sharkpunch.themasterplan 0x0000000101e119ed openfl::_v2::Lib_obj::create(Dynamic, int, int, hx::Null<double>, hx::Null<int>, hx::Null<int>, String, hx::ObjectPtr<openfl::_v2::display::BitmapData_obj>, hx::ObjectPtr<hx::Class_obj>) + 2061 (Lib.cpp:321)

55 com.sharkpunch.themasterplan 0x0000000101e2e879 ApplicationMain_obj::main() + 649 (ApplicationMain.cpp:165)

56 com.sharkpunch.themasterplan 0x0000000101e3372c main + 60 (__main__.cpp:12)

57 com.sharkpunch.themasterplan 0x0000000100000974 start + 52

On Windows I have not yet been able to capture this exact stack, but this crash might be somehow related:

Worker thread:

The Masterplan.exe!MyMutex::Lock() Line 124 + 0x14 bytes C++

The Masterplan.exe!TAutoLock<MyMutex>::TAutoLock<MyMutex>(MyMutex & inMutex={...}) Line 215 + 0x19 bytes C++

The Masterplan.exe!LocalAllocator::ExitGCFreeZone() Line 2804 + 0xe bytes C++

The Masterplan.exe!hx::ExitGCFreeZone() Line 3139 C++

The Masterplan.exe!gc_exit_blocking() Line 884 C++

std.dll!56d81bf6()

The Masterplan.exe!ExternalPrimitive::__run(const Dynamic & a={...}) Line 134 + 0x11 bytes C++

The Masterplan.exe!Dynamic::operator()(const Dynamic & inArg0={...}) Line 187 + 0x2b bytes C++

The Masterplan.exe!Sys_obj::sleep(double seconds=0.20000000000000001) Line 88 C++

The Masterplan.exe!openfl::_legacy::media::SoundChannel_obj::__checkCompleteBackgroundThread() Line 405 + 0x15 bytes C++

The Masterplan.exe!openfl::_legacy::media::__SoundChannel_obj__checkCompleteBackgroundThread() Line 417 + 0xd bytes C++

The Masterplan.exe!hx::CStaticFunction0::__run() Line 82 + 0xc bytes C++

The Masterplan.exe!hxThreadFunc(void * inInfo=0x1c08e63c) Line 260 C++

The Masterplan.exe!_callthreadstartex() Line 314 + 0x6 bytes C

The Masterplan.exe!_threadstartex(void * ptd=0x16045de8) Line 292 + 0x5 bytes C

vfbasics.dll!AVrfpStandardThreadFunction(void * Context=0x132ccfe0) Line 656 + 0x6 bytes C

kernel32.dll!76b13677()

ntdll.dll!77ce9f42()

ntdll.dll!77ce9f15()

Main thread:

>    The Masterplan.exe!malloc(unsigned int size=3229860610)  Line 119 + 0x5 bytes    C

     d93308b8()   

Main thread after stepping one step forward:

>    The Masterplan.exe!BlockData::Init(int inGID=417)  Line 374 + 0xb bytes    C++

     The Masterplan.exe!GlobalAllocator::GetEmptyBlock(bool inTryCollect=true)  Line 1809    C++

     The Masterplan.exe!GlobalAllocator::GetRecycledBlock(int inRequiredRows=1)  Line 1715 + 0x28 bytes    C++

     The Masterplan.exe!LocalAllocator::Alloc(int inSize=12, bool inIsObject=true)  Line 2944 + 0x16 bytes    C++

     The Masterplan.exe!hx::Object::operator new(unsigned int inSize=12, hx::NewObjectType inType=NewObjContainer, const char * inName=0x00000000)  Line 3332    C++

     The Masterplan.exe!hx::StringData::operator new(unsigned int inSize=12, hx::NewObjectType inAlloc=NewObjContainer)  Line 1116 + 0x12 bytes    C++

     The Masterplan.exe!String::__ToObject()  Line 1199 + 0xb bytes    C++

     The Masterplan.exe!Dynamic::Dynamic(const String & inVal={...})  Line 295 + 0x2d bytes    C++

     The Masterplan.exe!ash::core::Engine_obj::update(double time=0.032000000000000001)  Line 940 + 0x26 bytes    C++

     The Masterplan.exe!sharkpunch::engine::Game_obj::update(double frameTime=0.032000000000000001)  Line 1431    C++

     The Masterplan.exe!sharkpunch::engine::__Game_objupdate(hx::Object * inObj=0x0ae71b84, const Dynamic & inArg0={...})  Line 1443 + 0x26 bytes    C++

     The Masterplan.exe!hx::CMemberFunction1::__run(const Dynamic & inArg0={...})  Line 138 + 0x1c bytes    C++

     The Masterplan.exe!Dynamic::operator()(const Dynamic & inArg0={...})  Line 187 + 0x2b bytes    C++

     The Masterplan.exe!ash::signals::Signal1_obj::dispatch(Dynamic object1={...})  Line 58    C++

     The Masterplan.exe!sharkpunch::engine::RenderTickProvider_obj::dispatchTick(hx::ObjectPtr<openfl::_legacy::geom::Rectangle_obj> _={...})  Line 171    C++

     The Masterplan.exe!sharkpunch::engine::__RenderTickProvider_objdispatchTick(hx::Object * inObj=0x127eaf98, const Dynamic & inArg0={...})  Line 176 + 0x26 bytes    C++

     The Masterplan.exe!hx::CMemberFunction1::__run(const Dynamic & inArg0={...})  Line 138 + 0x1c bytes    C++

     The Masterplan.exe!Dynamic::operator()(const Dynamic & inArg0={...})  Line 187 + 0x2b bytes    C++

     The Masterplan.exe!openfl::_legacy::display::DirectRenderer_obj::__onRender(Dynamic rect={...})  Line 103    C++

     The Masterplan.exe!openfl::_legacy::display::__DirectRenderer_obj__onRender(hx::Object * inObj=0x127e2e48, const Dynamic & inArg0={...})  Line 108 + 0x26 bytes    C++

     The Masterplan.exe!hx::CMemberFunction1::__run(const Dynamic & inArg0={...})  Line 138 + 0x1c bytes    C++

     The Masterplan.exe!val_call1(hx::Object * arg1=0x127ea9c4, hx::Object * arg2=0x7ead674c)  Line 607 + 0x24 bytes    C++

 

When this crash happened the process had been running around 3 hours and used 1226MB of memory and the system had over 10GB of memory free.

I'm not really sure on how to go about debugging these issues forward. These are related to memory, the latter might be related to to threads doing GC related things at the same time. I can see that in newer OpenFL versions have something called 'disable_legacy_audio', https://github.com/openfl/openfl/search?utf8=%E2%9C%93&q=disable_legacy_audio which might be useful but I'm not sure. Any ideas?

Also, is there some kind of 32 sound limitation, see http://docs.openfl.org/openfl/media/Sound.html#play. What happens after that? :)

Any pointers on how to construct reproducible cases out of these or how to fix these issues would be greatly appreciated.

[Leo Lännenmäki]

I fiddled with hxScout over the weekend and experienced crashes that might or might not be related, posting here just in case as they look somewhat similar to my eye:

Crash 1:

Crashed Thread:        16

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)

Exception Codes:       EXC_I386_GPFLT

Thread 0:: Dispatch queue: com.apple.main-thread

0   com.sharkpunch.themasterplan   0x0000000101eaa35a hx::Telemetry::push_callstack_ids_into(hx::CallStack*, std::vector<int, std::allocator<int> >*) + 74

1   com.sharkpunch.themasterplan   0x0000000101eaa4b0 hx::Telemetry::ComputeCallStackId(hx::CallStack*) + 96

2   com.sharkpunch.themasterplan   0x0000000101eaaa32 hx::Telemetry::HXTAllocation(hx::CallStack*, void*, unsigned long, char const*) + 162

3   com.sharkpunch.themasterplan   0x0000000101eac9d3 hx::CallStack::HXTAllocation(void*, unsigned long, char const*) + 83

4   com.sharkpunch.themasterplan   0x0000000101eaafa0 __hxt_gc_new(void*, int) + 32

5   com.sharkpunch.themasterplan   0x0000000100010019 hx::Object::operator new(unsigned long, bool, char const*) + 73

6   com.sharkpunch.themasterplan   0x0000000101d4c689 cpp::vm::Thread_obj::operator new(unsigned long, bool) + 41

7   com.sharkpunch.themasterplan   0x0000000101d4b9ad cpp::vm::Thread_obj::__new(Dynamic) + 45

8   com.sharkpunch.themasterplan   0x0000000101d4bec2 cpp::vm::Thread_obj::create(Dynamic) + 178

9   com.sharkpunch.themasterplan   0x0000000100f42a7d openfl::_legacy::media::SoundChannel_obj::__checkComplete() + 1565

10  com.sharkpunch.themasterplan   0x0000000100f44797 openfl::_legacy::media::SoundChannel_obj::__pollComplete() + 311

11  com.sharkpunch.themasterplan   0x0000000100fc1a09 openfl::_legacy::display::Stage_obj::__pollTimers() + 217

12  com.sharkpunch.themasterplan   0x0000000100fb8250 openfl::_legacy::display::Stage_obj::__doProcessStageEvent(Dynamic) + 2544

13  com.sharkpunch.themasterplan   0x0000000100fb984f openfl::_legacy::display::Stage_obj::__processStageEvent(Dynamic) + 207

14  com.sharkpunch.themasterplan   0x0000000100fb9939 openfl::_legacy::display::__Stage_obj__processStageEvent(hx::Object*, Dynamic const&) + 89

15  com.sharkpunch.themasterplan   0x0000000101ee78c7 hx::CMemberFunction1::__run(Dynamic const&) + 71

16  com.sharkpunch.themasterplan   0x0000000101e623a6 val_call1 + 134

17  lime-legacy.ndll               0x0000000104e56d2f 0x104e00000 + 355631

18  lime-legacy.ndll               0x0000000104e41bb8 0x104e00000 + 269240

19  lime-legacy.ndll               0x0000000104e05d24 0x104e00000 + 23844

20  lime-legacy.ndll               0x0000000104e05b89 0x104e00000 + 23433

21  lime-legacy.ndll               0x0000000104e56938 0x104e00000 + 354616

22  com.sharkpunch.themasterplan   0x0000000101ea6852 ExternalPrimitive::__Run(Array<Dynamic> const&) + 578

23  com.sharkpunch.themasterplan   0x0000000101ee0cb2 Dynamic::operator()(Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&) + 226

24  com.sharkpunch.themasterplan   0x0000000101e3d09d openfl::_legacy::Lib_obj::create(Dynamic, int, int, hx::Null<double>, hx::Null<int>, hx::Null<int>, String, hx::ObjectPtr<openfl::_legacy::display::BitmapData_obj>, hx::ObjectPtr<hx::Class_obj>) + 2061

25  com.sharkpunch.themasterplan   0x0000000101e5aa09 ApplicationMain_obj::main() + 649

26  com.sharkpunch.themasterplan   0x0000000101e5f94c main + 60

27  com.sharkpunch.themasterplan   0x0000000100000e04 start + 52

Thread 1:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 2:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 3:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 4:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 5:: Dispatch queue: com.apple.libdispatch-manager

0   libsystem_kernel.dylib         0x00007fff8e385232 kevent64 + 10

1   libdispatch.dylib             0x00007fff8f515a6a _dispatch_mgr_thread + 52

Thread 6:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 7:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 8:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 9:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 10:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 11:

0   libsystem_kernel.dylib         0x00007fff8e37f4de mach_msg_trap + 10

1   libsystem_kernel.dylib         0x00007fff8e37e64f mach_msg + 55

2   com.apple.CoreFoundation       0x00007fff95017eb4 __CFRunLoopServiceMachPort + 212

3   com.apple.CoreFoundation       0x00007fff9501737b __CFRunLoopRun + 1371

4   com.apple.CoreFoundation       0x00007fff95016bd8 CFRunLoopRunSpecific + 296

5   com.apple.AppKit               0x00007fff9385f66b _NSEventThread + 137

6   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

7   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

8   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 12:: SDLTimer

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   lime-legacy.ndll               0x00000001050eb2de 0x104e00000 + 3060446

2   lime-legacy.ndll               0x00000001050eb650 0x104e00000 + 3061328

3   lime-legacy.ndll               0x000000010509a023 0x104e00000 + 2727971

4   lime-legacy.ndll               0x0000000105099c0c 0x104e00000 + 2726924

5   lime-legacy.ndll               0x00000001050eb819 0x104e00000 + 3061785

6   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

7   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

8   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 13:: com.apple.audio.IOThread.client

0   libsystem_kernel.dylib         0x00007fff8e37f4de mach_msg_trap + 10

1   libsystem_kernel.dylib         0x00007fff8e37e64f mach_msg + 55

2   com.apple.audio.CoreAudio     0x00007fff8fca857e HALB_MachPort::SendMessageWithReply(unsigned int, unsigned int, unsigned int, unsigned int, mach_msg_header_t*, bool, unsigned int) + 98

3   com.apple.audio.CoreAudio     0x00007fff8fca850c HALB_MachPort::SendSimpleMessageWithSimpleReply(unsigned int, unsigned int, int, int&, bool, unsigned int) + 42

4   com.apple.audio.CoreAudio     0x00007fff8fca6aa9 HALC_ProxyIOContext::IOWorkLoop() + 965

5   com.apple.audio.CoreAudio     0x00007fff8fca6612 HALC_ProxyIOContext::IOThreadEntry(void*) + 88

6   com.apple.audio.CoreAudio     0x00007fff8fca64e3 HALB_IOThread::Entry(void*) + 157

7   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

8   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

9   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 14:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101ec32ac MySemaphore::QWait() + 76

2   com.sharkpunch.themasterplan   0x0000000101ec239f Deque::PopFront(bool) + 143

3   com.sharkpunch.themasterplan   0x0000000101ec25af hxThreadInfo::ReadMessage(bool) + 47

4   com.sharkpunch.themasterplan   0x0000000101ec1841 __hxcpp_thread_read_message(bool) + 65

5   com.sharkpunch.themasterplan   0x0000000101d4c018 cpp::vm::Thread_obj::readMessage(bool) + 120

6   com.sharkpunch.themasterplan   0x00000001018bbabf hxtelemetry::HxTelemetry_obj::start_writer() + 3087

7   com.sharkpunch.themasterplan   0x00000001018bbe68 hxtelemetry::__HxTelemetry_objstart_writer() + 24

8   com.sharkpunch.themasterplan   0x0000000101ee7a9a hx::CStaticFunction0::__run() + 26

9   com.sharkpunch.themasterplan   0x0000000101ec1343 hxThreadFunc(void*) + 179

10  libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

11  libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

12  libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 15:

0   libsystem_kernel.dylib         0x00007fff8e38448a __semwait_signal + 10

1   com.sharkpunch.themasterplan   0x0000000101eb76a6 hx::Telemetry::ProfileMainLoop(void*) + 70

2   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

3   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

4   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 16 Crashed:

0   com.sharkpunch.themasterplan   0x0000000101eaa8a8 hx::Telemetry::StackUpdate(hx::CallStack*, hx::StackFrame*) + 24

1   com.sharkpunch.themasterplan   0x0000000101ebf608 hx::CallStack::PushStackFrame(hx::StackFrame*) + 104

2   com.sharkpunch.themasterplan   0x0000000101eab932 hx::CallStack::PushCallerStackFrame(hx::StackFrame*) + 34

3   com.sharkpunch.themasterplan   0x0000000101ea9cc5 hx::__hxcpp_register_stack_frame(hx::StackFrame*) + 21

4   com.sharkpunch.themasterplan   0x0000000100010108 hx::StackFrame::StackFrame(char const*, char const*, int, char const*, char const*, int, int) + 120

5   com.sharkpunch.themasterplan   0x000000010000e2a8 hx::StackFrame::StackFrame(char const*, char const*, int, char const*, char const*, int, int) + 88

6   com.sharkpunch.themasterplan   0x0000000100f443d6 openfl::_legacy::media::SoundChannel_obj::__checkCompleteBackgroundThread() + 86

7   com.sharkpunch.themasterplan   0x0000000100f44518 openfl::_legacy::media::__SoundChannel_obj__checkCompleteBackgroundThread() + 24

8   com.sharkpunch.themasterplan   0x0000000101ee7a9a hx::CStaticFunction0::__run() + 26

9   com.sharkpunch.themasterplan   0x0000000101ec1343 hxThreadFunc(void*) + 179

10  libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

11  libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

12  libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Crash 2:

Crashed Thread:        16

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)

Exception Codes:       EXC_I386_GPFLT

Thread 0:: Dispatch queue: com.apple.main-thread

0   libsystem_platform.dylib       0x00007fff98375348 OSAtomicCompareAndSwapPtrBarrier$VARIANT$mp + 8

1   com.sharkpunch.themasterplan   0x0000000101e740bc MyMutex::Lock() + 28

2   com.sharkpunch.themasterplan   0x0000000101eaaa25 hx::Telemetry::HXTAllocation(hx::CallStack*, void*, unsigned long, char const*) + 149

3   com.sharkpunch.themasterplan   0x0000000101eac9d3 hx::CallStack::HXTAllocation(void*, unsigned long, char const*) + 83

4   com.sharkpunch.themasterplan   0x0000000101eaafa0 __hxt_gc_new(void*, int) + 32

5   com.sharkpunch.themasterplan   0x0000000100010019 hx::Object::operator new(unsigned long, bool, char const*) + 73

6   com.sharkpunch.themasterplan   0x0000000101ec4909 hx::ArrayBase::iterator_dyn() + 57

7   com.sharkpunch.themasterplan   0x0000000101ec5b66 hx::ArrayBase::__Field(String const&, bool) + 534

8   com.sharkpunch.themasterplan   0x0000000101dc3bb8 Lambda_obj::has(Dynamic, Dynamic) + 200

9   com.sharkpunch.themasterplan   0x0000000100f3f78a openfl::_legacy::media::AudioThreadState_obj::add(hx::ObjectPtr<openfl::_legacy::media::SoundChannel_obj>) + 170

10  com.sharkpunch.themasterplan   0x0000000100f42b4a openfl::_legacy::media::SoundChannel_obj::__checkComplete() + 1770

11  com.sharkpunch.themasterplan   0x0000000100f44797 openfl::_legacy::media::SoundChannel_obj::__pollComplete() + 311

12  com.sharkpunch.themasterplan   0x0000000100fc1a09 openfl::_legacy::display::Stage_obj::__pollTimers() + 217

13  com.sharkpunch.themasterplan   0x0000000100fb8250 openfl::_legacy::display::Stage_obj::__doProcessStageEvent(Dynamic) + 2544

14  com.sharkpunch.themasterplan   0x0000000100fb984f openfl::_legacy::display::Stage_obj::__processStageEvent(Dynamic) + 207

15  com.sharkpunch.themasterplan   0x0000000100fb9939 openfl::_legacy::display::__Stage_obj__processStageEvent(hx::Object*, Dynamic const&) + 89

16  com.sharkpunch.themasterplan   0x0000000101ee78c7 hx::CMemberFunction1::__run(Dynamic const&) + 71

17  com.sharkpunch.themasterplan   0x0000000101e623a6 val_call1 + 134

18  lime-legacy.ndll               0x0000000105856d2f 0x105800000 + 355631

19  lime-legacy.ndll               0x0000000105841bb8 0x105800000 + 269240

20  lime-legacy.ndll               0x0000000105805d24 0x105800000 + 23844

21  lime-legacy.ndll               0x0000000105805b89 0x105800000 + 23433

22  lime-legacy.ndll               0x0000000105856938 0x105800000 + 354616

23  com.sharkpunch.themasterplan   0x0000000101ea6852 ExternalPrimitive::__Run(Array<Dynamic> const&) + 578

24  com.sharkpunch.themasterplan   0x0000000101ee0cb2 Dynamic::operator()(Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&) + 226

25  com.sharkpunch.themasterplan   0x0000000101e3d09d openfl::_legacy::Lib_obj::create(Dynamic, int, int, hx::Null<double>, hx::Null<int>, hx::Null<int>, String, hx::ObjectPtr<openfl::_legacy::display::BitmapData_obj>, hx::ObjectPtr<hx::Class_obj>) + 2061

26  com.sharkpunch.themasterplan   0x0000000101e5aa09 ApplicationMain_obj::main() + 649

27  com.sharkpunch.themasterplan   0x0000000101e5f94c main + 60

28  com.sharkpunch.themasterplan   0x0000000100000e04 start + 52

Thread 1:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 2:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 3:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 4:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 5:: Dispatch queue: com.apple.libdispatch-manager

0   libsystem_kernel.dylib         0x00007fff8e385232 kevent64 + 10

1   libdispatch.dylib             0x00007fff8f515a6a _dispatch_mgr_thread + 52

Thread 6:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 7:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 8:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 9:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 10:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 11:

0   libsystem_kernel.dylib         0x00007fff8e37f4de mach_msg_trap + 10

1   libsystem_kernel.dylib         0x00007fff8e37e64f mach_msg + 55

2   com.apple.CoreFoundation       0x00007fff95017eb4 __CFRunLoopServiceMachPort + 212

3   com.apple.CoreFoundation       0x00007fff9501737b __CFRunLoopRun + 1371

4   com.apple.CoreFoundation       0x00007fff95016bd8 CFRunLoopRunSpecific + 296

5   com.apple.AppKit               0x00007fff9385f66b _NSEventThread + 137

6   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

7   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

8   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 12:: SDLTimer

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   lime-legacy.ndll               0x0000000105aeb2de 0x105800000 + 3060446

2   lime-legacy.ndll               0x0000000105aeb650 0x105800000 + 3061328

3   lime-legacy.ndll               0x0000000105a9a023 0x105800000 + 2727971

4   lime-legacy.ndll               0x0000000105a99c0c 0x105800000 + 2726924

5   lime-legacy.ndll               0x0000000105aeb819 0x105800000 + 3061785

6   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

7   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

8   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 13:: com.apple.audio.IOThread.client

0   libsystem_kernel.dylib         0x00007fff8e37f4de mach_msg_trap + 10

1   libsystem_kernel.dylib         0x00007fff8e37e64f mach_msg + 55

2   com.apple.audio.CoreAudio     0x00007fff8fca857e HALB_MachPort::SendMessageWithReply(unsigned int, unsigned int, unsigned int, unsigned int, mach_msg_header_t*, bool, unsigned int) + 98

3   com.apple.audio.CoreAudio     0x00007fff8fca850c HALB_MachPort::SendSimpleMessageWithSimpleReply(unsigned int, unsigned int, int, int&, bool, unsigned int) + 42

4   com.apple.audio.CoreAudio     0x00007fff8fca6aa9 HALC_ProxyIOContext::IOWorkLoop() + 965

5   com.apple.audio.CoreAudio     0x00007fff8fca6612 HALC_ProxyIOContext::IOThreadEntry(void*) + 88

6   com.apple.audio.CoreAudio     0x00007fff8fca64e3 HALB_IOThread::Entry(void*) + 157

7   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

8   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

9   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 14:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101ec32ac MySemaphore::QWait() + 76

2   com.sharkpunch.themasterplan   0x0000000101ec239f Deque::PopFront(bool) + 143

3   com.sharkpunch.themasterplan   0x0000000101ec25af hxThreadInfo::ReadMessage(bool) + 47

4   com.sharkpunch.themasterplan   0x0000000101ec1841 __hxcpp_thread_read_message(bool) + 65

5   com.sharkpunch.themasterplan   0x0000000101d4c018 cpp::vm::Thread_obj::readMessage(bool) + 120

6   com.sharkpunch.themasterplan   0x00000001018bbabf hxtelemetry::HxTelemetry_obj::start_writer() + 3087

7   com.sharkpunch.themasterplan   0x00000001018bbe68 hxtelemetry::__HxTelemetry_objstart_writer() + 24

8   com.sharkpunch.themasterplan   0x0000000101ee7a9a hx::CStaticFunction0::__run() + 26

9   com.sharkpunch.themasterplan   0x0000000101ec1343 hxThreadFunc(void*) + 179

10  libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

11  libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

12  libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 15:

0   libsystem_kernel.dylib         0x00007fff8e38448a __semwait_signal + 10

1   com.sharkpunch.themasterplan   0x0000000101eb76a6 hx::Telemetry::ProfileMainLoop(void*) + 70

2   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

3   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

4   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 16 Crashed:

0   com.sharkpunch.themasterplan   0x0000000101eaa8a8 hx::Telemetry::StackUpdate(hx::CallStack*, hx::StackFrame*) + 24

1   com.sharkpunch.themasterplan   0x0000000101ebf608 hx::CallStack::PushStackFrame(hx::StackFrame*) + 104

2   com.sharkpunch.themasterplan   0x0000000101eab932 hx::CallStack::PushCallerStackFrame(hx::StackFrame*) + 34

3   com.sharkpunch.themasterplan   0x0000000101ea9cc5 hx::__hxcpp_register_stack_frame(hx::StackFrame*) + 21

4   com.sharkpunch.themasterplan   0x0000000100010108 hx::StackFrame::StackFrame(char const*, char const*, int, char const*, char const*, int, int) + 120

5   com.sharkpunch.themasterplan   0x000000010000e2a8 hx::StackFrame::StackFrame(char const*, char const*, int, char const*, char const*, int, int) + 88

6   com.sharkpunch.themasterplan   0x0000000100f443d6 openfl::_legacy::media::SoundChannel_obj::__checkCompleteBackgroundThread() + 86

7   com.sharkpunch.themasterplan   0x0000000100f44518 openfl::_legacy::media::__SoundChannel_obj__checkCompleteBackgroundThread() + 24

8   com.sharkpunch.themasterplan   0x0000000101ee7a9a hx::CStaticFunction0::__run() + 26

9   com.sharkpunch.themasterplan   0x0000000101ec1343 hxThreadFunc(void*) + 179

10  libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

11  libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

12  libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Crash 3:

Crashed Thread:        16

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)

Exception Codes:       EXC_I386_GPFLT

Thread 0:: Dispatch queue: com.apple.main-thread

0   com.sharkpunch.themasterplan   0x0000000101eaa3ea hx::Telemetry::GetNameIdx(char const*) + 58

1   com.sharkpunch.themasterplan   0x0000000101eaa37b hx::Telemetry::push_callstack_ids_into(hx::CallStack*, std::vector<int, std::allocator<int> >*) + 107

2   com.sharkpunch.themasterplan   0x0000000101eaa4b0 hx::Telemetry::ComputeCallStackId(hx::CallStack*) + 96

3   com.sharkpunch.themasterplan   0x0000000101eaaa32 hx::Telemetry::HXTAllocation(hx::CallStack*, void*, unsigned long, char const*) + 162

4   com.sharkpunch.themasterplan   0x0000000101eac9d3 hx::CallStack::HXTAllocation(void*, unsigned long, char const*) + 83

5   com.sharkpunch.themasterplan   0x0000000101eaafa0 __hxt_gc_new(void*, int) + 32

6   com.sharkpunch.themasterplan   0x0000000100010019 hx::Object::operator new(unsigned long, bool, char const*) + 73

7   com.sharkpunch.themasterplan   0x0000000101ec4909 hx::ArrayBase::iterator_dyn() + 57

8   com.sharkpunch.themasterplan   0x0000000101ec5b66 hx::ArrayBase::__Field(String const&, bool) + 534

9   com.sharkpunch.themasterplan   0x0000000101dc3bb8 Lambda_obj::has(Dynamic, Dynamic) + 200

10  com.sharkpunch.themasterplan   0x0000000100f3f78a openfl::_legacy::media::AudioThreadState_obj::add(hx::ObjectPtr<openfl::_legacy::media::SoundChannel_obj>) + 170

11  com.sharkpunch.themasterplan   0x0000000100f42b4a openfl::_legacy::media::SoundChannel_obj::__checkComplete() + 1770

12  com.sharkpunch.themasterplan   0x0000000100f44797 openfl::_legacy::media::SoundChannel_obj::__pollComplete() + 311

13  com.sharkpunch.themasterplan   0x0000000100fc1a09 openfl::_legacy::display::Stage_obj::__pollTimers() + 217

14  com.sharkpunch.themasterplan   0x0000000100fb8250 openfl::_legacy::display::Stage_obj::__doProcessStageEvent(Dynamic) + 2544

15  com.sharkpunch.themasterplan   0x0000000100fb984f openfl::_legacy::display::Stage_obj::__processStageEvent(Dynamic) + 207

16  com.sharkpunch.themasterplan   0x0000000100fb9939 openfl::_legacy::display::__Stage_obj__processStageEvent(hx::Object*, Dynamic const&) + 89

17  com.sharkpunch.themasterplan   0x0000000101ee78c7 hx::CMemberFunction1::__run(Dynamic const&) + 71

18  com.sharkpunch.themasterplan   0x0000000101e623a6 val_call1 + 134

19  lime-legacy.ndll               0x0000000106056d2f 0x106000000 + 355631

20  lime-legacy.ndll               0x0000000106041bb8 0x106000000 + 269240

21  lime-legacy.ndll               0x0000000106005d24 0x106000000 + 23844

22  lime-legacy.ndll               0x0000000106005b89 0x106000000 + 23433

23  lime-legacy.ndll               0x0000000106056938 0x106000000 + 354616

24  com.sharkpunch.themasterplan   0x0000000101ea6852 ExternalPrimitive::__Run(Array<Dynamic> const&) + 578

25  com.sharkpunch.themasterplan   0x0000000101ee0cb2 Dynamic::operator()(Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&, Dynamic const&) + 226

26  com.sharkpunch.themasterplan   0x0000000101e3d09d openfl::_legacy::Lib_obj::create(Dynamic, int, int, hx::Null<double>, hx::Null<int>, hx::Null<int>, String, hx::ObjectPtr<openfl::_legacy::display::BitmapData_obj>, hx::ObjectPtr<hx::Class_obj>) + 2061

27  com.sharkpunch.themasterplan   0x0000000101e5aa09 ApplicationMain_obj::main() + 649

28  com.sharkpunch.themasterplan   0x0000000101e5f94c main + 60

29  com.sharkpunch.themasterplan   0x0000000100000e04 start + 52

Thread 1:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 2:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 3:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 4:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101e77287 MySemaphore::Wait() + 87

2   com.sharkpunch.themasterplan   0x0000000101e777f7 GlobalAllocator::MarkerLoop(int) + 55

3   com.sharkpunch.themasterplan   0x0000000101e777b0 GlobalAllocator::SMarkerFunc(void*) + 32

4   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

5   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

6   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 5:: Dispatch queue: com.apple.libdispatch-manager

0   libsystem_kernel.dylib         0x00007fff8e385232 kevent64 + 10

1   libdispatch.dylib             0x00007fff8f515a6a _dispatch_mgr_thread + 52

Thread 6:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 7:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 8:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 9:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 10:

0   libsystem_kernel.dylib         0x00007fff8e38494a __workq_kernreturn + 10

1   libsystem_pthread.dylib       0x00007fff9232d40d start_wqthread + 13

Thread 11:

0   libsystem_kernel.dylib         0x00007fff8e37f4de mach_msg_trap + 10

1   libsystem_kernel.dylib         0x00007fff8e37e64f mach_msg + 55

2   com.apple.CoreFoundation       0x00007fff95017eb4 __CFRunLoopServiceMachPort + 212

3   com.apple.CoreFoundation       0x00007fff9501737b __CFRunLoopRun + 1371

4   com.apple.CoreFoundation       0x00007fff95016bd8 CFRunLoopRunSpecific + 296

5   com.apple.AppKit               0x00007fff9385f66b _NSEventThread + 137

6   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

7   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

8   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 12:: SDLTimer

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   lime-legacy.ndll               0x00000001062eb2de 0x106000000 + 3060446

2   lime-legacy.ndll               0x00000001062eb650 0x106000000 + 3061328

3   lime-legacy.ndll               0x000000010629a023 0x106000000 + 2727971

4   lime-legacy.ndll               0x0000000106299c0c 0x106000000 + 2726924

5   lime-legacy.ndll               0x00000001062eb819 0x106000000 + 3061785

6   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

7   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

8   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 13:: com.apple.audio.IOThread.client

0   libsystem_kernel.dylib         0x00007fff8e37f4de mach_msg_trap + 10

1   libsystem_kernel.dylib         0x00007fff8e37e64f mach_msg + 55

2   com.apple.audio.CoreAudio     0x00007fff8fca857e HALB_MachPort::SendMessageWithReply(unsigned int, unsigned int, unsigned int, unsigned int, mach_msg_header_t*, bool, unsigned int) + 98

3   com.apple.audio.CoreAudio     0x00007fff8fca850c HALB_MachPort::SendSimpleMessageWithSimpleReply(unsigned int, unsigned int, int, int&, bool, unsigned int) + 42

4   com.apple.audio.CoreAudio     0x00007fff8fca6aa9 HALC_ProxyIOContext::IOWorkLoop() + 965

5   com.apple.audio.CoreAudio     0x00007fff8fca6612 HALC_ProxyIOContext::IOThreadEntry(void*) + 88

6   com.apple.audio.CoreAudio     0x00007fff8fca64e3 HALB_IOThread::Entry(void*) + 157

7   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

8   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

9   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 14:

0   libsystem_kernel.dylib         0x00007fff8e384136 __psynch_cvwait + 10

1   com.sharkpunch.themasterplan   0x0000000101ec32ac MySemaphore::QWait() + 76

2   com.sharkpunch.themasterplan   0x0000000101ec239f Deque::PopFront(bool) + 143

3   com.sharkpunch.themasterplan   0x0000000101ec25af hxThreadInfo::ReadMessage(bool) + 47

4   com.sharkpunch.themasterplan   0x0000000101ec1841 __hxcpp_thread_read_message(bool) + 65

5   com.sharkpunch.themasterplan   0x0000000101d4c018 cpp::vm::Thread_obj::readMessage(bool) + 120

6   com.sharkpunch.themasterplan   0x00000001018bbabf hxtelemetry::HxTelemetry_obj::start_writer() + 3087

7   com.sharkpunch.themasterplan   0x00000001018bbe68 hxtelemetry::__HxTelemetry_objstart_writer() + 24

8   com.sharkpunch.themasterplan   0x0000000101ee7a9a hx::CStaticFunction0::__run() + 26

9   com.sharkpunch.themasterplan   0x0000000101ec1343 hxThreadFunc(void*) + 179

10  libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

11  libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

12  libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 15:

0   libsystem_kernel.dylib         0x00007fff8e38448a __semwait_signal + 10

1   com.sharkpunch.themasterplan   0x0000000101eb76a6 hx::Telemetry::ProfileMainLoop(void*) + 70

2   libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

3   libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

4   libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

Thread 16 Crashed:

0   com.sharkpunch.themasterplan   0x0000000101eaa8a8 hx::Telemetry::StackUpdate(hx::CallStack*, hx::StackFrame*) + 24

1   com.sharkpunch.themasterplan   0x0000000101ebf608 hx::CallStack::PushStackFrame(hx::StackFrame*) + 104

2   com.sharkpunch.themasterplan   0x0000000101eab932 hx::CallStack::PushCallerStackFrame(hx::StackFrame*) + 34

3   com.sharkpunch.themasterplan   0x0000000101ea9cc5 hx::__hxcpp_register_stack_frame(hx::StackFrame*) + 21

4   com.sharkpunch.themasterplan   0x0000000100010108 hx::StackFrame::StackFrame(char const*, char const*, int, char const*, char const*, int, int) + 120

5   com.sharkpunch.themasterplan   0x000000010000e2a8 hx::StackFrame::StackFrame(char const*, char const*, int, char const*, char const*, int, int) + 88

6   com.sharkpunch.themasterplan   0x0000000100f443d6 openfl::_legacy::media::SoundChannel_obj::__checkCompleteBackgroundThread() + 86

7   com.sharkpunch.themasterplan   0x0000000100f44518 openfl::_legacy::media::__SoundChannel_obj__checkCompleteBackgroundThread() + 24

8   com.sharkpunch.themasterplan   0x0000000101ee7a9a hx::CStaticFunction0::__run() + 26

9   com.sharkpunch.themasterplan   0x0000000101ec1343 hxThreadFunc(void*) + 179

10  libsystem_pthread.dylib       0x00007fff9232f268 _pthread_body + 131

11  libsystem_pthread.dylib       0x00007fff9232f1e5 _pthread_start + 176

12  libsystem_pthread.dylib       0x00007fff9232d41d thread_start + 13

[Philippe Elsass]

I can't really help to clarify the logs, but does it ever crash if you disable all sounds? Or rendering?

Did you profile the allocations of your app to try figuring if/where you have a leak? (xcode Instruments, Very Sleepy profiler...)

[Hugh]

openfl/_legacy/media/SoundChannel.hx does not appear to mutex-lock all access to channelList (eg "add").

Maybe it does - hard to tell without tracing - but this is the first place I would look given the stack trace.

Hugh

[Jonathan Ong]

Hey Leo,

It seems like your OpenAL errors could be caused by OpenAL either running out of audio buffers or audio sources. I've encountered similar problems before, caused by two issues:

1. A broken OpenFL asset caching system (at least in 1.4.0). For some reason, my sounds weren't being properly cached by openfl.Assets, resulting in a new flash.media.Sound object (and hence an OpenAL buffer allocation) every time I called Assets.getSound(...). Writing a simple cache system helped me to reduce the number of OpenAL buffer allocations.
   
   
2. Playing too many sound effects too quickly. A new flash.media.SoundChannel is created every time Sound::play is called, resulting in a new OpenAL audio source being created. This OpenAL audio source is only deleted when the SoundChannel object is garbage-collected. When sound effects are played too rapidly, the GC might not be able to collect these SoundChannel objects fast enough, causing OpenAL to reach its audio source limit (which varies depending on your system). Unfortunately, the solution I have requires modifying the OpenFL sources to call alDeleteSources(...) upon a Sound's completion or when a SoundChannel is stopped.
   The changes I made to OpenFL that solved my problems can be found in this gist: https://gist.github.com/jonongjs/5dc3ec8848e4bf1f3739
   My apologies for the untidy gist/diffs.

Hope this helps,

Jon

@jon_magus

[Leo Lännenmäki]

Thanks Philippe for pointers!

I don't have a 100% reproduction case that would tell me if the problem is gone if I disable all sounds, but worth a try!

I'll try Xcode's Instrument's leak profiler again, the last time I tried I couldn't see any leaks, but it's been a while since the last time. I didn't know Very Sleepy does allocation tracking, could try that as well.

...

[Leo Lännenmäki]

Thanks Hugh! I'll take a look or try to summon Joshua for help :)

Is there any documentation or examples you could point me to with regards to proper use of mutexes in a case like this?

[Leo Lännenmäki]

Hey Jonathan! 

This is good stuff. 

1. We only call Assets.getMusic/Assets.getSound once per sound and cache those, so this part shouldn't be a problem.

2. This is definitely worth investigating. We call Sound::play every time for example a weapon is fired, or when a character walks on each step. I'll check if I can instrument some of the sound playback code to see if we run into this audio source limit.

[Hugh]

There is already some mutex locking (acquiring) in this file.  I would start by being paranoid and add locking around each of the routines in this file - you can always pare it back later. 

[underscorediscovery]

You can also disable the threading entirely if that is the same code as older iterations, there was a compiler define to do that in the original.
Only streamed/music sources were threaded anyway, but disabling them (or audio entirely as someone suggested) would further isolate the potential issues stemming from audio specifically.