Cleaning up after a Posix fork

91 views
Skip to first unread message

Sam Saffron

unread,
Sep 13, 2020, 7:46:28 PM9/13/20
to v8-users

It is clear that there is no support for OS level forking in v8, this has been mentioned previously.

However what is unclear is how to perform rudimentary cleanups so isolates and contexts can continue to be used post forking.

In MiniRacer (v8 binding for Ruby) I have a trivial repro of a hang:


```
(gdb) bt
#0  0x00007f18d4fb202d in pthread_cond_signal@@GLIBC_2.3.2 () from /usr/lib/libpthread.so.0
#1  0x00007f18d084a5fc in v8::platform::DelayedTaskQueue::Append(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) ()
   from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#2  0x00007f18d0849db0 in v8::platform::DefaultWorkerThreadsTaskRunner::PostTask(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) () from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#3  0x00007f18d0848f51 in v8::platform::DefaultPlatform::CallOnWorkerThread(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) () from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#4  0x00007f18d1014653 in v8::internal::ConcurrentMarking::ScheduleTasks() ()
   from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#5  0x00007f18d09732ab in v8::internal::MarkCompactCollector::MarkLiveObjects() ()
   from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#6  0x00007f18d0972bb1 in v8::internal::MarkCompactCollector::CollectGarbage() ()
   from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#7  0x00007f18d09468ea in v8::internal::Heap::MarkCompact() () from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#8  0x00007f18d0943389 in v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) ()
   from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#9  0x00007f18d0940f10 in v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) () from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#10 0x00007f18d094141e in v8::internal::Heap::CollectAllAvailableGarbage(v8::internal::GarbageCollectionReason) ()
   from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#11 0x00007f18d087cdec in v8::Isolate::LowMemoryNotification() () from /home/sam/Source/mini_racer/lib/mini_racer_extension.so
#12 0x00007f18d08445f6 in rb_isolate_low_memory_notification (self=94876640147440)
    at ../../../../ext/mini_racer_extension/mini_racer_extension.cc:783
```

Contexts in mini_racer always get clean isolates, so somehow the task queue appears to be wedged post fork due to some global state maybe?

What rudimentary cleanup can we do post for to ensure that new isolates and contexts can continue to be used?

isolate->Dispose() seems insufficient.

We are using v8 8.4

Ben Noordhuis

unread,
Sep 14, 2020, 8:17:57 AM9/14/20
to v8-users
You'll also have to call V8::Dispose() and V8::ShutdownPlatform(), in
that order, then call V8::InitializePlatform() and V8::Initialize()
again, also in that order. Even then I wouldn't want to bet it
actually works, or keeps working going forward.

You can go a step further and add a dlclose+dlopen libv8.so in between
to force even more cleanup, but again, that's still no guarantee it'll
work. A better bet is to prefork before initializing V8.

Sam Saffron

unread,
Sep 14, 2020, 9:00:40 PM9/14/20
to v8-users
Thanks Ben,

This is such a tricky problem. Sadly I segfault trying to shut down platform and the docs seems to say that that spinning up a new platform is not supported after you release resources :(

Discussing with the team the option of --single_threaded mode, which works okish, but then hangs on exit handlers.

This is such a pickle cause Ruby users love forking given we have high memory usage and a big GIL that limits multithreading.

Alex Kodat

unread,
Sep 14, 2020, 10:05:25 PM9/14/20
to v8-users
What are you trying to accomplish by forking instead of just creating a child process (strictly speaking, still a fork but then an exec<xx>)? If you're going to shut down isolates and reinstantiate them, even if you could get it to work, I'm not sure I see how that would be better than starting a child process from a clean slate. If you're concerned about recompiling your own code or basic startup costs, using a snapshot allows a V8 process+Isolate+your-code to start up pretty quickly. Well, maybe not so much for Windows as loading the executable can be a bit pokey but maybe there are ways to speed that up...
Reply all
Reply to author
Forward
0 new messages