Re: [nodejs] Embedding Node.js in the Chromium render process

[Ben Noordhuis]

On Tue, Jul 10, 2012 at 8:31 PM, Nathan Sobo <nat...@github.com> wrote:
> I'm interested in embedding Node.js into the Chromium renderer process for
> an internal tool I'm building for GitHub. This is a situation where running
> Node as a subprocess won't suffice, so I'm curious what the barriers are do
> doing this in process.
>
> Using Chromium's V8 Context
> Node is current with the version of V8 used by Chromium, and does not seem
> to have applied any patches. I'm imagining I can write an alternative to
> Node::start that uses Chromium's existing V8 context instead of building a
> new one. Is there anything I'm missing on that front?

We float patches on top of V8 from time to time. Most eventually land
upstream but it means the V8 that ships with Chromium is not
necessarily the V8 that ships with Node.

> Embedding Node's Event Loop Without Blocking The Renderer Thread
> The next challenge seems more daunting. The Chromium renderer thread has its
> own run loop that can't really be modified and obviously can't be blocked by
> just firing up a blocking libuv loop. Following the advice of the libev
> documentation, I performed a small experiment with libev. I was able to run
> the libev event loop inside the Chromium renderer process on an auxiliary
> thread, but still retain the ability to add watches from the renderer thread
> and–equally important–have my callback code execute on the renderer thread.
> This ability to interact from the application thread (not the thread the
> loop is blocking on) is essential in my case, because the V8 context I'd be
> injecting Node into lives in the renderer thread.
>
> This mode of operation is officially supported / promoted by libev (and
> libeio, via a different API). The technique requires the following
> functions. Some have clear analogs in libuv; others I'm less sure about.
>
> ev_set_loop_release_cb
> This function allows you to set two callbacks that acquire and release a
> mutex while the loop thread is accessing the loop data. Release is called
> just before the loop thread goes to sleep, and acquire is called when it
> wakes up.
>
> ev_async_*
> The async family of functions are present in libuv. They allow the main
> thread to wake up the loop thread whenever the main thread makes a change to
> the loop. Since the loop thread is sleeping, it has to be woken to recognize
> these changes.
>
> ev_set_invoke_pending_cb
> This function allows you to replace the code used by the loop to invoke
> pending callbacks when processing events. You use this callback to tell the
> main thread to invoke the loop's pending callbacks instead of doing it on
> the loop thread. You then wait on a condition variable for the main thread
> to finish invoking them.
>
> ev_invoke_pending
> This is libev's default mechanism for invoking pending callbacks. The
> important thing is that it's exposed so you can call it from the main thread
> when it is signalled by the invoke pending callback.
>
>
> How hard would it be to add similar functions in the libuv API? Are there
> any additional complications (besides Windows) that would make this scenario
> more difficult than it is in the libev example? I think allowing Node to be
> embedded in applications that have their own event loops would be a
> generally valuable feature. I know the standard line is just to embed it as
> a child process, but in-process embedding obviously allows for much richer
> bindings to the host application's API. In our case, we want to embed it
> directly into a browser, so an external process isn't an option. I'd like to
> gauge how feasible such a feature is, as well as the Node developers overall
> attitude toward this use-case.
>
> Thanks very much for your time!

First off, don't use libev - we're dropping it.

Embedding libuv in another event loop is an oft-requested feature and
it's something we're adding in the near future (possibly as a
UNIX-only feature for now). The API is TBD but it will probably look
something like this:

int uv_loop_poll_fd(uv_loop_t*);
int uv_loop_poll_timeout(uv_loop_t*);

In other words, you get a file descriptor that you embed in your own
event loop and a timeout to observe. When the file descriptor becomes
active or the timeout expires, you call uv_run_once().

[Nathan Rajlich]

^ That sounds like exactly the feature I need for NodObjC :)

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Brandon Benvie]

Appjs runs Chromium on the same thread as Node, though we're still not exactly the best way to go about bridging the contexts. Presumably this should be feasible now and maybe I'll give it a go to day. Doing this means modifying CEF to give up its vice grip on the V8 context and everything in V8. uv_async_t is used along with uv_timer_t which drives the event loop for Chromes render process subordinate to Node's. I would give this would cause issues for things like 3d rendering and whatnot but for normal GUI type stuff it's fine.

 https://github.com/milani/appjs

[Brandon Benvie]

Oh and this is the related code https://github.com/milani/appjs/blob/master/src/includes/cef_loop.cpp

[Nathan Sobo]

I'm glad to hear you're thinking about this. What's the mutual exclusion situation look like? When I call functions that modify the uv_loop_t struct from another thread, will I need to synchronize that access, or would that synchronization be done internally? Are there any branches started on this?

Thanks for your reply.

 

[Nathan Sobo]

Glad to hear from you again. App.js looks cool. I'd love to talk on gchat about this if you have time.

[Ben Noordhuis]

On Wed, Jul 11, 2012 at 12:56 AM, Nathan Sobo <nat...@github.com> wrote:
> I'm glad to hear you're thinking about this. What's the mutual exclusion
> situation look like? When I call functions that modify the uv_loop_t struct
> from another thread, will I need to synchronize that access, or would that
> synchronization be done internally? Are there any branches started on this?
>
> Thanks for your reply.

You need to handle synchronization yourself.

I should mention that libuv does not support manipulating an event
loop from more than one thread. Calling e.g. uv_run_once(loop) from
alternately thread #1 and thread #2 - even with proper locking in
place - won't work. That's mostly a uv-win limitation but uv-unix
makes no guarantees either.

You can have multiple event loops, one for each thread, however.

Re: branches: no, not yet. I'm working on a couple of other features first.

[Nathan Sobo]

On Tuesday, July 10, 2012 6:06:22 PM UTC-6, Ben Noordhuis wrote:

On Wed, Jul 11, 2012 at 12:56 AM, Nathan Sobo <nat...@github.com> wrote:
> I'm glad to hear you're thinking about this. What's the mutual exclusion
> situation look like? When I call functions that modify the uv_loop_t struct
> from another thread, will I need to synchronize that access, or would that
> synchronization be done internally? Are there any branches started on this?
>
> Thanks for your reply.

You need to handle synchronization yourself.
 

I should mention that libuv does not support manipulating an event
loop from more than one thread. Calling e.g. uv_run_once(loop) from
alternately thread #1 and thread #2 - even with proper locking in
place - won't work. That's mostly a uv-win limitation but uv-unix
makes no guarantees either.

I wouldn't want to call run_once from two different threads in this case. I would just want to start watches and execute callbacks on the renderer thread but do the looping and blocking on an auxiliary thread so as not to block the application event loop on the renderer thread.

 

You can have multiple event loops, one for each thread, however.

I notice a lot of use of default_loop in Node. So I guess it still assumes a single loop. This is looking like it might involve a lot of changes :-/

 

Re: branches: no, not yet. I'm working on a couple of other features first.

Okay... not sure we're quite on the same page, but that's okay. Perhaps this is not a fruitful path to attempt right now. Thanks again!