Addon: Native vs JS?

[Dan Søndergaard]

Hi,

I'm trying to write my first node addon (I have no previous experience with C++ whatsoever). It's basically a wrapper for the PortAudio library.

I started out thinking that I would do the whole thing in C++, but now I realize that it is very difficult to implement the EventListener and Stream "interfaces" in native code, which tells me that I should just write a very basic, direct wrapper to the library, and then build up a usable API in Javascript afterwards. Is this correct?

Assuming that I'm going to write the exposed API in JS, how would I approach that?

If it's possible to write the entire thing in C++, how do I then "inherit" EventListener and Stream?

I apologize if the explanation is confusing, I'm still just trying to figure out the different layers of abstraction in node :-)

Thanks,

Dan

[Dominic Tarr]

I have not written any C addons, but I do know that EventEmitter has
been removed from the C side of node.

a tight wrapper based on callbacks, and then wrap that with a Stream
api on the js side is indeed how node does it.

reading through github.com/joyent/node/blob/master/lib/fs.js should be helpful.

there is quite a bit to writing a Stream that works well.

I am developing a Stream spec here https://gist.github.com/2850882

I'm just building it from studying stream.js, reading code in node.js
core streams.

cheers, Dominic

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[Tim Caswell]

I would recommend doing minimal work in C++, especially if you're more comfortable in JS.  As far as performance, keep in mind that crossing the JS <-> C++ boundary is very expensive.  The less calls into C++, the faster you'll be.

Even for people experienced in C++, the recommendation I often hear is to do a 1-1 binding in C++ and then wrap in a pretty JS api from that.

--

[Dan Søndergaard]

Thanks, both replies were very helpful. I'll study fs.js and do a 1 <-> 1 wrapper in C++ :-)

[Nathan Rajlich]

I'll also throw in that this is the exact approach I used for node-lame: https://github.com/TooTallNate/node-lame

And it's also similar to the approach that node's zlib module does: https://github.com/joyent/node/blob/master/src/node_zlib.cc

So it sounds like you're on the right track to me.

On Tue, Jun 26, 2012 at 8:11 AM, Dan Søndergaard <dan...@gmail.com> wrote:

Thanks, both replies were very helpful. I'll study fs.js and do a 1 <-> 1 wrapper in C++ :-)

[Ben Noordhuis]

On Tue, Jun 26, 2012 at 6:10 PM, Nathan Rajlich <nat...@tootallnate.net> wrote:
> I'll also throw in that this is the exact approach I used for
> node-lame: https://github.com/TooTallNate/node-lame
>
> And it's also similar to the approach that node's zlib module
> does: https://github.com/joyent/node/blob/master/src/node_zlib.cc
>
> So it sounds like you're on the right track to me.

I'll throw in my $0.02 and say that [1] uses an approach that's quite elegant.

[1] https://github.com/WindowsAzure/node-sqlserver

[Dan Søndergaard]

Thanks again. node-lame is an interesting example, especially because it's so simple :-)

Regarding node's event loop, PortAudio has the following function for opening a stream:

Pa_OpenStream (PaStream **stream, const PaStreamParameters *inputParameters, const PaStreamParameters *outputParameters, double sampleRate, unsigned long framesPerBuffer, PaStreamFlags streamFlags, PaStreamCallback *streamCallback, void *userData)

Since this function takes a PaStreamCallback I'd guess that PortAudio has its own event loop somewhere. My idea right now is to call a JS callback function from the PaStreamCallback function in C++. Again, I'm a bit in doubt about the interaction between node and the other abstraction layers. Can the two event loops interfere?

Thanks again, you're all very helpful :-)

[Ben Noordhuis]

On Tue, Jun 26, 2012 at 8:19 PM, Dan Søndergaard <dan...@gmail.com> wrote:
> Regarding node's event loop, PortAudio has the following function for
> opening a stream:
>
> Pa_OpenStream (PaStream **stream, const PaStreamParameters *inputParameters,
> const PaStreamParameters *outputParameters, double sampleRate, unsigned long
> framesPerBuffer, PaStreamFlags streamFlags, PaStreamCallback
> *streamCallback, void *userData)
>
> Since this function takes a PaStreamCallback I'd guess that PortAudio has
> its own event loop somewhere. My idea right now is to call a JS callback
> function from the PaStreamCallback function in C++. Again, I'm a bit in
> doubt about the interaction between node and the other abstraction layers.
> Can the two event loops interfere?

It depends on how PortAudio works internally. If it's like some other
audio libraries, there will be one or more background threads that do
the actual encoding/decoding/mixing/etc.

V8 is thread-safe nor re-entrant so don't call it from your PA
callback. Use a uv_async_t handle to wake up the event loop and do it
from your async callback.

[Maurits]

I have a bit of experience in trying to write something and bumping into a few issues with it. 

I have tried my hand at portaudio and rtaudio, rtaudio being the easiest library to wrap.

It all depends on what you want to achieve with it. As is already mentioned, crossing the JS<->C++ border is expensive. 
I tried at first with having the callback function call a JS function in order to gather samples, (using a js callback pa-style) 

but that crashed pretty quickly as node just couldn't handle the torrent of function calls fast enough.

I was explained that this was happening because a call to a JS function from the wrapper is put on a stack in the event loop in order to be executed, but that the amount of stacked functions quickly overflowed the speed in which node was able to execute them.

So, the model I would go for is to implement one or more circular buffers (non-js) inside the wrapper, and have the audio callback read and write into those buffers, and expose functions to JS to read and write into those buffers. Perhaps exposing these buffers as either a Stream or a Buffer might work, though, as far as my experience can tell me, JS is (currently?) just not fast enough to do realtime audio. 

I must admit though that I sincerely hope to be proven wrong, as I would _love_ to have such a library.

Maurits

[Dan Søndergaard]

Ben: Thanks, I'll give it a try.

If Maurits is right I'm probably never going to finish this. I thought it would be a good project for a beginner ;) I do however think it's weird if node can't handle the load. I like the approach with the circular buffer and exposing it as a stream. Did you use uv_async_t as Ben proposes?

On Tuesday, June 26, 2012 4:40:41 PM UTC+2, Dan Søndergaard wrote: