Brainstorming options for a post-NPAPI world

[Richard Bateman]

Alright.  Let me first start by stating what we already know: The writing is on the wall, the browser manufacturers have decided to kill NPAPI.  I think this is a bad idea, I think it’s stupid… however, it seems that anything that doesn’t have at least 1% of the entire world’s population (small number, right? it’s just 1%) is safe to throw under the bus as far as Google is concerned.

Anyway, all frustration aside, it’s time to start figuring out other solutions; it’s possible that those solutions will fragment this community into several projects, each aimed at a different problem. I don’t know.  The important thing, though, is that now we need to identify what things can and can’t be done.  I have been in touch with Stuart Morgan, who is one of the Chrome developers who has worked with some Chrome NPAPI stuff, and his request is that we find a list of things that can’t be done with the other technologies listed in the blog post.

To that end, I have created a page on firebreath.org that I want us to all start working on:

http://www.firebreath.org/x/kQCt

I think we’re going to need to break things down more, but frankly I need help.  If I get the help, then I will work with those willing to work with me until we get something figured out and at the very least can go to the browser manufacturers with a list of “this is what we can do, this is what we need to be able to do” items and have at least some documentation on alternatives.  My hope is to create a framework that possibly uses FireBreath, possibly is totally different, but solves 90% of the issues in whatever way we need.  I don’t yet know for sure if that’s possible.  This is going to cost us all a lot of time.

If I don’t get that help then I’ll solve my problems and let everyone else figure it out on their own time.  I’m sorry to put it that way, but I just can’t do this all myself; I need the help, and right now I’m tired and cranky and frustrated =]

My thoughts so far

===============

I think we will need to break things down into subtypes of plugins; the goal is to build something that simplifies cases for those types.  Here are the groupings I’ve thought of so far, please let me know what you think:

**

Video / Drawing plugins

**

As I’ve researched, the best chance for most of these will be to use a NaCl plugin, at least from a drawing perspective.  You could alternatively try to draw using WebGL or Canvas, but if you need any native code running, it’ll have to be NaCl.  For network access, we’ll have to look into Native Messaging as a backend to the NaCl plugin, accessed asynchronously — and yes, I can see this will be messy.  it’s the only solution I see.

**

Hidden / hardware access plugins

**

Plugins that are used for hardware access or to provide behind-the-scenes services to plugins I think we can handle using Native Messaging; after installing, a chrome extension can launch communicate with the other application, thus allowing us to make requests for the other application to handle things.  We should be able to abstract this with a javascript API, possibly utilizing JSAPI on the other side to automatically create wrapper objects with common code that would let us make asynchronous calls that still look and feel like javascript and serialize the requests and responses with JSON.

—

I’m sure there are other types, but this is the current direction of my thoughts.  What other ideas do we have, and who is willing to help? I need someone to research — in depth — each of these things and then I’d like to discuss it with them to see if we can hammer it together into a unified solution.

Richard

[John Dexter]

You mention NaCl as the solution to rendering plugins, and indeed it allows re-using existing code rather than rewriting everything in javascript (ugh!) But thus far I've not seen anyone other than Chrome plan to support it which makes it an unhelpful solution. AND I'm not aware if it's supported on mobile. AND, Google have a history of killing technologies they decide aren't going anywhere (Gears). So I'm not sure NaCl is a sensible option. For plugins which simply want to do 3D rendering in the browser, I reckon JS/WebGL probably is the best route currently... but I see many such plugins and think "why is this needing to be browser based other than because you can?"

Richard, I think it would be very interesting if some discussion with people at Unity3D could be started. They must have similar concerns even if their use cases span a subset of ours.

John.

--
 
---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Georg Fritzsche]

On 24.09.2013, at 23:34, John Dexter <jdxsol...@gmail.com> wrote:

You mention NaCl as the solution to rendering plugins, and indeed it allows re-using existing code rather than rewriting everything in javascript (ugh!)

You don't necessarily have to rewrite everything in JS... Let me throw Emscripten in the hat:

https://github.com/kripken/emscripten/wiki

Georg

[John Tan]

Hi Richard,

 

Count me in! We are on the same boat now....

 

As you may know, we are developing our own video player. One major issue we are facing now is that we have our own video codec which has already been optimized to x86 and x64 in assembly language. Seem to me, such optimization will not be portable to nexe by just dropping in the code. So, what I could think of is

1) Write a NaCl app as the front player

2) Separate out the decoder engine into a local service which will decode the video and locally stream the decoded video to the NaCl app via websocket .

 

My concern to the above proposal is

1) Imagine, if the video is 1920x1080 at 24-bit and 30fps, that's going to be 186 MB per sec of data! How taxing will it be to the system, even if it's streamed via localhost?

2) Multiple concurrent player would seem impossible.

 

Also, has anyone taken a look at zeromq? I wonder if it's usable in NaCl.

[John Tan]

The other thing that I'm not clear is on one of the con of NaCl is "Does not allow raw TCP/UDP". Actually, we have studied NaCl last year and one of the concern we had was, the provided TCP communication doesn't seem to support transfer of data other than text format. In that case, I wonder how we could grab our video data from the server.

[Steven Lybeck]

We've been using a hidden browser plugin to provide access to webcams and custom hardware. Our contingency has been to design so it could also run behind a web API running as a local service in addition to the the standard plugin API. So this is the direction we're planning to head.

Concerns with the solution include:

• How to run from HTTPS sites?
• Will there be interference by local firewall software?

Depending on the performance needs, this might be an architecture applicable to plugins with UI as well. Have the local web service shuttle UI information into the browser and have JavaScript deal with getting it onto the screen. Local VNC? Maybe!

[Sylvain]

There was a discussion here in January already about Chrome OS not supporting NPAPI from the start but only Pepper/PPAPI/NaCl and whether that could come into FireBreath.

For hardware access with NaCl, there seems to be APIs for audio, File IO and OpenGL.

[re6d...@gmail.com]

For non-visual plugins I've briefly inspected the options available and to me Web Sockets seem the best choice. You'd need to replace the locally installed plugin with a local background service that is installed as a system service that is always automatically started by the operating system. This service would then listen a Web Socket socket to which the web browser would open a connection to. 

• Theoretically there is no limit what the service process can do on the local system.
  
• There would be no browser or OS specific dependencies on the HTML side of things.
• Browser support looks good with the exception of IE. IE only from version 10 and above supports Web Sockets. This locks out Windows Vista and XP IE users.
  
• The local service needs operating system specific implementation to wrap around the things you need to do

I am not sure if any of this actually works in practice. I've just dug this up in an hour or two. But it looks plausible on paper.

[Richard Bateman]

Let me be clear; I am not expecting to find one solution that will solve all problems or work on all browsers.  I am hoping to find a way -- possibly through an abstraction -- to create a solution that will be able to target multiple methods.

The NaCl option is one of very few that I can think of that even gives us a shot at doing drawing properly, though I too see many obstacles.

I don't see anyone else picking up NaCl support either; I simply mention it because I don't see another way to do the drawing that many plugins need, short of WebGL.

Another possible option we've thought of is to use WebRTC and somehow send the pre-composited video across that into the browser.

[Richard Bateman]

Pretty sure zeromq wouldn't be usable, because direct TCP/UDP is not possible in nacl.  You *might* be surprised and your assembly *might* work w/ the NaCl compiler, but you also might be better off keeping it in a seperate layer or process.

As you describe is roughly how it seems to me such plugins will have to work -- in some way communicate cross-process to get the video data in, and maybe just render to a HTML5 Canvas or WebGL context with javascript, since it seems that getting the data into NaCl might be at least as problematic.

What we need is someone who can really dig into NaCl and verify what level of communication is possible between the browser and the NaCl plugin; I know it is all async (through an event-driven message posting mechanism) but I don't know what data types you can send.  Can we even send a large chunk of data efficiently into NaCl? If not, probably won't work. If you can send large data and you can get it across the native messaging or websocket protocols then you could maybe do decoding in the nacl plugin.

Richard

--

[Richard Bateman]

My experience doing something similar in the past is that local firewalls won't care as long as it's listening on localhost.  Running from an https site is more problematic, though; my research shows that some browsers will work, some will not.

--

[Richard Bateman]

This is one of the best solutions I have come up with as well; the main thing that one of us needs to ascertain is whether or not this will work from an https website (since the localhost server would not be https)

--

[John Dexter]

If only one browser is supporting the only way of doing something, my position would be to examine really deeply if it absolutely HAS TO be browser based in the first place. Building your technology stack around something like this is a massive risk because we all know (I hope) about Google's spring-cleaning policies!

Sorry to only have depressing input but I'd be very wary indeed of allowing my business/product to be 100% dependent on a single other product.

:(

[John Tan]

Indeed.... One possible way is to create a local web service which will launch an exe-based app. On the web page side, just create a minimal webapp that talks to this local web service.

[Richard Bateman]

I agree with this as well, and to be honest we're discussing some strong possibilities that we may end up using a communication model that just goes through our server and forget any direct browser integration for just this reason.

WebSockets is definitely the most portable alternative, but I am concerned about the issues with trying to use from an SSL page.  Anyone have any experience or thoughts on that?

Richard

[Steven Lybeck]

I did a fun little partial experiment for HTTPS:

Turns out I can set up my DNS server to point a domain like local.mydomain.com to 127.0.0.1. This opens up the possibility to ship the local service with an SSL cert for local.mydomain.com and voila, HTTPS from localhost.

Shipping an SSL cert out to a bunch of people may (probably does?) open up a raft of security issues that I'm not thinking about. But maybe no more than the fact that we're already installing privileged software into user systems.

[Vladimir Kukushkin]

We are using WebSockets for communication of one of our extensions with main native app. And we ran into two major problems: 1. browsers can change ws protocol they're using, so you have to change web server inside the app and 2. you have to listen on some port on localhost and if this port is occupied, app and extension should automatically use another one. With browsers and main app restarting this port-finding becomes tricky. But yes, this approach works: extension sends data to and receives reply from native app. Also, another problem here is that extensions are completely open sourced after they are installed into browsers, so anyone can use the same handshake and communicate with your app, which is not really secure. Native code, like NPAPI plugin, is a better solution here.

[Georg Fritzsche]

Check the last post here:

https://groups.google.com/forum/m/#!msg/mozilla.dev.security/zPm2M-ieI20/jPnWmS9OHdsJ

[Sylvain]

Native client would have the advantage of also running on chromebooks as it is the only way to run native code on these machines.

It also seems like the way to interact with a native client plugin is similar to websockets (and web workers), so it could be quite straightforward to have one API for both

And also, having this completely asynchronous way to communicate with native extension makes it easier to port to phonegap plugins for instance, as on iOS, only asynchronous calls are possible.

[Ajay Prabandham]

Hi Richard,

                      What is the nature of help you are looking at? And what is the skill set you are looking at? I would like to contribute on a consistent basis, since i am motivated to learn more about browsers and i have great respect for your organization and technology... 

Regards,
Ajay

[Richard Bateman]

Ajay,

Right now we need to start researching technologies.  Recently I got a lead that indicated that it might be possible to use PPAPI in chrome even when not in NaCL; if this is true, that's probably the best option.  If you want to do some research on that and report back, it'd be helpful.

Other technologies I need to know more about are:

NaCL

Native Messaging

Basically we're in the research stage right now; I'm usually in the IRC room (npapi.com/chat) during business hours, GMT-0600 (-0700 soon), and we can chat there.  I have a wedding to attend today, though, so I'm not 100% certain what my schedule will be like.

The person getting married is on this list, though... =]

Richard

--

[Ajay Prabandham]

Sure Richard, i shall start on the R&D activity asap, which is tomorrow morning...

Regards,

Ajay

[Vikram Ranabhatt]

Richard I am interested  in NPAPI replacement. Please let me know how can I contribute in it.

-Vikram

[Richard Bateman]

Vikram,

Right now we mainly need to pick technologies and learn more about them.  Are there any of the possible alternatives that you'd like to take point on researching?

Richard

On Thu, Oct 3, 2013 at 3:08 PM, Vikram Ranabhatt <vikramr...@gmail.com> wrote:

Richard I am interested  in NPAPI replacement. Please let me know how can I contribute in it.

-Vikram

--

[John Tan]

Hi Richard,

I did do some search on the possibility of using ppapi without nacl the day I heard of this news. Although it seems possible, I'm afraid it's shaky path as Google may tweak it such that it will have to be used together with nacl.

[Richard Bateman]

I have had the same concern, but as they will have to leave it open enough for Flash to use it seems worth investigating. I have also seen some evidence to support the idea that Google may accept this is a viable option.  If they will do so, then that is probably our best option.  All I have currently is hearsay, but I'm working on that.

Richard

On Thu, Oct 3, 2013 at 5:13 PM, John Tan <tjk...@gmail.com> wrote:

Hi Richard,

I did do some search on the possibility of using ppapi without nacl the day I heard of this news. Although it seems possible, I'm afraid it's shaky path as Google may tweak it such that it will have to be used together with nacl.

[Joel Anderson]

So I wanted to point out a couple of solution that would have to be written per-platform and maybe per browser but might have other advantages,

All these require a service running in the background.

Setparent on windows (not sure what it is on other platform) - this allows one to take a top level window (of any app) and put it into amother window/control such as a picture.  On Windows you can also just pass in the window handle on creation.  This would only solve part of the issue for visible plugins... You would still need some form of IPC.  It seems very secure and would be faster than tcping images over.

Another area worth investigating is taking over the control window.  Hooking could be used for full control... I use hooking a lot as part of my job and there are lots of hooking libraries out there like easyhooks.  You could hook every browser and createwindowA/W (or if that does not work the much lower level ntusercreatewindow).  Then you could u could go and hook other things:

- Ipc to a service (if it that is the easiest approach) could be done by overwriting a commonly used Windows function like createfile or wsaconnect and called by java script.  Shared memory or pipes would be faster than tcp.

- You could hook individual functions however the more you hook the more you would have to do on a browser specific bases... I might be a more special case thing.

It should be possible to update firebreath in such a way to emulate most of its common functionality.

Anyway those are my quick thoughts at the moment.

As to what we need in chrome.

We have a particular realtime implementation of h264 and other video formats.  At a minimum having access to a very fast h264 decoder(or vp8 or vp9 or h265) that can decode each frame slice we send it might be enough however it would be preferable to be able to write this code ourselves because there are so many edge cases when you cant buffer frames.  It needs to run 720p at least 80fps so it can catchup when it gets behind.  JavaScript does not have intrinsics, fast multthreading, fast memory operations or any of the other tools we use to make this fast.

An opus sound decoder that can take a opus frame and play it back in real time.

[John Tan]

If you are thinking of grabbing an HWND from the NaCl window and hook your window onto it, it's not going to work as NaCl windowless. I've tried looking for one using Spy++.

[Richard Bateman]

Setparent on windows is not, in my opinion, a good solution because it’s very nearly impossible to be certain that you get the correct window; in fact, it’s very difficult to know that there *is* a correct window that will even work.  It also entirely excludes needs of windowless plugins, though of course if you could somehow pull it off it would be better than nothing.  On Mac OS this approach is, to the best of my knowledge, entirely impossible.

These are the types of thoughts we need to be having, though; I’m still hoping that the PPAPI idea may pan out.

Richard

[mgt]

For a web socket server running as on OS background process I have the following question. In firebreath I can use security zones to prevent the plugin from functioning if loaded from a domain which the plugin doesn't trust (for example the plugin will only function if loaded from domain X for some reason). If we use a local web socket server I'm not sure if a similar safety net could be used? I'm not sure how the local web socket server could authenticate that the client is actually trying to connect from a page loaded from domain X for example, would it be possible? 

[Richard Bateman]

I can’t think of a way to do it offhand :-/

Richard

On Oct 17, 2013, at 11:26 , mgt <m...@home.se> wrote:

For a web socket server running as on OS background process I have the following question. In firebreath I can use security zones to prevent the plugin from functioning if loaded from a domain which the plugin doesn't trust (for example the plugin will only function if loaded from domain X for some reason). If we use a local web socket server I'm not sure if a similar safety net could be used? I'm not sure how the local web socket server could authenticate that the client is actually trying to connect from a page loaded from domain X for example, would it be possible? 

[Taran Rampersad]

SSL Certificate? :-)

Richard Bateman wrote:

I can�t think of a way to do it offhand :-/

Richard

On Oct 17, 2013, at 11:26 , mgt <m...@home.se> wrote:

For a web socket server running as on OS background process I have the following question. In firebreath I can use security zones to prevent the plugin from functioning if loaded from a domain which the plugin doesn't trust (for example the plugin will only function if loaded from domain X for some reason). If we use a local web socket server I'm not sure if a similar safety net could be used? I'm not sure how the local web socket server could authenticate that the client is actually trying to connect from a page loaded from domain X for example, would it be possible?�

--
�

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
�

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
--
Taran Rampersad
LinkedIn Profile
KnowProSE.com

[Richard Bateman]

That still wouldn’t tell you for sure which website contacted the plugin.

Richard

On Oct 17, 2013, at 18:18 , Taran Rampersad <taran.a....@gmail.com> wrote:

SSL Certificate? :-)

Richard Bateman wrote:

I can’t think of a way to do it offhand :-/

Richard

On Oct 17, 2013, at 11:26 , mgt <m...@home.se> wrote:

For a web socket server running as on OS background process I have the following question. In firebreath I can use security zones to prevent the plugin from functioning if loaded from a domain which the plugin doesn't trust (for example the plugin will only function if loaded from domain X for some reason). If we use a local web socket server I'm not sure if a similar safety net could be used? I'm not sure how the local web socket server could authenticate that the client is actually trying to connect from a page loaded from domain X for example, would it be possible? 

--

 
---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
 

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
--
Taran Rampersad
LinkedIn Profile
KnowProSE.com

--

[Rich Compton]

The way we are dealing with it is hacky but better than nothing.� We will be sending a specific message (code) that is generated from our web server through the browser (web app) to the web socket.� First thing the web socket does on connection is verify with our server that it is indeed coming from a connection we expect.�

Somewhat of a hack and not 100% secure but it is the best option we have thought of thus far.� Definitely open to other options.�

Richard Compton

On 10/18/2013 1:04 PM, Richard Bateman wrote:

That still wouldn�t tell you for sure which website contacted the plugin.

Richard

On Oct 17, 2013, at 18:18 , Taran Rampersad <taran.a....@gmail.com> wrote:

SSL Certificate? :-)

Richard Bateman wrote:

I can�t think of a way to do it offhand :-/

Richard

On Oct 17, 2013, at 11:26 , mgt <m...@home.se> wrote:

For a web socket server running as on OS background process I have the following question. In firebreath I can use security zones to prevent the plugin from functioning if loaded from a domain which the plugin doesn't trust (for example the plugin will only function if loaded from domain X for some reason). If we use a local web socket server I'm not sure if a similar safety net could be used? I'm not sure how the local web socket server could authenticate that the client is actually trying to connect from a page loaded from domain X for example, would it be possible?�

--
�

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
�

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
--
Taran Rampersad
LinkedIn Profile
KnowProSE.com

--
�

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
�
---
You received this message because you are subscribed to a topic in the Google Groups "firebreath-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebreath-dev/gqgrUg5orwk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebreath-de...@googlegroups.com.

[Taran Rampersad]

It could. You'd have to set up a certificate specifically for the application and for that site. A signed SSL certificate is also verified by an external authority - it's specific to a server - so the only way past it is if someone compromises the SSL certificate on the server itself.

That said, the same concept applies. What Richard presented is the same concept. In looking at it from a perspective based on a present Firebreath project I'm working on, saving to the registry or an ini equivalent leaves a big opening for someone who really wants to subvert the security of the system. With the SSL certification framework, it's been pretty hardened over the years.

I'm not saying it's perfect. I'm not even saying it's a great solution. It's a stopgap. But it's a functional stopgap.

Richard Bateman wrote:

That still wouldn�t tell you for sure which website contacted the plugin.

[Richard Bateman]

I must be missing something in your explanation, then.  In the scenario I’m envisioning, you will contact your localhost websocket server from your web page; it doesn’t matter if the web page itself is on SSL, because that ssl verification wouldn’t be passed on to the websocket server.  If the localhost websocket server has ssl, that doesn’t matter because any page could be doing the contacting.  How would you verify which web page actually contacted the websocket?

I may well be missing some important piece; I haven’t yet used websockets, so I’m not 100% familiar with them.

Richard

On Oct 18, 2013, at 14:37 , Taran Rampersad <taran.a....@gmail.com> wrote:

It could. You'd have to set up a certificate specifically for the application and for that site. A signed SSL certificate is also verified by an external authority - it's specific to a server - so the only way past it is if someone compromises the SSL certificate on the server itself.

That said, the same concept applies. What Richard presented is the same concept. In looking at it from a perspective based on a present Firebreath project I'm working on, saving to the registry or an ini equivalent leaves a big opening for someone who really wants to subvert the security of the system. With the SSL certification framework, it's been pretty hardened over the years.

I'm not saying it's perfect. I'm not even saying it's a great solution. It's a stopgap. But it's a functional stopgap.

Richard Bateman wrote:

That still wouldn’t tell you for sure which website contacted the plugin.

Richard

On Oct 17, 2013, at 18:18 , Taran Rampersad <taran.a....@gmail.com> wrote:

SSL Certificate? :-)

[Steven Lybeck]

It looks like WebSocket requires the cross-origin request headers of modern browsers. That means the browser will send a header like "Origin: http://www.domain.com/" that corresponds to the web page attempting to communicate with the WebSocket, and the server will send a corresponding "Access-Control-Allow-Origin: http://www.domain.com/" header with the WebSocket upgrade response.

In our case, it's up to the WebSocket server fronting the plugin code to examine the Origin header and decide whether it wants to continue communicating with the web page.

Here is the relevant part of the WebSocket spec: http://tools.ietf.org/html/rfc6455#section-10.2

Compared to a lot of the other concerns with transitioning away from NPAPI, this might be pretty straightforward!

[Steven Lybeck]

One of the things we really appreciate about browser plugins is that they can be put in unprivileged directories without requiring permission elevation to install.

In the case of a WebSocket server, I assume the best solution is installing as a system service. Do you guys know if there is any way to set up system services on OSX and/or Windows without permission elevation? Or is there another method to keep a service running through reboots - perhaps some kind of way to run our own daemon supervisor at startup without installing it as a service?

I know a lot more about web than native development, so I'm kind of out of my depth here. Our main goal is minimizing hoops for our users to jump through.

[mgt]

Sounds like a good idea to use the origin header information to solve my concern about malicious sites trying to use your local websocket server then. Thanks for sharing.

[vikram ranabhatt]

Port number will be an issue as standard port 80/443 can  used by other application running on the user machine.

We choose other port than we need open it in the firewall as well ?

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "firebreath-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebreath-dev/gqgrUg5orwk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebreath-de...@googlegroups.com.

[Neil Griffiths]

No. The firewall isn't involved if you're binding to a port on localhost (127.0.0.1). But yes, with this technique you're always going to have a potential issue where you couldn't bind to a port because something else was already bound to it!

--
 
---

You received this message because you are subscribed to the Google Groups "firebreath-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.

[vikram ranabhatt]

I found one more thing interesting.I was testing Web socket client  in IE 11.It seems it doesn't allow loop back as call to localhost is failing.

I guess micro soft has blocked loop back option.

[dougma]

That's correct since Windows 8.

This webpage has a lot of stuff to read and understand:

http://blogs.msdn.com/b/ieinternals/archive/2012/03/23/understanding-ie10-enhanced-protected-mode-network-security-addons-cookies-metro-desktop.aspx

[kh...@chromium.org]

среда, 25 сентября 2013 г., 2:50:47 UTC+4 пользователь John Tan написал:

Hi Richard,

 

Count me in! We are on the same boat now....

 

As you may know, we are developing our own video player. One major issue we are facing now is that we have our own video codec which has already been optimized to x86 and x64 in assembly language. Seem to me, such optimization will not be portable to nexe by just dropping in the code.

You can not "just drop the code" but you can manually adjust it. It's relatively simple but somewhat manual process (you'll need to shuffle your code a bit to make sure it'll be accepted by validator which usually imposes some speed penalty). Of course it'll only work for NaCl (== applications from web store), not for PNaCl (== applications from the world wide web).

 

So, what I could think of is

1) Write a NaCl app as the front player

2) Separate out the decoder engine into a local service which will decode the video and locally stream the decoded video to the NaCl app via websocket .

 

Unfortunatelly it'll not work. All known communication channels between browser and native code max out at about 10MB/sec.

 

My concern to the above proposal is

1) Imagine, if the video is 1920x1080 at 24-bit and 30fps, that's going to be 186 MB per sec of data! How taxing will it be to the system, even if it's streamed via localhost?

2) Multiple concurrent player would seem impossible.

 

Also, has anyone taken a look at zeromq? I wonder if it's usable in NaCl.

It may be possible to port zeromq to NaCl, but it'll not change the fact that you max out at about 10MB/sec.

[John Tan]

Thanks for the valuable feedback! I guess you've just about nailed the last nail into the coffin.... (><)

[vikram ranabhatt]

Whats the conclusion of the discussion ?.Did we find any good alternative to NPAPI?

--

[Francesco Cantarella]

In this page http://www.chromium.org/developers/npapi-deprecation at chapter "Exceptions" I read this:

Enterprise administrators will be able to whitelist specific NPAPI plug-ins by adding them to the EnabledPlugins policy list.

So, if I create an exe installer that adds registry keys, Chrome will work with "our firebreath" plugin???

[Francesco Cantarella]

It seems that we need an ADM/ADMX template.

[Richard Bateman]

If would be awesome if you could share with the group any info you find on how this all works.  Note that this is just a short-term solution according to what I have seen.

Richard

--
 
---

You received this message because you are subscribed to the Google Groups "firebreath-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.

[ValB]

I've been following the discussion here and on the Firebreath page.  We have a plug-in using Firebreath and would (probably) have to move towards a Native Messaging API solution if we are going to be on Chrome going forward.  I guess my question is, is there any plan afoot to either expand Firebreath or create a new framework, that would support either Native Messaging API implementations or PPAPI/NaCl (or PNaCl)?  I understand there are a lot of barriers and the burden is on the community to support this . . . but I am unclear whether there is consensus or some idea about moving forward or if it is all just DIY at this point?   It seems to me that time is running short . . .

On Tuesday, September 24, 2013 2:00:43 PM UTC-7, Richard Bateman (taxilian) wrote:

Alright.  Let me first start by stating what we already know: The writing is on the wall, the browser manufacturers have decided to kill NPAPI.  I think this is a bad idea, I think it’s stupid… however, it seems that anything that doesn’t have at least 1% of the entire world’s population (small number, right? it’s just 1%) is safe to throw under the bus as far as Google is concerned.

Anyway, all frustration aside, it’s time to start figuring out other solutions; it’s possible that those solutions will fragment this community into several projects, each aimed at a different problem. I don’t know.  The important thing, though, is that now we need to identify what things can and can’t be done.  I have been in touch with Stuart Morgan, who is one of the Chrome developers who has worked with some Chrome NPAPI stuff, and his request is that we find a list of things that can’t be done with the other technologies listed in the blog post.

To that end, I have created a page on firebreath.org that I want us to all start working on:

http://www.firebreath.org/x/kQCt

I think we’re going to need to break things down more, but frankly I need help.  If I get the help, then I will work with those willing to work with me until we get something figured out and at the very least can go to the browser manufacturers with a list of “this is what we can do, this is what we need to be able to do” items and have at least some documentation on alternatives.  My hope is to create a framework that possibly uses FireBreath, possibly is totally different, but solves 90% of the issues in whatever way we need.  I don’t yet know for sure if that’s possible.  This is going to cost us all a lot of time.

If I don’t get that help then I’ll solve my problems and let everyone else figure it out on their own time.  I’m sorry to put it that way, but I just can’t do this all myself; I need the help, and right now I’m tired and cranky and frustrated =]

My thoughts so far

===============

I think we will need to break things down into subtypes of plugins; the goal is to build something that simplifies cases for those types.  Here are the groupings I’ve thought of so far, please let me know what you think:

**

Video / Drawing plugins

**

As I’ve researched, the best chance for most of these will be to use a NaCl plugin, at least from a drawing perspective.  You could alternatively try to draw using WebGL or Canvas, but if you need any native code running, it’ll have to be NaCl.  For network access, we’ll have to look into Native Messaging as a backend to the NaCl plugin, accessed asynchronously — and yes, I can see this will be messy.  it’s the only solution I see.

**

Hidden / hardware access plugins

**

Plugins that are used for hardware access or to provide behind-the-scenes services to plugins I think we can handle using Native Messaging; after installing, a chrome extension can launch communicate with the other application, thus allowing us to make requests for the other application to handle things.  We should be able to abstract this with a javascript API, possibly utilizing JSAPI on the other side to automatically create wrapper objects with common code that would let us make asynchronous calls that still look and feel like javascript and serialize the requests and responses with JSON.

—

I’m sure there are other types, but this is the current direction of my thoughts.  What other ideas do we have, and who is willing to help? I need someone to research — in depth — each of these things and then I’d like to discuss it with them to see if we can hammer it together into a unified solution.

Richard

[Richard Bateman]

I'm definitely willing to help with (or even act as project manager for, if that's desired) an effort such as this but so far we don't have a particular plan as to what type of abstraction we can do.

You and others may have noticed that I've been less and less active and responsive over the last two years.  I have chalked this up to being more and more overwhelmed with my normal job with everything I need to do, but recently I have discovered that there is actually another reason. It seems that I am suffering from fairly severe sleep apnea, and the result is that my level of concentration has not been nearly at the level it was when I originally wrote FireBreath.  Between that and not feeling like I have gotten much help from the community I've not spent as much time looking into this as I really ought to.

That said, now that I know the problem I am working on finding solutions; initial tests with a CPAP machine have been uncomfortable but surprisingly dramatic in the results (a few hours sleeping with a CPAP actually left me more alert than a normal 9 hours of sleep in bed) and I am hoping to have my own soon and that it will help restore me to my previous level of ability.  Once that happens I hope to be able to start getting a lot more done again.

In the mean time, it's only my concentration and short term memory that seems to be dramatically affected, so if you'd like to hit me up on IRC I'd be happy to talk over any ideas you may have.  It sounds like your needs and ours (at my day job) may be similar enough to make a joint effort productive.

Richard a.k.a. taxilian

FireBreath founder

--

---
You received this message because you are subscribed to the Google Groups "firebreath-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebreath-de...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[John Tan]

Hi Richard,

Since you mentioned sleep apnea, I'll share with you my experience. I had sleep apnea since early 30s. At one point of time, it was so bad that part of my palate collapsed as I must have sucked it so hard while breathing during my sleep. After a sleep study, I was recommended to use a CPAP and the first machine I had was a Puritan Bennett APAP. I had it for a few years, followed by a second device which I forgot the brand as I used it for a short period of time before I stop using it altogether. I have stopped using it for past 3 years ever since I lost weight by changing my diet and lifestyle. So, while in the short term, CPAP is essential, change to a healthier lifestyle is definitely the way to go... :-)

[Richard Bateman]

I totally agree with you there, John, but it's sure easier to change your lifestyle when you're not tired and incapable of concentrating....

Step one of the plan is to get the cpap, then we'll work on spending a bit less time in front of the computer and a bit more time moving around, as well as other related changes =]

Richard

On Fri, Mar 7, 2014 at 9:19 PM, John Tan <tjk...@gmail.com> wrote:

Hi Richard,

Since you mentioned sleep apnea, I'll share with you my experience. I had sleep apnea since early 30s. At one point of time, it was so bad that part of my palate collapsed as I must have sucked it so hard while breathing during my sleep. After a sleep study, I was recommended to use a CPAP and the first machine I had was a Puritan Bennett APAP. I had it for a few years, followed by a second device which I forgot the brand as I used it for a short period of time before I stop using it altogether. I have stopped using it for past 3 years ever since I lost weight by changing my diet and lifestyle. So, while in the short term, CPAP is essential, change to a healthier lifestyle is definitely the way to go... :-)

--

[John Tan]

Totally agree with you too that CPAP is a must now to solve the immediate problem...

[Levi Taule]

Hi,

Is FireBreath going to die now? Is it actually worth learning it?

The thing is: 

I have a project idea ( though it isn't my idea :D) - browser plugin (cross-browser, hopefully :D). The plugin is going to pretty complex:

• rendering of all kinds (OpenGL will be used)
• audio
• networking

However, if FireBreath won't support creation of plugins for all the major browsers (Chrome, Mozzila, Safari, Opera, IE, maybe Konqueror) due to the fact that those browsers won't support NPAPI, and FireBreath won't find any solution, programmers will have to create 4 or 5 plugins (well, not completely, but still, it will be a lot of work) instead of one.
So finally, the question(s) is/are: Have the FireBreath guys found the solution? Is FireBreath going to support creation for all the major browsers? If not, is there any alternative out there, so the programmer can create one port of his application/library and not many of them?

BTW, does somebody know how folks in Unity are going to solve this? NPAPI is going to thrown away soon, so the had to find the solutions.

Thanks for the replies. Levi

Dňa utorok, 24. septembra 2013 23:00:43 UTC+2 Richard Bateman (taxilian) napísal(-a):

[Richard Bateman]

Currently the only browser that has announced plans to stop supporting NPAPI is Chrome.  Mozilla has introduced Click to Play, but to the best of my knowledge does not currently plan to actually discontinue NPAPI support.

If you look at recent posts on this list, you'll see that I've addressed what we know and where we're at very recently.  To summarize, FireBreath is still the best solution for all browsers but Chrome, and for Chrome you'll have to find another solution.  We don't have any other solutions that solve all cases.

As in all cases, you should never create a plugin if there is another method that will work; you might be able to use webgl, html5 audio, and websockets to solve your problems.  If you can, you should absolutely look at doing that instead of using a plugin.

Richard

[Sylvain Chevalier]

> As in all cases, you should never create a plugin if there is another method
> that will work; you might be able to use webgl, html5 audio, and websockets
> to solve your problems. If you can, you should absolutely look at doing
> that instead of using a plugin.
>

You could add web workers, and emscripten which allows you to
"compile" C or C++ into JavaScript. There is very little remaining
that can not be done directly in a (modern) web browser.

[Victor Khimenko]

On Thu, Jul 10, 2014 at 10:04 PM, Levi Taule <levi....@gmail.com> wrote:

Hi,

Is FireBreath going to die now? Is it actually worth learning it?

The thing is: 

I have a project idea ( though it isn't my idea :D) - browser plugin (cross-browser, hopefully :D). The plugin is going to pretty complex:

• rendering of all kinds (OpenGL will be used)
• audio
• networking

However, if FireBreath won't support creation of plugins for all the major browsers (Chrome, Mozzila, Safari, Opera, IE, maybe Konqueror) due to the fact that those browsers won't support NPAPI, and FireBreath won't find any solution, programmers will have to create 4 or 5 plugins (well, not completely, but still, it will be a lot of work) instead of one.
So finally, the question(s) is/are: Have the FireBreath guys found the solution? Is FireBreath going to support creation for all the major browsers? If not, is there any alternative out there, so the programmer can create one port of his application/library and not many of them?

BTW, does somebody know how folks in Unity are going to solve this? NPAPI is going to thrown away soon, so the had to find the solutions.

Hmm... Why? 90% of their users don't care about browser support at all, which means that NPAPI death will barely be even noticed.

 

Thanks for the replies. Levi

Dňa utorok, 24. septembra 2013 23:00:43 UTC+2 Richard Bateman (taxilian) napísal(-a):

Alright.  Let me first start by stating what we already know: The writing is on the wall, the browser manufacturers have decided to kill NPAPI.  I think this is a bad idea, I think it’s stupid… however, it seems that anything that doesn’t have at least 1% of the entire world’s population (small number, right? it’s just 1%) is safe to throw under the bus as far as Google is concerned.

Anyway, all frustration aside, it’s time to start figuring out other solutions; it’s possible that those solutions will fragment this community into several projects, each aimed at a different problem. I don’t know.  The important thing, though, is that now we need to identify what things can and can’t be done.  I have been in touch with Stuart Morgan, who is one of the Chrome developers who has worked with some Chrome NPAPI stuff, and his request is that we find a list of things that can’t be done with the other technologies listed in the blog post.

To that end, I have created a page on firebreath.org that I want us to all start working on:

http://www.firebreath.org/x/kQCt

I think we’re going to need to break things down more, but frankly I need help.  If I get the help, then I will work with those willing to work with me until we get something figured out and at the very least can go to the browser manufacturers with a list of “this is what we can do, this is what we need to be able to do” items and have at least some documentation on alternatives.  My hope is to create a framework that possibly uses FireBreath, possibly is totally different, but solves 90% of the issues in whatever way we need.  I don’t yet know for sure if that’s possible.  This is going to cost us all a lot of time.

If I don’t get that help then I’ll solve my problems and let everyone else figure it out on their own time.  I’m sorry to put it that way, but I just can’t do this all myself; I need the help, and right now I’m tired and cranky and frustrated =]

My thoughts so far

===============

I think we will need to break things down into subtypes of plugins; the goal is to build something that simplifies cases for those types.  Here are the groupings I’ve thought of so far, please let me know what you think:

**

Video / Drawing plugins

**

As I’ve researched, the best chance for most of these will be to use a NaCl plugin, at least from a drawing perspective.  You could alternatively try to draw using WebGL or Canvas, but if you need any native code running, it’ll have to be NaCl.  For network access, we’ll have to look into Native Messaging as a backend to the NaCl plugin, accessed asynchronously — and yes, I can see this will be messy.  it’s the only solution I see.

**

Hidden / hardware access plugins

**

Plugins that are used for hardware access or to provide behind-the-scenes services to plugins I think we can handle using Native Messaging; after installing, a chrome extension can launch communicate with the other application, thus allowing us to make requests for the other application to handle things.  We should be able to abstract this with a javascript API, possibly utilizing JSAPI on the other side to automatically create wrapper objects with common code that would let us make asynchronous calls that still look and feel like javascript and serialize the requests and responses with JSON.

—

I’m sure there are other types, but this is the current direction of my thoughts.  What other ideas do we have, and who is willing to help? I need someone to research — in depth — each of these things and then I’d like to discuss it with them to see if we can hammer it together into a unified solution.

Richard

--

[Alex Charbonnet]

> There is very little remaining
> that can not be done directly in a (modern) web browser.

I think there's a long tail on that. There are a lot of critical little
things that can't be done any other way. For example, we need to print
receipts automatically from the browser, without popping up a dialog.
No browser has a way to do that without a plugin.

If the SSL problem could be solved, web sockets might work. It's not
nearly as elegant as a plugin though.

[Spike McLarty]

> There is very little remaining
> that can not be done directly in a (modern) web browser.

What is the phrase "very little" even doing in a technical discussion?

There's "very little" iron in your body.  "Very little" of the sun's output reaches the earth.

There's "very little" difference between U-235 and U-238.

The idea that what applications do is display information and collect mouse events is an extremely Google view of the world. It seems beyond their imagining that an application would actually interact with the local operating system, much less local hardware.  'What wudya wanna do that for? Everything important happens on the web!!"   Hmm... not quite.

[Richard Bateman]

At the end of the day, you're preaching to the choir.  These facts remain:

1) You can do far more with native HTML5 than most people realize.  Before using a plugin, you should absolutely look into this and see if you can solve the problem *without* a plugin.

2) There exist many things, particularly dealing with unusual hardware interface requirements, complex networking tasks, and some types of rendering, which cannot currently be done without a plugin.  Many of these things can be partially solved with NaCL or Native Client, which will only work on Chrome, and in some cases that will still not be enough.

3) There is absolutely no benefit to arguing amongst ourselves here on the list.

It is my hope that browser manufacturers will continue to keep these facts in mind and will not fully drop out NPAPI support until they have resolved those things.  I am personally working on alternate methods for GradeCam so that if they do pull the plug I won't be up a creek without a paddle, but we'll just have to wait and see what happens.

Richard

[John Dexter]

Yeah Spike but then why are you forcing it into the browser into the first place ;)

[Sylvain Chevalier]

> those things. I am personally working on alternate methods for GradeCam so
> that if they do pull the plug I won't be up a creek without a paddle, but
> we'll just have to wait and see what happens.
>

Richard, have you looked at getUserMedia, for which there is also a
Flash fallback (https://github.com/addyosmani/getUserMedia.js)? At
least for audio, things that required a plug-in just a few months ago
can now be done entirely with JavaScript/HTML/CSS (like speech
recognition https://github.com/syl22-00/pocketsphinx.js).

[Richard Bateman]

That is what I'm using in cases where it is supported. I wasn't aware of the flash fallback -- that might be worth looking into.

Richard