Bind to Async Call

[Joel Croteau]

I would like to have a function that makes an async call, and returns control to the browser during the call, then resumes once the async call is complete. However, I would like this to appear to the calling code to be a synchronous call. So for instance, I could do something like this:

struct FileInfo

{

    void* buffer;

    unsigned size;

};

FileInfo loadUrl(const char* url)

{

    FileInfo info;

    emscripten_async_wget2_data(url, "GET", "", &info, false, onload, onerror, onprogress);

    emscripten_pause(); // Blocks until emscripten_resume() is called

    return info;

}

void onerror(void* ptr, int code, const char* desc)

{

    // Handle not found, etc.

}

void onprogress(void* ptr, int loaded, int total)

{

    // Update progress bar

}

void onload(void* ptr, void* buffer, unsigned size)

{

    static_cast<FileInfo*>(ptr)->buffer = buffer;

    static_cast<FileInfo*>(ptr)->size = size;

    emscripten_resume(); // Now loadUrl resumes and returns FileInfo.

}

Is there any way to do this?

[Alon Zakai]

Currently not. It would require a very different code generation approach than we currently have.

In theory something similar could be done using generators, in very recent versions of firefox and chrome. But even with the help of generators, this is not easy to do, and it is not clear how it would affect performance - likely very adversely.

One similar thing you can do is manage your own code using c++11 lambdas. You could write a little system that continues to run the current function once a callback triggers that.

- Alon

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Joel Croteau]

I can understand using Javascript generators, but how would c++ lambdas help?

You received this message because you are subscribed to a topic in the Google Groups "emscripten-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/emscripten-discuss/Op5ifeMjauA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to emscripten-disc...@googlegroups.com.

[Alon Zakai]

It can capture its environment, so you can put the second half of the function in a lambda, and call the lambda later on when you want to, in response to something.

- Alon

[Joel Croteau]

I was able to implement this using Javascript Promises (I haven't made this a pull request because it's rather experimental):

  emscripten_sync_wget2_data: function(url, request, param, arg, free, onload, onerror, onprogress) {

    var _url = Pointer_stringify(url);

    var _request = Pointer_stringify(request);

    var _param = Pointer_stringify(param);

    var promise = new Promise(function(resolve, reject) {

      var http = new XMLHttpRequest();

      http.open(_request, _url, true);

      http.responseType = 'arraybuffer';

      // LOAD

      http.onload = function http_onload(e) {

        if (http.status == 200 || _url.substr(0,4).toLowerCase() != "http") {

          resolve(http.response);

        }

        else {

          reject(http.status, http.statusText);

        }

      };

      // ERROR

      http.onerror = function http_onerror(e) {

        reject(http.status, http.statusText);

      };

      // PROGRESS

      http.onprogress = function http_onprogress(e) {

        if (onprogress) Runtime.dynCall('viii', onprogress, [arg, e.loaded, e.lengthComputable || e.lengthComputable === undefined ? e.total : 0]);

      };

      // Useful because the browser can limit the number of redirection

      try {

        if (http.channel instanceof Ci.nsIHttpChannel)

          http.channel.redirectionLimit = 0;

      } catch (ex) { /* whatever */ }

      if (_request == "POST") {

        //Send the proper header information along with the request

        http.setRequestHeader("Content-type", "application/x-www-form-urlencoded");

        http.setRequestHeader("Content-length", _param.length);

        http.setRequestHeader("Connection", "close");

        http.send(_param);

      } else {

        http.send(null);

      }

    });

    promise.then(function(data) {

      var byteArray = new Uint8Array(data);

      var buffer = _malloc(byteArray.length);

      HEAPU8.set(byteArray, buffer);

      if (onload) Runtime.dynCall('viii', onload, [arg, buffer, byteArray.length]);

      if (free) _free(buffer);

    }, function(status, statusText) {

      if (onerror) Runtime.dynCall('viii', onerror, [arg, status, statusText]);

    });     

  },

Promises are still fairly new, and only supported by the most recent versions of Chrome and Firefox, but they do give us the functionality we want here. I think the best way to implement this in code would be if I could use the C++ promise and future APIs. You can use these if you compile with -std=c++11. All this seems to do at present is send the code into an infinite loop, but if they could map into equivalent Javascript promise calls, I think it could work quite well. At least in the case of emscripten_sync_wget2_data, it could be set to use synchronous XHRs as a fallback. This would still work, just require some extra conversion work, and wouldn't call onprogress events.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "emscripten-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/emscripten-discuss/Op5ifeMjauA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to emscripten-discuss+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-discuss+unsub...@googlegroups.com.

[Alon Zakai]

I didn't know about the JavaScript Promises stuff, that's interesting. To allow sync-looking code on the C++ side, it would need to integrate with something there, if there is a c++11 feature that works there, as you say, then this might work very well.

- Alon

To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.

[Joel Croteau]

The syntax is a little different, but the C++11 promise/future template offers essentially the same functionality. See http://www.cplusplus.com/reference/future/promise/ for the usage of this, and http://promises-aplus.github.io/promises-spec/ for the proposed Javascript spec.

To unsubscribe from this group and all its topics, send an email to emscripten-disc...@googlegroups.com.

[Alon Zakai]

Interesting, thanks. This summer I am hoping an intern will work on investigating sync/async transformations, this stuff is very relevant to that.

- Alon

[王璐]

Hello Alon & Joel,

   I think that c++/js promise is more like syntax sugar, I mean you are still writing async programs, but the syntax allows you to write in a sync way, which could be more comfortable. I doubt if `emscripten_sync_wget2_data` is a simple solution as long as you want a real sync function in C++, and I'm really interested if you make it works without an overhaul of other parts of your program.
 
   I also learned a little bit about generators before, I'm not sure if it will work deep inside the calling chain: For example, if A calls B and B calls C, where C is a generator, I doubt if it is possible for A to control (pause/resume) B & C, without letting B be aware of it. If this is not possible, it might be necessary to rewrite all the functions to handle generator pause/resume events, so essentially we are still writing async C++.

   No matter what library/syntax you are using, async C++ can always be translated with emscripten out-of-the-box, but it might be better to use the JS engine instead of a separate event loop implemented in C++ -- maybe this is what you meant in `emscripten_sync_wget2_data`?


   On the other hand, if you want a real sync C++ which is then magically transformed into async JS, AFAIK one possible solution is to mark async function in JS, then transform it using streamline.js or similar tools, this is what I have done with vim.js. But it's too hacky, and might not work well with the optimization phases in emscripten (reloop, ASM.js etc), so it might not be worth it for a large project.

   Hopefully we will have an elegant and effective solution after the summer.


   regards,
   - Lu