How to return successive arrays from C++ to Java

[キャロウ マーク]

Hi,

I have an Embind interface to native code for transcoding textures that needs to return the, potentially large, array containing the transcoded texture. Currently it does this:

std::vector<uint8_t> dst;
dst.resize(getTranscodedImageByteLength(static_cast<transcoder_texture_format>(cTargetFormat),
width, height));

// code to fill dst with the transcoded image ...

val ret = val::object();
ret.set("error", static_cast<uint32_t>(error));
if (error == KTX_SUCCESS) {
// FIXME: Who deletes dst and how?
ret.set("transcodedImage", typed_memory_view(dst.size(), dst.data()));
}
return std::move(ret);
}

This is broken. Successive calls overwrite the image returned in earlier calls. Obviously this can be fixed by copying the data but I don’t want to do that. Can I replace

std::vector<uint8_t> dst;

with

std::vector<uint8_t> dst = new std::vector<uint8_t>;

If so, how does the receiving JS side free the memory when done with it? The Emscripten documentation says to use `.delete()` but on what you call that is not made clear. Also will I need to add an Emscripten compile option to allow memory growth if I use `new`?

Regards

-Mark

[Gabriel Cuvillier]

Hi,

Of course the current code can't work, as the typed_memory_view points
to some memory space that will be deleted at end of the fonction (as the
dst vector will be deleted since it is in the stack, regardless of the
std::move(ret) which does not move the vector but the emscripten::val).
The deleted memory space might be re-used later on, explaining why you
have the impression data is "overwritten" during successive calls.

Using "new std::vector" will prevent the memory delete/overwrite issue,
but as you noticed, it is not possible to .delete anything.


The best for your use case would be to create a new class
"TranscodedImage", containing the std::vector dst, and exposed using
Embind.  That class will hold responsibility of the vector, and exposing
the class using Embind will allow you to call .delete() on it later on.


Something like (pseudo-code):

class TranscodedImage {

    std::vector<uint8_t> dst;

    val get_typed_memory_view() {

return typed_memory_view(dst.size(), dst.data())

    }

    void compute_data( ... ) {

        // actually fills 'dst'

    }

}

EMSCRIPTEN_BINDINGS( mylib )
{
emscripten::class_<TranscodedImage>( "TranscodedImage" )
      .constructor( )
      .function( "compute_data", &TranscodedImage::compute_data )
      .function( "get_typed_memory_view()",
&TranscodedImage::get_typed_memory_view )

}


// In your javascript code

let trans_img = new Module.TranscodedImage()

trans_img.compute_data(... whatever needed to compute the image ...)

let img_data = trans_img.get_typed_memory_view()

... do whatever needed with img_data. Then, later on, when the data is
no more needed

trans_img.delete();    // will call the TranscodedImage class destructor
=> deletes the vector contents. So, everything is reclaimed in the end
(NB: img_data will no more point to something valid!!)


Note the .delete() call : this is called on the Emscripten-exposed C++ class


Hope this helps a bit,

Cheers,

Gabriel

[キャロウ マーク]

Obviously I mean to Javascript not Java. Sorry about the error.

[キャロウ マーク]

On Feb 24, 2020, at 16:01, Gabriel Cuvillier <gabriel....@gmail.com> wrote:

...

The best for your use case would be to create a new class "TranscodedImage", containing the std::vector dst, and exposed using Embind.  That class will hold responsibility of the vector, and exposing the class using Embind will allow you to call .delete() on it later on.

Thank you for your very helpful response.

Regards

    -Mark

[キャロウ マーク]

Gabriel

Something like (pseudo-code):

class TranscodedImage {

    std::vector<uint8_t> dst;

    val get_typed_memory_view() {

return typed_memory_view(dst.size(), dst.data())

    }

I’m trying to do this now but I’m getting the error

/Users/mark/Projects/github/KTX-Software/interface/js_binding/transcoder_wrapper.cpp:39:19: error: 

      no viable conversion from returned value of type 'memory_view<unsigned

      char>' to function return type 'emscripten::val'

           return typed_memory_view(image.size(), image.data())

                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

/Users/mark/Projects/github/emsdk/upstream/emscripten/system/include/emscripten/val.h:357:9: note: 

      candidate constructor not viable: no known conversion from

      'memory_view<unsigned char>' to 'emscripten::val &&' for 1st argument

        val(val&& v)

        ^

/Users/mark/Projects/github/emsdk/upstream/emscripten/system/include/emscripten/val.h:363:9: note: 

      candidate constructor not viable: no known conversion from

      'memory_view<unsigned char>' to 'const emscripten::val &' for 1st argument

        val(const val& v)

Any ideas. According to val.h "A val can be constructed by explicit construction from any C++ type.”.

Regards

    -Mark

[キャロウ マーク]

I’m trying to do this now but I’m getting the error

/Users/mark/Projects/github/KTX-Software/interface/js_binding/transcoder_wrapper.cpp:39:19: error: 

      no viable conversion from returned value of type 'memory_view<unsigned

      char>' to function return type 'emscripten::val'

           return typed_memory_view(image.size(), image.data())

                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Never mind. Changing it to

    return val(typed_memory_view(image.size(), image.data())); fixed the problem.