performance Issue when SKIA is used with JNI

[Pavel Dey]

Hi team,

I am an employee of Adobe India. I am using SKIA library for prototyping an android based video editor.

I am using JNI to call native code (C++), which uses Skia library. While measuring performance I am seeing a huge overhead. For example, I am applying gaussian blur on each frame of a 1980-by-1080 video at 30 fps. The average execution time on the CPP side is ~2.88 ms. But when I measure total execution time of the JNI native call from Java side (this obviously includes JNI overhead), it is coming around 22 ms. I have tried other custom effects like edge feather. There also the results are similar. JNI itself has some overhead, but it should not be this much. Is there any reason why using Skia with JNI is performing poorly? Any lead will be helpful.

Thanks

Pavel

[K. Moon]

(Just some general profiling advice, I don't have Skia-specific knowledge.)

JNI is pretty slow, especially on Android. I wouldn't be too surprised if it's really that bad, but I think the obvious place to start would be to profile more points in the execution. If the time from starting the JNI call from Java to C++ is the bulk of the delay, there's no point in looking at Skia (for example). A simple way to measure this would be to comment any work done in the native call (isolating it to just the JNI parts).

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/2a135618-fd72-4375-b74a-2169b1ade6b6n%40googlegroups.com.

[Pavel Dey]

Thanks for the suggestion. By debugging in that way I realised that JNI overhead is actually not much significant. Rather the code I am executing to apply effects on each frame is taking major time. It's just that the CPU was not able to capture the execution time for functions running on GPU correctly. Adding a flush() command before taking the timestamp on such functions exposed the actual execution time. But as per my latest observation few Skia library functions are taking unexpectedly high time.

When I am applying Edge feather on a 1920x1080 video at 30 fps. I have written a custom shader in SKSL. The call to CPP layer is coming from Android (through JNI).

1. The time taken to execute GrGLMakeNativeInterface() to get the GL interface is ~3.5 ms for each frame. This is used to get GrDirectContext and then finally create the SkSurface object and the corresponding canvas.

2. Time taken to execute the function responsible for applying edge feather takes ~12 ms. I feel this is way too high for any video editing application. It degrades the performance severely. For more clarity I am putting the code snippet for only edge feather.

Except for the first two lines inside the function, the rest of the code takes ~12 ms. I was initially doubting makeImageSnapshot(). But that is not at al costly. It's the code following that which is taking time.

void doEdgeFeather(SkCanvas* canvas, int featherVal){
SkSurface* surface = canvas->getSurface();
sk_sp<SkImage> image = surface->makeImageSnapshot();
//Using the API to apply effect
const char* sksl = R"(
uniform shader child;
uniform float imHeight;
uniform float imWidth;
uniform float pixWidth;
float max(float a, float b){
return (a>b)?a:b;
}
half4 main(float2 xy) {
float dist;
half4 color = sample(child);
dist = max(max(max(pixWidth-xy.x, xy.x-(imWidth - pixWidth)), pixWidth-xy.y), xy.y-(imHeight - pixWidth));

if(dist>0){
color.a *= (1-dist/pixWidth);
}
return color;
}
)";

SkPaint paint, p2;
sk_sp<SkRuntimeEffect> effect = std::get<0>(SkRuntimeEffect::Make(SkString(sksl)));
sk_sp<SkShader> shader = image->makeShader(SkSamplingOptions());
SkRuntimeShaderBuilder builder(effect);

builder.child("child") = shader;
builder.uniform("imHeight") = float(image->height());
builder.uniform("imWidth") = float(image->width());
builder.uniform("pixWidth") = float(featherVal);
shader = builder.makeShader(nullptr, false);
paint.setShader(shader);
canvas->clear(SK_ColorBLACK);
canvas->drawRect(SkRect::MakeWH(image->width(),image->height()), paint);
canvas->flush();
}

I mainly need suggestion on what should be changed if this is an inefficient way of applying shaders. Then probably I can make performance improvement for all such custom shaders. Edge feather is merely an example of that kind.

Thanks in advance for your help.

[Brian Osman]

It sounds like you're creating a new GL context and GrDirectContext every frame? You should absolutely be creating those once and re-using them. Otherwise you'll end up re-compiling the shaders and doing quite a bit of other redundant work.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/aa103fca-67e2-4750-979c-ad5974b07c0an%40googlegroups.com.

[Mike Reed]

After you're reusing context, you should try to reuse the effect itself

sk_sp<SkRuntimeEffect> effect = std::get<0>(SkRuntimeEffect::Make(SkString(sksl)));

Only make that once. You can call effect->makeShader() in your loop if you are changing the values of the uniforms.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAMcBjo7mWEmkGg5CLmNGm26WDg894tb-jvECXmXjrXyVBO661w%40mail.gmail.com.

[Pavel Dey]

Thanks for the suggestion. I tried both and the performance improved significantly. Now the effect time has come down to almost 1 ms.

One follow up question on Brian's suggestion about grDirectContext. I'm creating gl interface and only once now. From there I'm creating back end render target and finally getting surface. Is it necessary to delete the back end render target object after every use? Especially since I'm creating it in a loop. I saw some optional argument in the API to get surface object ,where I can pass a function pointer to delete the backend render target object once the need is fulfilled. But finding it little difficult to use. What's your opinion on this?

 I'm facing some other issues where I'm seeing some effects are not appearing in certain scenarios. I'm actually using two custom shaders and one blur in cascade. I'm debugging it now. Will share once I have more information on that.

Thanks again.

You received this message because you are subscribed to a topic in the Google Groups "skia-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/skia-discuss/EEyR_7DytkA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAERTzqz_W5g0EweZ6RAuSRD1e3udYg_9Ra9bfiarHsZFmK2RHQ%40mail.gmail.com.