ML Kit on Device Face detection API is Very Slow

[Charlize Smith]

ML Kit on Device Face detection API is Very Slow

I want to detect face from gallery Image. I tried with Mobile vision Api and detected face successfully. On website of mobile vision api, they have mentioned about Firebase MLKIT.
I also tried firebase ML Kit and detected face.

Issues
1. I noticed that Firebase MLKIT is performing very slow on Gallery Image Bitmap.
2. Can i still used mobile vision api for detection of face in image.( I want only detect face, dont want eyes,nose,etc)
3. What should i do to improve performance for detecting face with Firebase MLKIT.
4. I used Firebase Image Labeling. Firebase Image Labeling is performed fast but Face Detection is very slow comparatively.

[Shiyu Hu]

Hi Charlize,

In general, ML Kit should have very similar performance with Mobile Vision. It would be nice if you could provide some code snippets for the two cases?  More comments inline below.

On Tuesday, August 20, 2019 at 6:55:44 AM UTC-7, Charlize Smith wrote:

ML Kit on Device Face detection API is Very Slow

I want to detect face from gallery Image. I tried with Mobile vision Api and detected face successfully. On website of mobile vision api, they have mentioned about Firebase MLKIT.
I also tried firebase ML Kit and detected face.

Issues
1. I noticed that Firebase MLKIT is performing very slow on Gallery Image Bitmap.

Do you need to download the images before run detection? The download may take long time.

 

2. Can i still used mobile vision api for detection of face in image.( I want only detect face, dont want eyes,nose,etc)

We will deprecate the mobile vision APIs soon, so I would encourage you to use ML Kit if it is feasible. To detect face only, there are two ways to do so. 

Option 1. Use ALL_LANDMARKS

 FirebaseVisionFaceDetectorOptions options = new FirebaseVisionFaceDetectorOptions.Builder()   

            .setLandmarkMode(FirebaseVisionFaceDetectorOptions.ALL_LANDMARKS)

            .build());

Option 2. Use Face contour bounding box

FirebaseVisionFaceDetectorOptions options =

        new FirebaseVisionFaceDetectorOptions.Builder()

                .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)

                .build();

 

3. What should i do to improve performance for detecting face with Firebase MLKIT.

The option 2 above should have better performance for now, and please add those to the gradle file:

  implementation 'com.google.firebase:firebase-ml-vision:23.0.0'

  implementation 'com.google.firebase:firebase-ml-vision-face-model:18.0.0'

See https://firebase.google.com/docs/ml-kit/android/detect-faces for more details

[Charlize Smith]

Hello Shiyu Hu,

Thanks for your reply.

 I am providing some code snippets for the two cases as you requested. please let me know if i am doing anything wrong. I am providing image with byte count 38340864 (approx 38MB) to below function.

options = new FirebaseVisionFaceDetectorOptions.Builder()
            .setContourMode(FirebaseVisionFaceDetectorOptions.ALL_CONTOURS)
        .build();

detector = FirebaseVision.getInstance()
        .getVisionFaceDetector(options);

FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(myBitmap);

if (image != null) {
    detector.detectInImage(image)
            .addOnSuccessListener(
                    new OnSuccessListener<List<FirebaseVisionFace>>() {
                        @Override
                        public void onSuccess(List<FirebaseVisionFace> faces) {
                            long endTime = System.currentTimeMillis();
                            long MethodeDuration = (endTime - startTime);
                            if (faces.size() > 0) {
                                Toast.makeText(MainActivity.this,"Time Required "+MethodeDuration,Toast.LENGTH_SHORT).show();

                                for (int i = 0; i < faces.size(); ++i) {
                                    FirebaseVisionFace face = faces.get(i);

                                    PointF myMidPoint = null;


                                    Rect rect1 = face.getBoundingBox();
                                    int xCenter = rect1.centerX();
                                    int yCenter = rect1.centerY();

                                    Canvas canvas = new Canvas(myBitmap);
                                    Paint paint = new Paint(Paint.ANTI_ALIAS_FLAG);
                                    paint.setColor(Color.GREEN);
                                    paint.setShadowLayer(1f, 0f, 1f, Color.WHITE);
                                    paint.setStyle(Paint.Style.STROKE);
                                    paint.setStrokeWidth(6f);
                                    canvas.drawRect(rect1, paint);
                                    paint.setStyle(Paint.Style.FILL);
                                    canvas.drawCircle(xCenter, yCenter, 10, paint);
   }
            }
        })

.addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(@NonNull Exception e) {
        // Task failed with an exception
        // ...
        Log.d("FaceFragment2", "onFailure: " + "Firebase failed to detect face");
    }
});

I tried with your FirebaseVisionFaceDetectorOptions.ALL_CONTOURS but it still take 12 second to perform operation. Is it necessary to call detector.detect() method on background thread. I am performing all operation on Main Thread.
If i provide bitmap with some compression, it perform face detection fast but lost accuracy of face detection.
please help me out. Thank you