MLKit Text Recognition with Swift takes years until it recognizes first elements

[fuhrma...@gmail.com]

I'm desperate. For the past few days I've tried to implement Text Recognition on a simple live camera view. 

The code works, as in text is being recognized. But it's way form being perfect.

One problem is that either the text recognition starts after a few seconds or it recognizes text late:

To illustrate my problem I've set up a print("success") call after it recognizes the word "Mastercard" (see code below). 

When I started the camera I immediately faced it directly to this word, but it took a while until the console printed "success". 

Additionally the live view started to "lag", another problem which may be related to this one.

class CameraViewController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate {

    @IBOutlet weak var cameraView: UIView!
    var textRecognizer: VisionTextRecognizer!
    var lastFrame: CMSampleBuffer?


    override func viewDidLoad() {
        super.viewDidLoad()
        let vision = Vision.vision()
        textRecognizer = vision.onDeviceTextRecognizer()
        captureSession()
    }

    func captureSession () {
        let captureSession = AVCaptureSession()

        guard let captureDevice = AVCaptureDevice.default(for: .video) else { return }
        guard let input = try? AVCaptureDeviceInput(device: captureDevice) else{ return }
        captureSession.addInput(input)

        captureSession.startRunning()

        let previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
        cameraView.layer.addSublayer(previewLayer)
        previewLayer.frame = cameraView.frame

        let dataOutput = AVCaptureVideoDataOutput()
        dataOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "videoQueue"))
        captureSession.addOutput(dataOutput)
    }

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

        let metadata = VisionImageMetadata()

        let devicePosition: AVCaptureDevice.Position = .back

        let deviceOrientation = UIDevice.current.orientation

        switch deviceOrientation {
        case .portrait:
            metadata.orientation = devicePosition == .front ? .leftTop : .rightTop
        case .landscapeLeft:
            metadata.orientation = devicePosition == .front ? .bottomLeft : .topLeft
        case .portraitUpsideDown:
            metadata.orientation = devicePosition == .front ? .rightBottom : .leftBottom
        case .landscapeRight:
            metadata.orientation = devicePosition == .front ? .topRight : .bottomRight
        case .faceDown, .faceUp, .unknown:
            metadata.orientation = .leftTop
        }

        lastFrame = sampleBuffer
        let visionImage = VisionImage(buffer: sampleBuffer)
        visionImage.metadata = metadata
        textRecognizer.process(visionImage) { (features, error) in
            self.processResult(from: features, error: error)
        }
    }

    func processResult(from text: VisionText?, error: Error?) {
        guard let features = text else { return }
        for block in features.blocks {
            for line in block.lines {
                for element in line.elements {
                    if element.text == "Mastercard" {
                        print("success")
                    }
                }
            }
        }
    }

}

[Kato Richardson]

Hello fuhrmanntimde,

Have you already tried capturing the image at a lower resolution to see if that helps? It's mentioned in the tips to improve realtime performance.

☼, Kato

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/6f531568-b28f-4fec-83ca-f938dfd28938%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Kato Richardson |

Developer Programs Eng |

kato...@google.com |

775-235-8398