cant recognize yuv images

444 views
Skip to first unread message

Fatih Halimoglu

unread,
May 1, 2021, 4:25:32 PM5/1/21
to doubango-ai
When we run recognizer with sample jpegs in assets folder there is no problem. (probably because they are rgb24)

If we try with images taken from our camera stream it doesnt work and gives the error below (probably because they are yuv)

----------------------------------------------
*[COMPV INFO]: /!\ Code in file '/home/ultimate/compv/base/image/compv_image_conv_to_rgbx.cxx' in function 'rgbx' starting at line #560: Not optimized -> No in-place conversion found for BGRA32/BGR24 -> RGB24. You should consider using RGBA32/RGB24 instead of BGRA32/BGR24
***[COMPV ERROR]: function: "Report()"
file: "/home/ultimate/ultimateOCR/lib/source/ultimate_ocr_tensorflowlite_interpreter.cxx"
line: "64"
message: [UltOcrTensorflowLiteInterpreter] [TFLITE] Reporting message: tensorflow/lite/kernels/detection_postprocess.cc:404 ValidateBoxes(decoded_boxes, num_boxes) was not true.
----------------------------------------------




Our cameras can provide h264 and h265 and mjpeg streams. 
Currently we are using ffmpeg to export jpeg files from mjpeg stream then
running ultimatealpr to recognize plate numbers but fails.

How can we use ultimatealpr with our cameras? What do you suggest?
Our capturing device has 6 cores and lots of ram. aarch64 ubuntu 20.04 -orangepi 4b-


The below code works for rgb but not yuv. How should we change it for yuv?
-please remember we dont use android-

                    result = CheckResult("Process", UltAlprSdkEngine.process(
                            (bytesPerPixel == 1) ? ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_Y : (bytesPerPixel == 4 ? ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_BGRA32 : ULTALPR_SDK_IMAGE_TYPE.ULTALPR_SDK_IMAGE_TYPE_BGR24),
                            nativeBuffer,
                            image.getWidth(),
                            image.getHeight(),
                            image.getWidth(), 
                            orientation
                    ));



Thanks in advance

Mamadou DIOP

unread,
May 1, 2021, 4:39:59 PM5/1/21
to Fatih Halimoglu, doubango-ai
- Off course you have to change the format from RGB to YUV. Use YUV420P: https://www.doubango.org/SDKs/anpr/docs/cpp-api.html#_CPPv4N15ultimateAlprSdk22ULTALPR_SDK_IMAGE_TYPEE
- Update your code to v3.3.2 if not already done: https://github.com/DoubangoTelecom/ultimateALPR-SDK/issues/135

To check if your format is correctly handled enable dumping the input and inspect “ultimateALPR-input.png”: https://www.doubango.org/SDKs/anpr/docs/Debugging_the_SDK.html

--
You received this message because you are subscribed to the Google Groups "doubango-ai" group.
To unsubscribe from this group and stop receiving emails from it, send an email to doubango-ai...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/doubango-ai/d0897d81-c54b-48ad-92ad-92fef0f09ccfn%40googlegroups.com.

Mamadou DIOP

unread,
May 1, 2021, 4:54:17 PM5/1/21
to Fatih Halimoglu, doubango-ai
I missed that you’re using RPI with ARM64. There is no v3.3.2 for that platform: https://github.com/DoubangoTelecom/ultimateALPR-SDK/issues/135#issue-811055182
Also, we don’ t provide commercial licenses for that platform.
Commercial licenses are only available for:
- Windows and Linux x64
- Android ARM32/64
- NVIDIA Jetson ARM64
- Raspbian ARM32

Mamadou DIOP

unread,
May 1, 2021, 6:50:23 PM5/1/21
to Fatih Halimoglu, doubango-ai


On 5/2/2021 12:24 AM, Fatih Halimoglu wrote:
C++ sample code doesn't help though we use Java.
This is no a "C++ sample code", it's a single-line code. If you can't convert it to Java we have a bigger problem.
and sample java code uses android libraries which are not present in java se

If you take time to check the code you'll see there is Java sample for all platforms (RPI, Linux, Windows...): https://github.com/DoubangoTelecom/ultimateALPR-SDK/tree/master/samples/java/recognizer



Mamadou DIOP <diopm...@doubango.org>, 1 May 2021 Cmt, 23:39 tarihinde şunu yazdı:

Fatih Halimoglu

unread,
May 7, 2021, 10:27:24 AM5/7/21
to doubango-ai
Could anyone help me to fill A and B. 
For A I can try all possible types. But B is really unknown. 
How do you convert Mat to ByteBuffer 
How do you obtain planes from Mat object ?
/*
planes[0].getBuffer(),
planes[1].getBuffer(),
planes[2].getBuffer(),
planes[0].getRowStride(),
planes[1].getRowStride(),
planes[2].getRowStride(),
planes[1].getPixelStride(),
*/


        CameraCapture app = new CameraCapture();     
        Mat mat = new Mat();
        VideoCapture capturedVideo = new VideoCapture();

        boolean isOpened = capturedVideo.open(""rtsp://login:password@ip:port/");      
        boolean isRead = capturedVideo.read(mat);        
        //isOpened and isRead is true
        
        if(!mat.empty()){
            UltAlprSdkEngine.process( A?,B?, mat.width(), mat.height()).json();
        }


Thanks

2 Mayıs 2021 Pazar tarihinde saat 01:50:23 UTC+3 itibarıyla Mamadou şunları yazdı:

Mamadou DIOP

unread,
May 7, 2021, 2:15:48 PM5/7/21
to Fatih Halimoglu, doubango-ai

OpenCV should return BGR. Don't specify the planes if the memory is continious



Sent from my Galaxy

Fatih Halimoglu

unread,
May 10, 2021, 8:23:52 AM5/10/21
to doubango-ai
I found a solution to convert but it seems slow

    public static ByteBuffer GetBufferedImageFromMat(Mat matImg) {        
        int type = BufferedImage.TYPE_BYTE_GRAY;
        if (matImg.channels() > 1) {
            type = BufferedImage.TYPE_3BYTE_BGR;
        }

        int bufferSize = matImg.channels() * matImg.cols() * matImg.rows();
        byte[] buffer = new byte[bufferSize];
        matImg.get(0, 0, buffer);
        BufferedImage image = new BufferedImage(matImg.cols(), matImg.rows(), type);
        final byte[] targetPixels = ((DataBufferByte) image.getRaster().getDataBuffer()).getData();
        System.arraycopy(buffer, 0, targetPixels, 0, buffer.length);

        ByteBuffer byteBuffer = ByteBuffer.allocateDirect(targetPixels.length);
        byteBuffer.order(ByteOrder.nativeOrder());
        byteBuffer.put(targetPixels, 0, targetPixels.length);
        byteBuffer.flip();

        return byteBuffer;
    }

7 Mayıs 2021 Cuma tarihinde saat 21:15:48 UTC+3 itibarıyla Mamadou şunları yazdı:

Mamadou DIOP

unread,
May 10, 2021, 1:15:23 PM5/10/21
to Fatih Halimoglu, doubango-ai

I'm not familiar with Java OpenCV but would definitely not do it like this. For 720p BGR image (1280*720*3), you'll be copying 2.7MB back and forth for each call. You may end spending more timing doing memory copy than ANPR processing.

[1] shows you can retrieve the address of the data wrapped by the Mat object

[2] shows you can create a direct byte buffer using that address and pass it to a native code (zero copy)

[1] https://docs.opencv.org/3.4/javadoc/org/opencv/core/Mat.html#dataAddr()

[2] https://stackoverflow.com/questions/16465477/is-there-a-way-to-create-a-direct-bytebuffer-from-a-pointer-solely-in-java

Reply all
Reply to author
Forward
0 new messages