64 Bit Image Decoder
Timothee Cazares
@kisvegabor thank you for your reply! I did look at that page. However, I do not know how to implement an open, size, read, and close callback. How do I write these callbacks? The documentation only tells me how to write the prototypes of these functions. Alternatively, is there anyway to open the file using the standard fopen instead of registering a driver? @embeddedt if I could get your help too that would be great!
Thank you @embeddedt! Increasing LV_MEM_SIZE did allow me to display the png but scrolling the png around made the entire screen lag. I tried increasing the display buffer size in main.c but that did not resolve the issue. However, I noticed that when I load the png image from a pre-converted file and use LV_IMG_DECLARE(variable name from binary C file), it does not lag at all. Do you know if there is anyway I can use lodepng to generate this C binary file so that the scrolling is a lot smoother? And after generating the RGBA values in the C binary file how would I create a struct to declare the image and eventually display it on the screen?
Did you happen to find an answer as to why pre-converting an image makes the lag go away? We have gif and png that unless pre-converted will lag upon animating. Lag occurs when any animation occurs and at least one displayed image is not pre-converted.
LittlevGL uses the info_cb to get info about the image. If the source is an initialized lv_img_dsc_t variable then it already has all the required info. If the source was path to the file then the file needs to be opened to get the required info. But the way to get the info depends on the implementation of the decoder.
Unfortunately, there is no more examples then the mentioned blog post. After the release of v6.0 we will work on adding more examples. And the image decoder interface definitely required more examples.
If you have a 300x300 image but only 300x10 display buffer (VDB) then LittlevGL will call open and close 30 times. (once for each chunk) Probably it would be reasonable to keep the last image opened. Or free it if it was not used for 1 sec or so.
Hi,
OSMC cannot display the iphone photo file with the extension .heic
I see that there is an image decodef for that at GitHub - xbmc/imagedecoder.heif: HEIF image decoder add-on for Kodi i wonder if somebody has compiled it for the vero4k?
thanks,
Panagiotis
I've been trying to figure out what this attribute actually does for a while now. Not full time of course (I'm not that sad! Honestly I'm not! Really!), but every so often I read another article where this comes up, or see advice to add this to massively boost image performance and I get curious again as to what it actually does.
In order to aid the user agent in deciding whether to perform synchronous or asynchronous decode, the decoding attribute can be set on img elements. The possible values of the decoding attribute are the following image decoding hint keywords:
Err... thanks for the technical explanation. But what does this actually mean in real life? Which setting should you use? Does it even matter? And if it does, why don't those clever browser engineers just set it to the best setting?
Seriously, that's just not how browsers work! How many times have you seen a page load without images being there yet? Loads of times right? Images are not typically render-blocking and if they were the web would be a very slow and painful place to be.
I'm not going to link to the article I pulled this example from, but it's the first one that comes up when you search for "what does decoding=async do" so that's depressing. Hopefully this post will displace that if I pray to the SEO Gods enough.
Modern browsers all decode images off the main thread, and have done so for a while now, leaving it free for other stuff. And any older, or more simpler, browsers out there that do still decode on the main thread are almost certainly not going to support this attribute. So, in theory, you are not going to free up the main thread with this attribute.
Maybe this wasn't the case when the attribute was originally proposed (in which case it would have been more important than it was now), I'm not sure, but browser engineers tell me it's definitely not the case now.
What's that you say? What's the difference? Well the main thread is where all the critical stuff happens in browsers - all your JavaScript, and also lots of browser processing to layout the page and stuff. Doing it in one place makes lots of nice things on the web work a lot simpler than if there was more parallelisation. But the downside of it is that hogging the main thread for any intensive stuff is very frowned upon and leads to serious performance issues. So if you've an expensive calculation (say decoding an image!) then ideally you don't want to do it on the main thread and want to offload it to its own thread, or if you can't do that, then chunk it up and allow other critical processes to get in on some main thread time. Browsers realised this a while back and so moved image decoding off the main thread leaving it free to run all that JavaScript you love to add to your pages, or all the other stuff.
But after you do all your lovely processing, then you will likely want to update your page. So for image decoding you'll want to display the image. If you hold up all rendering (which is what decoding=sync can do), then some might say you have effectively blocked the main thread. They're wrong as other stuff can often happen in parallel, but if the effect of that other stuff processing can't be displayed, then it can appear the main thread isn't doing what it's supposed to.
So it might be a bit of a pedantic point, but given how critical the main thread is to do other stuff, I still think it's important. Plus I'm a pedant. If decoding happened on the main thread, and then that other stuff had to happen after and only then it could be rendered it would be even slow. Anyway, I'll get back to why this often isn't as important a thing (or that it might be!), later...
However, even after all that pedantry, for images in the main screen we shall see they can also actually block the main thread, even if the decoding is happening off the main thread! It's a little complicated so we'll get back to this later.
Adding this attribute will not make images display faster (though there's some nuance here, mostly about JavaScript-inserted images). This attribute is about allowing other content to potentially display faster (including other images - so there's the nuance).
And similarly it doesn't "defer" decoding to later making images slower. Side note: browsers actually do defer decoding until the image is in, or near the viewport. Decoded images are large, so browsers won't decode all images on a page, until it needs to. Similarly they can discard decoded images to save memory if they aren't used for a while and will re-decode them if needed again.
The parser is happily processing the document on the main thread. It sees an element, so it needs to download that. That also doesn't happen on the main thread either btw, it just asks the Network chappy to deal with that, and happily processes some more content. After the network fetch is done, the main thread can get the image and see it needs decoded so passes it off to another decoder thread to deal with that. And it happily carries on and processes another thing. Once the decoding is done it's finally got the full image ready to render (which might involve another bit of main thread processing, or might not, depending on... things).
The decoding attribute lets us decide whether rendering can happen at D, or whether we should wait until E to render the image (that we have, but haven't decoded) at the same time as the other content that's ready to render.
The image will still take the same time to download and display (in general - nuance coming later!). The image will still use the same amount of the main thread's time. Other things can still process while all this is happening. It's all a matter of whether we should sync up the "process another thing" changes with the image display or not.
In the above example, probably not that much. The image is semi-independent. It's being fetched from the network (or the cache) and that time isn't guaranteed. Then it's decoded. So the relationship to the content in "process something else" and "process another thing" isn't guaranteed anyway. The content that was a result of "process another thing" could be rendered while the image downloading is happening for example if that download took a bit longer. So, given there aren't any guarantees to the ordering here, syncing these updates isn't that important to be honest.
What potentially is more important is not to block the other content from displaying if the decoding is quite lengthy. Why shouldn't we display the results of "process another thing" earlier instead of making it wait since I've just said it's independent?
So maybe we should all use decoding=async for our images and get a slight performance boost for our other content? Look at the time difference between D and E - that's noticeable!! Stop whinging Barry and just accept this is better and all the advice is right!
In fact it's even less of an issue than that, as most in-viewport images are progressively rendered - yes not just progressive JPEGs. Non-progressive images are rendered progressively top to bottom, rather than blurry to sharp. So in reality the decode time will be even smaller for these chunks of images. There also often may not even be any other content to display between each of these smaller chunks depending what else is happening.
However, for reasons we'll come back to later, that really only affects on-screen images loading at the same time as the page. Images that are already loaded (for example, being fetched from the cache), or off-screen images are not decoded immediately, meaning the progressive nature of them loading isn't as relevant. We'll return to this at the end...
c80f0f1006