Fast way to create QIcon in PySide2?

[Michael Boon]

Hey all,

I've just discovered that QIcon is really slow to create in Maya 2017. I'm creating what is essentially a file browser, and my old version had a QStandardItemModel, a QListView with setViewModel(QtWidgets.QListView.IconMode) and about 50,000 icons. Creating the icons used to take ~1 second and now takes ~5 minutes.

Has anyone else found this? Does anyone know a way around it? Or has anyone else found that they can create QIcons fast?

Thanks!

Boon

[Marcus Ottosson]

How do you mean creating the icons? Could the icons not be created as they are shown, or are you showing 50,000 items all at once? Also what is the content of your icons, are they .png files, fetched from the web etc.?

Overall though, it sounds like something that qualifies as a Qt 4 vs 5 question, alternatively PySide vs PySide2. You might be able to get help on Stackoverflow or the like.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/185bdb63-7a77-4201-83e8-3f361b1cca4b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Justin Israel]

On Mon, May 22, 2017, 8:10 PM Michael Boon <boon...@gmail.com> wrote:

Hey all,

I've just discovered that QIcon is really slow to create in Maya 2017. I'm creating what is essentially a file browser, and my old version had a QStandardItemModel, a QListView with setViewModel(QtWidgets.QListView.IconMode) and about 50,000 icons. Creating the icons used to take ~1 second and now takes ~5 minutes.

Are they actually 50k discreet source images, or is it 50k items in your model which make use of a smaller collection of icons? 

Do you make any use of reusing the same QIcon for the same source file or is a new instance created per item for the same file each time? If you know all your icons up front, you can create the set of them up front. Or you can use a cache to create them once as needed and retrieved from the cache on subsequent needs. 

If nothing has changed in your own code between Qt4 and Qt5, maybe there is a difference in how they implicitly cache images for you. 

Has anyone else found this? Does anyone know a way around it? Or has anyone else found that they can create QIcons fast?

Thanks!

Boon

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

[Michael and Amanda]

Thanks for the replies. I should have been more clear. What I'm doing that takes the time is instantiating 50k QIcons using file paths to 50k jpegs, each 512x512 pixels. Creating items, telling the icons what size I want them to display, adding them to my QStandardItemModel and displaying them in my view is still fast. I run the same code in Maya 2016 and 2017.

I'm at home now but I'll try to be more specific tomorrow.

On 22 May 2017 9:42 PM, "Justin Israel" <justin...@gmail.com> wrote:

On Mon, May 22, 2017, 8:10 PM Michael Boon <boon...@gmail.com> wrote:

Hey all,

I've just discovered that QIcon is really slow to create in Maya 2017. I'm creating what is essentially a file browser, and my old version had a QStandardItemModel, a QListView with setViewModel(QtWidgets.QListView.IconMode) and about 50,000 icons. Creating the icons used to take ~1 second and now takes ~5 minutes.

Are they actually 50k discreet source images, or is it 50k items in your model which make use of a smaller collection of icons? 

Do you make any use of reusing the same QIcon for the same source file or is a new instance created per item for the same file each time? If you know all your icons up front, you can create the set of them up front. Or you can use a cache to create them once as needed and retrieved from the cache on subsequent needs. 

If nothing has changed in your own code between Qt4 and Qt5, maybe there is a difference in how they implicitly cache images for you. 

Has anyone else found this? Does anyone know a way around it? Or has anyone else found that they can create QIcons fast?

Thanks!

Boon

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/185bdb63-7a77-4201-83e8-3f361b1cca4b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python_inside_maya/4J47CxyM11A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python_inside_maya+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAPGFgA27psr3di%3Dv2JksA6pdhapKUJNqnTERESs8vMiU4LG2_g%40mail.gmail.com.

[Marcus Ottosson]

Loading 50k jpegs at 512x512 at ~1 second?

It sounds like Qt 4 was deferring the actual load of these files until you viewed them, whereas Qt 5 is loading them into memory. Not impossible, could be an optimisation on their part.

But why on earth would you do this? :O Sounds incredibly inefficient. Why not have icons instantiated when you view them, through the .data() member of your model, and optionally cache them yourself where necessary?

512 x 512 x 8 bits x 4 color channels x 50,000 images comes to 419 gb of memory utilisation, unless some of your images are the same.

What’s wrong with this picture?

​

[Justin Israel]

That would be gigaBITS ;-) . Also jpegs are 3 channel usually. So it would be more like:

  512x512x(8x3)/8 =  ~39GB of uncompressed image data. But it would probably end up mostly going into swap memory. If you don't run out of virtual memory and you aren't swapping too hard then your program could function. It would then depend on how much of that data Qt wants to load active at once. 

That suggestion you made of implementing data() on the model, loading on demand, and caching a realistic max number at a time, seems like it would work great to solve this problem. 

​

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODrO8hMjrAvYQPP1BT7eXRm6RVXCVtZZrUDm4DYn0cXBQ%40mail.gmail.com.

[Michael Boon]

Thanks. That sounds like good advice. I'll try it and get back to you. It bears out so far though - trying it in Maya 2017 I see a huge amount of swapping going on.

To answer your question Marcus, I guess I was doing it this way because it used to work! The documentation for QIcon says "The file will be loaded on demand," and although it's not entirely clear what that means, in PySide2 it's definitely loading it before the icon has even been added to a visible widget.

On Tuesday, 23 May 2017 05:41:01 UTC+10, Justin Israel wrote:

On Tue, May 23, 2017, 3:42 AM Marcus Ottosson <konstr...@gmail.com> wrote:

Loading 50k jpegs at 512x512 at ~1 second?

It sounds like Qt 4 was deferring the actual load of these files until you viewed them, whereas Qt 5 is loading them into memory. Not impossible, could be an optimisation on their part.

But why on earth would you do this? :O Sounds incredibly inefficient. Why not have icons instantiated when you view them, through the .data() member of your model, and optionally cache them yourself where necessary?

512 x 512 x 8 bits x 4 color channels x 50,000 images comes to 419 gb of memory utilisation, unless some of your images are the same.

What’s wrong with this picture?

That would be gigaBITS ;-) . Also jpegs are 3 channel usually. So it would be more like:

  512x512x(8x3)/8 =  ~39GB of uncompressed image data. But it would probably end up mostly going into swap memory. If you don't run out of virtual memory and you aren't swapping too hard then your program could function. It would then depend on how much of that data Qt wants to load active at once. 

That suggestion you made of implementing data() on the model, loading on demand, and caching a realistic max number at a time, seems like it would work great to solve this problem. 

​

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

[Justin Israel]

Related? 

https://bugreports.qt.io/browse/QTBUG-59621

Justin

On Tue, May 23, 2017 at 9:43 AM Michael Boon <boon...@gmail.com> wrote:

Thanks. That sounds like good advice. I'll try it and get back to you. It bears out so far though - trying it in Maya 2017 I see a huge amount of swapping going on.

To answer your question Marcus, I guess I was doing it this way because it used to work! The documentation for QIcon says "The file will be loaded on demand," and although it's not entirely clear what that means, in PySide2 it's definitely loading it before the icon has even been added to a visible widget.

On Tuesday, 23 May 2017 05:41:01 UTC+10, Justin Israel wrote:

On Tue, May 23, 2017, 3:42 AM Marcus Ottosson <konstr...@gmail.com> wrote:

Loading 50k jpegs at 512x512 at ~1 second?

It sounds like Qt 4 was deferring the actual load of these files until you viewed them, whereas Qt 5 is loading them into memory. Not impossible, could be an optimisation on their part.

But why on earth would you do this? :O Sounds incredibly inefficient. Why not have icons instantiated when you view them, through the .data() member of your model, and optionally cache them yourself where necessary?

512 x 512 x 8 bits x 4 color channels x 50,000 images comes to 419 gb of memory utilisation, unless some of your images are the same.

What’s wrong with this picture?

That would be gigaBITS ;-) . Also jpegs are 3 channel usually. So it would be more like:

  512x512x(8x3)/8 =  ~39GB of uncompressed image data. But it would probably end up mostly going into swap memory. If you don't run out of virtual memory and you aren't swapping too hard then your program could function. It would then depend on how much of that data Qt wants to load active at once. 

That suggestion you made of implementing data() on the model, loading on demand, and caching a realistic max number at a time, seems like it would work great to solve this problem. 

​

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODrO8hMjrAvYQPP1BT7eXRm6RVXCVtZZrUDm4DYn0cXBQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/eb4e85bd-fe2c-41b7-b6da-96939f156156%40googlegroups.com.

[Michael Boon]

Yeah I had decided it was probably the same thing. The comments on the bug indicate that the person trying to fix it has misunderstood though - I think they are trying to cache image files, rather than defer loading them.

On Tuesday, 23 May 2017 08:02:36 UTC+10, Justin Israel wrote:

Related? 

https://bugreports.qt.io/browse/QTBUG-59621

Justin

On Tue, May 23, 2017 at 9:43 AM Michael Boon <boon...@gmail.com> wrote:

Thanks. That sounds like good advice. I'll try it and get back to you. It bears out so far though - trying it in Maya 2017 I see a huge amount of swapping going on.

To answer your question Marcus, I guess I was doing it this way because it used to work! The documentation for QIcon says "The file will be loaded on demand," and although it's not entirely clear what that means, in PySide2 it's definitely loading it before the icon has even been added to a visible widget.

On Tuesday, 23 May 2017 05:41:01 UTC+10, Justin Israel wrote:

On Tue, May 23, 2017, 3:42 AM Marcus Ottosson <konstr...@gmail.com> wrote:

Loading 50k jpegs at 512x512 at ~1 second?

It sounds like Qt 4 was deferring the actual load of these files until you viewed them, whereas Qt 5 is loading them into memory. Not impossible, could be an optimisation on their part.

But why on earth would you do this? :O Sounds incredibly inefficient. Why not have icons instantiated when you view them, through the .data() member of your model, and optionally cache them yourself where necessary?

512 x 512 x 8 bits x 4 color channels x 50,000 images comes to 419 gb of memory utilisation, unless some of your images are the same.

What’s wrong with this picture?

That would be gigaBITS ;-) . Also jpegs are 3 channel usually. So it would be more like:

  512x512x(8x3)/8 =  ~39GB of uncompressed image data. But it would probably end up mostly going into swap memory. If you don't run out of virtual memory and you aren't swapping too hard then your program could function. It would then depend on how much of that data Qt wants to load active at once. 

That suggestion you made of implementing data() on the model, loading on demand, and caching a realistic max number at a time, seems like it would work great to solve this problem. 

​

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFRtmODrO8hMjrAvYQPP1BT7eXRm6RVXCVtZZrUDm4DYn0cXBQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsub...@googlegroups.com.

[Marcus Ottosson]

That would be gigaBITS

Realised this shortly after posting. :)

To answer your question Marcus, I guess I was doing it this way because it used to work!

Hah, fair enough!

​

[Michael Boon]

I created a simple class for my model. Overrode __init__() to create an empty list for my icon cache. Overrode data() to cache and return an icon if role==QtCore.Qt.DecorationRole. Works like a charm. Thanks for the help!

(I also tried creating all the icons in the model items, and setting their images on demand instead of maintaining my own icon cache list, but that was much slower.)