Memory Error when calling get_texture() on image stemming from access to raw ctype buffer data

[Manuel Kaufmann]

Hi,

I am using Pyglet 1.2.4 to render a texturized 3D Mesh composed of triangles and stored as an OBJ file. The mesh is not small, but also not extremely huge: It takes up about 52 MB on the harddisk and consists of roughly 800'000 faces (~ 366'500 vertices). The texture is available as a .jpeg, takes up about 6.5 MB on the harddisk and its resolution is 8192x8192.

In order to upload the texture to the GPU, I do the following:

self._image = pyglet.image.load(image_path)

...
self._texture = self._image.get_texture()

which sometimes throws the following error:

...
    self._texture = self._image.get_texture()
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 818, in get_texture
    force_rectangle)
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 809, in create_texture
    self.anchor_x, self.anchor_y, 0, None)
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 953, in blit_to_texture
    data = self._convert(data_format, data_pitch)
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 1008, in _convert
    self._ensure_string_data()
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 1076, in _ensure_string_data
    self._current_data = buf.raw
MemoryError

I tried to find out why this happens, but to no avail. I know that the computer I am using is powerful enough to handle mesh sizes like that, both because the error only happens sometimes and because I can visualize it without problems using programs like e.g. Meshlab. Any help/hints how I can resolve or further debug this are greatly appreciated!

Thanks a lot and best regards
Manuel

[claudio canepa]

As a long shot you may try with the  patch in   https://code.google.com/archive/p/pyglet/issues/752

It probably not address directly your problem, but as it eliminates a buffer copy... 

--
You received this message because you are subscribed to the Google Groups "pyglet-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users+unsubscribe@googlegroups.com.
To post to this group, send email to pyglet...@googlegroups.com.
Visit this group at https://groups.google.com/group/pyglet-users.
For more options, visit https://groups.google.com/d/optout.

[Manuel Kaufmann]

Thanks for your answer @claudio canepa. I'm not sure if this patch would fix my problem. I tried it ad hoc and the memory error was not thrown in _ensure_string_data anymore, but later in the _convert method when calling data = asbytes('').join(rows) on line 1068.

I tried out some more stuff and called e.g. get_data() before calling get_texture() just to see what happens:

data = self._image.get_data('BGR', self._image.width*3)
self._texture = self._image.get_texture()

The call to get_data() forces a conversion, because the original pitch after loading the image is negative. This conversion runs through successfully it seems but the subsequent call to get_texture() now breaks at another point:

    self._texture = self._image.get_texture()
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 818, in get_texture
    force_rectangle)

  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 803, in create_texture
    rectangle, force_rectangle)
  File "D:\Python27\lib\site-packages\pyglet\image\__init__.py", line 1508, in create
    blank = (GLubyte * (texture_width * texture_height * 4))()
MemoryError

I.e., it throws an error when it tries to allocate a blank data buffer of the required texture size ... What is this MemoryError anyway? Does it come from Pyglet, from OpenGL, from the ctypes module?

To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users...@googlegroups.com.

[Richard Jones]

That MemoryError looks like a ctypes-level error since the line in
question is just trying to allocate a GLubyte array in ctypes. I guess
something is getting cranky at you trying to allocate a 256MB memory
chunk. Are you perhaps actually running into memory exhaustion on your
system? Assuming that texture_width and texture_height are what you're
expecting, that is.

[Manuel Kaufmann]

@Richard Jones: I'm running on Windows 10 with 8 GB RAM. Taskmanager is telling me that 3.9 GB are available, details of that showing that 3.1 GB is on standby and 0.8 GB is actually free. So, it is possible that I'm running into memory issues, especially if the allocated memory chunk must be contiguous, but it would surprise me a bit, though (?). I just restarted the computer, ran only the program in question and still got the same error. Also, if I can open three meshes of similar sizes using Meshlab must mean that somehow the needed resources are available (of course Meshalb is probably much more optimized than my code, but still ...)

[Manuel Kaufmann]

By mere chance I found out about the python -m pyglet.info command, which revealed that the program was actually using my onboard Intel Chipset, instead of the also available NVIDIA GeForce GT 650M. I changed that in the NVIDIA Settings and now the program runs through without a problem ... This is weird though, as I thought the memory problems I described happened on the host, not on the device?

[Benjamin Moran]

Hi guys,

To follow up with what Richard said, this is very likely a GPU related memory error. Unfortunately It's hard to say where the issue lies because a lot is abstracted with OpenGL drivers, and they tend to fudge a lot of things behind the scenes (especially with mobile chipsets).

Just for fun, here's an example of trying to make a texture that my GPU says it can handle, but doesn't actually have the ability to create. (Notice the two different exceptions):

>>> pyglet.image.Texture.create(16384, 16384)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ben/Documents/PycharmProjects/pyglet/pyglet/image/__init__.py", line 1519, in create
    blank)
  File "/home/ben/Documents/PycharmProjects/pyglet/pyglet/gl/lib.py", line 105, in errcheck
    raise GLException(msg)
pyglet.gl.lib.GLException: b'out of memory'
>>>
>>>
>>> pyglet.image.Texture.create(16384, 16384)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ben/Documents/PycharmProjects/pyglet/pyglet/image/__init__.py", line 1513, in create
    blank = (GLubyte * (texture_width * texture_height * 4))()
MemoryError
>>>

[Manuel Kaufmann]

Hi everyone,

Thanks a lot for all your inputs!

This was indeed a GPU related memory error, which was resolved once I realized OpenGL was configured to use the onboard Intel chipset instead of the NVIDIA card. Just for reference: the NVIDIA graphics card can be selected through NVIDIA settings > Manage 3D Settings > Program Settings, where I added Python as a new program and selected "High-performance NVIDIA processor" as the preferred graphics processor.

Hence, I consider this issue closed.

Best regards
Manuel