blit_into overwrites previous image

[Jonathon Parker]

A minimal example

# rawdata1 and rawdata2 are a list of c_ubytes

my_texture_region = pyglet.image.ImageData(x, y, 'RGB', rawdata1).get_texture()

my_image = pyglet.image.ImageData(x, y, 'RGB', rawdata2)

my_texture_region.blit_into(my_image, 0, 0, 0)

When I create a sprite using my_texture region, I only see my_image.  What I want is my_image on top of my_texture_region.

What am I doing wrong?

[Charles M]

This is normal behavior, blit overwrites the pixel data with new data you are providing. Which in this case is an entirely new image.

[Jonathon Parker]

In my test the images are the same size, but I thought the background was transparent.  Any way to specify a background color is transparent?  Or is there a better way to layer images?

[Charles M]

Your images have no transparency (RGB). You need will need to specify an alpha channel to do transparency: RGBA and a fourth value for each pixel.

[Jonathon Parker]

I tried that and it did not work.  I have verified that if I draw a smaller image on top of the new, I can see that.  But even with using RGBA, I only see the last image if the sizes are the same.  Here is a more expanded example of what I am trying to do.

---

import pyglet
import pyglet.gl
import ctypes
import itertools
import numpy as np
import cv2

pyglet.gl.glEnable(pyglet.gl.GL_BLEND)
pyglet.gl.glBlendFunc(pyglet.gl.GL_SRC_ALPHA,pyglet.gl.GL_ONE_MINUS_SRC_ALPHA)

image = np.zeros((50, 50, 4), np.uint8) # type(image) = numpy.ndarray

pt1 = (12, 1)
pt2 = (48, 24)
pt3 = (12, 48)
pt4 = (1, 24)
pt5 = (1, 26)
pt6 = (48, 26)
pt7 = (48, 24)

triangle_cnt = np.array([pt1, pt2, pt3])
cv2.drawContours(image, [triangle_cnt], 0, (100, 255, 100, 255), -1)
image = image.tolist()
px = list(itertools.chain(*list(itertools.chain(*image))))

rawData = (ctypes.c_ubyte * len(px))(*px)
player_image = pyglet.image.ImageData(50, 50, 'RGBA', rawData).get_texture()

image2 = np.zeros((50, 50, 4), np.uint8)
rect_cnt = np.array([pt4, pt5, pt6, pt7])
cv2.drawContours(image2, [rect_cnt], 0, (100, 255, 100, 255), -1)
image2 = image2.tolist()
px2 = list(itertools.chain(*list(itertools.chain(*image2))))

rawData2 = (ctypes.c_ubyte * len(px2))(*px2)
rect = pyglet.image.ImageData(50, 50, 'RGBA', rawData2)

player_image.blit_into(rect, 0, 0, 0)

---

When I display player_image, I only see the thin rectangle.  What am I doing wrong?

My goal is to change images for sprites during a simulation depending on simulation events.  If you know a better way to accomplish this, I would like to hear it.

[Charles M]

Ah I see you want to blend them together in the same data. In your examples you are still overwriting the image data. If you have a red channel that's at 100 bytes and you replace the data with another pixel that's 100 bytes, it's still 100. If you are looking to blend two images together you have a few options:

1) Create separate ImageData, put them into pyglet.sprite.Sprite objects and draw them where you want. By default Sprites should blend, and it should be easier to keep track of things/move them when they are separate. I'm curious why the need to combine the data instead of treating them separate but move them where you want?

2) If you actually need them combined into one image data for some reason, then you can combine the pixels together with math based on blending formulas. https://learnopengl.com/Advanced-OpenGL/Blending Here is the formula and an example.

[Jonathon Parker]

I need them as one image.  I have tried many different blend function things and if they don't throw an error they all do the same things.  Namely, only display rect.

Something has to be wrong somewhere else.  I tried

pyglet.gl.glBlendFunc(pyglet.gl.GL_ZERO,pyglet.gl.GL_ONE)

If I understand it right, it should only display the destination image (the triangle) and set the source image to have a zero factor.  But that is not what it does.  It still displays the thin rectangle.

[Charles M]

Blend modes themselves aren't going to help you with combining textures into one. That's only for things like sprites and other things in the framebuffer.

It's not OpenGL at this point, it's the data you are providing it. You need to use math to change your pixel data with the blend mode formula equation in the link I provided in the last post. 

[Jonathon Parker]

Please look at my code example.  There is no math to do.  I have two images the same size (50 x 50).  Each image has a shape with A=255.  The rest of the image A = 0.

I want to draw one image on the other.  So I convert one image into a texture and blit_into with the other image.  I should be able to see through the transparent parts of the top image and see the opaque parts of the bottom image.  But I can't.  It won't work.

[Jonathon Parker]

So there is no solution for this?  All I want to do is:

# imgA, imgB are identically sized instances of pyglet.image.ImageData in RGBA format

imgC = imgA + imgB

# imgC is now imgA with imgB superimposed on top of it. Transparent pixels in imgB allow opaque pixels in imgA to be visible

I never thought something so simple would be so hard!

[Greg Ewing]

On 20/03/20 5:08 am, Jonathon Parker wrote:
> imgC = imgA + imgB
>
> # imgC is now imgA with imgB superimposed on top of it. Transparent

> pixels in imgB allow translucent pixels in imgA to be visible

Did you turn on blending with glEnable(GL_BLEND)?

--
Greg

[Jonathon Parker]

Yes.  If you look into the thread I have a complete "working" example https://groups.google.com/d/msg/pyglet-users/h0uVWBjkeBU/67zbZwJcDQAJ

I turn on blend and tried more functions that the one listed and nothing worked.  Do you see anything in the code I missed?

[Adam Bark]

Charles is right; blit_into is just copying pixel values. If you want blending you either have to figure it out yourself or let OpenGL do it for you.

--
You received this message because you are subscribed to the Google Groups "pyglet-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyglet-users/bac6d6cb-c2fa-4f93-860d-a25534465655%40googlegroups.com.

[Charles M]

Hey Jonathon, I tried to explain it to you earlier (math), but I think you still have a fundamental misunderstanding of how blending and blit_into works. Blending is combining pixels using the blending formula.  That formula determines the output of the resulting pixel. Normally this is done automatically in the framebuffer of drawn images, but you aren't doing; that you have two separate sets of data. With blitting you are writing your data to the texture. Writing data twice doesn't mean they combine automatically, even if they are different data sets.

Example: If you take two text files A and B, and output a file [not in append] and write file A to the same output, then write the file B to the same output. It doesn't become an intertwined mix of A and B, it just becomes B, since you are REPLACING the data. They don't just combine on their own in the data. This is the same as what you are doing, you are just replacing the data in those spots. It doesn't matter about alpha or colors; it's all data and you are replacing it. You need to COMBINE the data using the formula I linked you.

It's a slow day for me, but I made you an example using two images of the same size:

import pyglet
import ctypes


image_one = pyglet.image.load("one.png")
image_two = pyglet.image.load("two.png")


window = pyglet.window.Window()


def blend_color(color_src, color_dst, src_factor, dst_factor):
    # Blending Formula = source * src_factor + dst * dst_factor. Make sure they are int on return
    return int((color_src * src_factor + color_dst * dst_factor) * 255)
   
def blend_alpha(alpha_src, alpha_dst, src_factor, dst_factor):
    # Blend alpha, no source factor, can change if you want.
    return int((alpha_src + alpha_dst * dst_factor) * 255)


def combine_pixels(source, dst):
    """ source and dst are pixels of RGBA
    This demonstarates formula for glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)
    """
    converted_src = [px / 255 for px in source] # From bytes to values in relation to 1.
    converted_dst = [px / 255 for px in dst]


    # blend mode
    src_factor = converted_src[3]  # Source Factor: Source alpha. AKA GL_SRC_ALPHA
    dst_factor = (1 - converted_src[3])  # Dst Factor: 1 minus source alpha AKA GL_ONE_MINUS_SRC_ALPHA
   
    r = blend_color(converted_src[0], converted_dst[0], src_factor, dst_factor)
    g = blend_color(converted_src[1], converted_dst[1], src_factor, dst_factor)
    b = blend_color(converted_src[2], converted_dst[2], src_factor, dst_factor)
    a = blend_alpha(converted_src[3], converted_dst[3], src_factor, dst_factor)
         
    return r, g, b, a


   
# Turn both images in a list of RGBA int values.
pixel_data_one = list(image_one.get_image_data().get_data('RGBA', image_one.width * 4))
pixel_data_two = list(image_two.get_image_data().get_data('RGBA', image_two.width * 4))


combined_image = []
for i in range(0, len(pixel_data_one), 4):
    pixel_one = pixel_data_one[i:i + 4]
    pixel_two = pixel_data_two[i:i + 4]
   
    # Pixel_two (source) on top of pixel_one (destination)  
    combined_image.extend(combine_pixels(pixel_one, pixel_two))

data = (ctypes.c_ubyte * len(pixel_data_one))(*combined_image)
new_image = pyglet.image.ImageData(image_one.width, image_two.height, "RGBA", data)


new_image.save("combined_image.png")


# draw new image.
sprite = pyglet.sprite.Sprite(new_image, x=0, y=0)


@window.event
def on_draw():
    window.clear()
   
    sprite.draw()

pyglet.app.run()
   

[Greg Ewing]

On 21/03/20 2:37 am, Jonathon Parker wrote:
> Yes.  If you look into the thread I have a complete "working" example
> https://groups.google.com/d/msg/pyglet-users/h0uVWBjkeBU/67zbZwJcDQAJ
>
> I turn on blend and tried more functions that the one listed and nothing
> worked.  Do you see anything in the code I missed?

I just looked at the implementation of ImageData.blit_into(),
and I can see what the problem is.

It's NOT an OpenGL drawing operation. It just uses one of the
glTexSubImage* calls to overwrite part of the texture data in
vram. The blending options have no effect on that -- it's just
copying bytes.

To get blending to work here, you'll have to set the texture
up as a rendering target and then use OpenGL drawing operations
to draw into it. I don't know whether pyglet provides any
high-level support for that. There's a post here about doing it
using low-level gl calls:

https://leovt.wordpress.com/2015/10/04/render-to-texture-with-python-3-and-pyglet/

Alternatively, you could do the blending calculations yourself,
using numpy or otherwise, and then blit the result onto the texture.

--
Greg

[Jonathon Parker]

Charles,

Thank you.  After I got objects from my code in a compatible datatype it worked on the first go.

I will study your code later in an effort to learn what I did not understand.

Jonathon