How to use vertex buffer for spritesheet

[Guillaume Ludwig]

Hi everyone,

I'm trying to make a simple 2d game with pi3d on the RPi. For that I want to use spritesheets, with coordinates written in an XML file.
I started by making the simplest code a could using Buffer() in order to understand how it could works.

(I'm almost totally new to OpenGL, and I've just discovered pi3d.)

Here is my code :

#!/usr/bin/python
import pi3d

DISPLAY = pi3d.Display.create(background=(1.0, 1.0, 1.0, 1.0),
                              w=800, h=480,
                                frames_per_second=30, tk=False)

shader = pi3d.Shader("uv_flat")
CAMERA = pi3d.Camera(is_3d=False)
mykeys = pi3d.Keyboard()

img = pi3d.Texture("textures/zero.png", mipmap=False, i_format=pi3d.GL_RGBA, filter=pi3d.GL_NEAREST)

ww = 32
hh = 32

verts = ((-ww, hh, 0.0), (ww, hh, 0.0), (ww, -hh, 0.0), (-ww, -hh, 0.0),
         (-ww, hh+64, 0.0), (ww, hh+64, 0.0), (ww, -hh+64, 0.0), (-ww, -hh+64, 0.0))
norms = ((0, 0, -1), (0, 0, -1),  (0, 0, -1), (0, 0, -1),
         (0, 0, -1), (0, 0, -1),  (0, 0, -1), (0, 0, -1))
texcoords = ((0.0, 0.0), (0.5, 0.0), (0.5, 1.0), (0.0 , 1.0),
             (0.0, 0.0), (0.5, 0.0), (0.5, 1.0), (0.0 , 1.0))

inds = ((3, 0, 1), (1, 2, 3),
        (3, 0, 1), (1, 2, 3))

sprites = pi3d.Shape(CAMERA, None, "points", 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
                                1.0, 1.0, 1.0, 0.0, 0.0, 0.0)
sprites.buf = [pi3d.Buffer(sprites, verts, texcoords, inds, norms)]
sprites.set_draw_details(shader, [img])

while DISPLAY.loop_running():
   sprites.draw()
   if mykeys.read() == 27:
      mykeys.close()
      DISPLAY.destroy()
      break

The shader used is the one shipped with pi3d, with no modification.
I have a texture (32x64). I was expecting that code to display two images, but the second one seems to be ignored.

Could someone explain what I'm doing wrong ?

Regards,
Guillaume

[Paddy]

@Guillaume, The system you probably want to use eventually is the one for the SpriteMulti demo. However you are definitely right to work out how the vertex, normal, texture coordinates and element arrays (triangles) fit together and what the shaders do with the information that is passed to them.

In your example you don't seem to be using the 'top' four vertices. i.e. your triangles re-use the same four vertices, you also uv map your textures strangely. The direction of uv for texture sampling and screen pixel coordinates has an opposite y direction from the 3D space coordinate system used by OpenGL which means you often need to flip things upside down. With things like this the only possible way to sort things out is to actually draw the locations of the vertices on a sheet of paper, along with the triangles and uv coordinates.

texcoords = ((0.0, 0.5), (1.0, 0.5), (01.0, 1.0), (0.0, 1.0),

            (0.0, 0.0), (1.0, 0.0), (1.0, 0.5), (0.0 , 0.5))

#texcoords = ((0.0, 0.0), (1.0, 0.0), (1.0, 0.5), (0.0 , 0.5),

#            (0.0, 0.5), (1.0, 0.5), (01.0, 1.0), (0.0, 1.0))

#texcoords = ((1.0, 0.0), (1.0, 0.5), (0.0 , 0.5),(0.0, 0.0), 

#            (0.0, 1.0), (0.0, 0.5), (1.0, 0.5), (01.0, 1.0))

#mess around with different orientations and relfections.

inds = ((3, 0, 1), (1, 2, 3),

        (7, 4, 5), (5, 6, 7)) # you need to define the 2nd and 3rd triangles to use the top 8 verts

[Guillaume Ludwig]

Thanks for your reply Paddy. I found the exact same solution yesterday evening, glad to see I was in the right direction :)

For the coordinates, it was just a quick test. After fixing the indices all went right.

Here is the result (coordinates were changed), at 60fps and with a movement : https://www.youtube.com/watch?v=9n3bO2cL-kQ
Perfectly smooth !

I want to go with vertex array because I think it's the best way for my aim. I want to make an easy to use small engine for 2d games usable on the Pi Zero, and thus make the Pi Zero the cheapest (humble) game developing platform. I plan to use two softwares : Tiled for the map making, it can export maps in xml, with different layers. And Aseprite : for making graphics and/or only used it as a spritesheet maker. Aseprite can (and even in command line) take all the images in a folder, make a spritesheet and an xml file describing it. All the tools needed to easily make fast python games on Pi…

BTW, I also made a custom pi zero based portable console, it would be magical to see my daughter playing a game I made on this console :)

Regards,
Guillaume

[Paddy]

@Guillaume, There could well be some advantages to your system, I will be interested to see how it gets along. With only a few sprites it's probably not too bad to remake the Buffer each frame, with different vertex locations as the sprites move. However with lots of sprites (missiles, space invaders or fragments of an explosion, say) the computation might get slower. You can speed it up by using the Buffer.re_init() method (so long as you are only moving sprites and not adding new ones) also the calculation process will be much faster if you do everything with numpy

[Guillaume Ludwig]

I'll post my progress as soon as I have something nice to show.

One year ago, I tried to learn a little bit of OpenGL (mostly fixed pipeline + vertex array) and game making by doing this :

https://www.youtube.com/channel/UCsrwwwt2bSvc-t-XIabqcPQ/videos
(an action platfomer with a custom tiny engine, totally from scratch, but never released/finished - buuut yet fun to play during hours)

It was really slow at the beginning, but at the end a got on a low-end Celeron with a Intel HD2000 90fps with 2 players, tons of bullets, 150 quite smart enemies, 6000 objects (collision + visual), on 3-9 layers with lighting effect. Only had to use a little bit of cython.

Effectively almost everything was done with numpy.
I think a game like xeodrifter (http://store.steampowered.com/app/319140/) can be done on Pi Zero (but perhaps not by me… :p).

[Paddy]

Excellent. I can see that it makes sense to reuse as much of the work you've already done. Which is quite a bit!

I've been tempted to use cython a couple of times wth pi3d but it would make the whole installation and portability significantly more complicated so I've avoided it. In pi3d, especially when running on the Raspberry Pi, the thing that slows the frame rate significantly is calling the draw() method for lots of Shape and Buffer objects. So having all the bullets you might need included as different lines in the vertex array (but not drawn by moving off screen or flipping the face when not needed) will be quicker than having one bullet and drawing it lots of times.

I think that having a 2D game engine for pi3d that people could hack to whatever skill level they had would be great, I would be happy to help where I can.

[Guillaume Ludwig]

The only _big_ difference with my previous work, is that a was using a custom in-game editor (and a random maze generator). Now I want to use maps from Tiled. I saw a pyTMX lib, maybe it could be a good start.
For the spritesheet, Aseprite generates rather simple json, it will be easy I guess.

The only thing that "scares" me is : will I be competent enough to make lighting (torch, …) and manage transparency (the biggest problem I had). Maybe I could ask you if I'm stalled ?


My next steps :
- use pygame for controls (maybe you have a good example for it in pi3d ?)
- make the spritesheet importer
- make an animation from a spritesheet

[Paddy]

In the longer term it might be worth making a custom shader but your torch effect can be done pretty well using uv_light and specifying a Light with argument is_point=True. At the moment pi3d shaders only use one light but it would only require a few extra lines in the shader to use more. Transparency can sometimes be messy but with both that and lighting (and any other issues) I'm happy to help.

Json would be easier to use than xml (imho). pi3d uses pygame for keyboard and mouse input on other platforms but I've not used it myself, there was a game thing (search on the raspberry pi forum) with music and fruit that used pygame and it all seemed to work (but I wasn't any good at the game). However if you want to eventually control using GPIO maybe you don't need to get too wrapped up in pygame.

[Guillaume Ludwig]

uv_light seems to do the job. But I'm sure to be stuck somewhere sometimes, it was so hard the last time… :)

Yes json is a little bit simpler, but pytmx seems to be rather good (nevertheless requires another dependency…). Here is my first step :

This was produced by this command line : aseprite *.png --sheet sheet.png --data sheet.json --batch
And this Python source :

#!/usr/bin/python
import pi3d
import raspigame
import pygame
from pygame import *

DISPLAY = pi3d.Display.create(background=(1.0, 1.0, 1.0, 1.0),
                              w=800, h=480,

                                frames_per_second=60, tk=False, use_pygame=True)

shader = pi3d.Shader("uv_flat")
CAMERA = pi3d.Camera(is_3d=False)

img = pi3d.Texture("textures/sheet.png", mipmap=False, i_format=pi3d.GL_RGBA, filter=pi3d.GL_NEAREST)

sheet = raspigame.Spritesheet('textures/sheet.json')
sbuffer = raspigame.SpriteBuffer()

for x,sprite in enumerate(sheet.sprites):
  offset = 32 * x
  sbuffer.add_sprite(-16+offset,16, sheet.sprites[sprite])

sprites = pi3d.Shape(CAMERA, None, "points", 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
                                1.0, 1.0, 1.0, 0.0, 0.0, 0.0)

sprites.buf = [pi3d.Buffer(sprites, sbuffer.verts, sbuffer.texcoords, sbuffer.inds, sbuffer.norms)]
sprites.set_draw_details(shader, [img])

x = 0
y = 0
move_x = 0
move_y = 0
move = 4
LEFT = RIGHT = UP = DOWN = False
while DISPLAY.loop_running():
  for e in pygame.event.get():
    if e.type == QUIT: raise SystemExit, "QUIT"
    if e.type == KEYDOWN and e.key == K_ESCAPE:
      DISPLAY.destroy()
      raise SystemExit, "QUIT"

    if e.type == KEYDOWN and e.key == K_LEFT:
      LEFT = True
    if e.type == KEYDOWN and e.key == K_RIGHT:
      RIGHT = True
    if e.type == KEYDOWN and e.key == K_UP:
      UP = True
    if e.type == KEYDOWN and e.key == K_DOWN:
      DOWN = True

    if e.type == KEYUP and e.key == K_LEFT:
      LEFT = False
    if e.type == KEYUP and e.key == K_RIGHT:
      RIGHT = False
    if e.type == KEYUP and e.key == K_UP:
      UP = False
    if e.type == KEYUP and e.key == K_DOWN:
      DOWN = False
     

  if LEFT and not RIGHT: move_x = -move
  if RIGHT and not LEFT: move_x = move
  if not LEFT and not RIGHT: move_x = 0

  if UP and not DOWN: move_y = 4
  if DOWN and not UP: move_y = -4
  if not UP and not DOWN: move_y = 0

  x += move_x
  y += move_y
  sprites.position(x, y, 0.0)
  sprites.draw()

I will upload my code (raspigame.py) as soon as I have the map import function.

As you can see I use directly pygame for events. It works as expected, I thought you have to use pygame surface/display but not. This code allow use to move the sprites (all at once) in every directions, x and y at the same time.

Next : AnimatedSprite() and the map import function.

[Paddy]

@Guillaume, good progress. When running on the RPi the use_pygame argument is ignored. It's to tell the DisplayOpenGL routines to try and use a drawing surface from pygame (which uses SDL) rather than the X11 one. This doesn't work on the Pi, it has to use the standard bcm_host. (but it doesn't do any harm to have it in I suppose)

To move the sprites relative to each other rather than en bloc you could do with a function that does something along the lines of

verts[offset:offset+4,0:2] = verts_initial_xy[offset:offset+4,:] + [x, y]
#setting a flag in sprites True so that in sprites
..
def draw(self):
  if self.redraw_flag:
    self.buf[0].re_init()
  self.draw()

Probably everything should be inside a class that inherits from Shape. Buffer keeps the verts, normals and uv_coords in a numpy array array_buffer and the faces in element_array_buffer

see https://github.com/tipam/pi3d/blob/master/pi3d/Buffer.py#L86

so the above lines, when put inside the new Shape descendent would become

self.buf[0].array_buffer[offset:offset+4,0:2] = verts_initial_xy[offset:offset+4,:] + [x, y]

and, of course, you would use this function to do the initial positioning as well as animation.

[Guillaume Ludwig]

Oops, didn't know that. So pygame is not usable on the Pi for pi3d ?

I've tested the pi3d.Keyboard() but it wasn't really good (seems not to respond correctly). Do you have any tips regarding this class ?

[Guillaume Ludwig]

I precise my question : how to read more than 1 key at a time with Keyboard ? (without pygame)

[Paddy]

Sorry for the confusion. Yes pygame works fine on the RPi and is a good way to do things like key input, playing sounds etc (i.e. game stuff) But putting the use_pygame=True in Display.create() doesn't do any more than import pygame does. It's always in a form such as this

    if PLATFORM == PLATFORM_ANDROID:
      self.surface = openegl.eglGetCurrentSu
...
    elif PLATFORM == PLATFORM_PI:
      self.dispman_display = bcm.vc_dispmanx_display_open(0) #LCD setting
...
    elif pi3d.USE_PYGAME: # does't get here on the Raspberry Pi
      import pygame
      flags = pygame.OPENGL

So really you might as well include in your code

import pygame
pygame.init()

and take it out of the Display.create()

[Guillaume Ludwig]

Ok, thanks for the precision. I was reading the code to understand what could go wrong with pygame and pi3d on Pi. Such a relief :)

I will test it on my Pi this evening. I have animations working now, but I think I've taken a wrong way… I was coding like in my previous project, but the way to update buffer here is different. Need to re-think.

Current state : https://youtu.be/TXX1GjY4uQg
Can import animation from spritesheet, and set a frame speed.

[Paddy]

That looks to be coming on well. I'm not sure what your system is but if you look at the code for pi3d.Buffer you will see that there is quite a bit of code run in __init__() as well as the fact that the first time you draw() a Buffer it calls _load_opengl() which has the glBufferData() calls. Which takes a bit of processing both by the CPU and GPU. However if the size of the array_buffer hasn't changed you can just call Buffer.re_init() (without any arguments if you modify array_buffer in place, which again saves a numpy copying operation) which simply runs glBufferSubData() and is much quicker. This is what I meant when I suggested having a sprite list with all the sprites you might need, even though you don't show them every frame (you can set the x,y locations off screen or use a negative width etc)

PS Now I look at the code I see that I always run the function on the whole buffer whereas I allow re_init() to be 'fed' part of the array with an offset, and that info can be passed to glBufferSubData() I suppose that would only be efficient if just one sprite was moving. 

[Guillaume Ludwig]

I'm finally not sure to have understood the unconfusing confusion :)

On my Pi, I do :

import pygame
pygame.init()

But the event system returns nothing. It's because it doesn't use a pygame surface IMHO.
Did you (or someone else) successfully used pygame.event on a Pi with pi3d ?

[Paddy]

Hi, I (eventually) found the project that had been done using pi3d and pygame. It's all a bit hacked but I did get it to work on the Raspberry Pi. Unfortunately I can't remember the exact details...

This is the main app (I think)

https://github.com/danielpontello/beatfever-legacy/blob/master/Launcher.py

or else this (much duplicated code!)

https://github.com/danielpontello/beatfever-legacy/blob/master/OsuPi.py

As you point out they seem to have to make a 'FakeScreen', which is behind the default pi3d render layer.

Here's a link to some discussion/feedback I gave on the project

https://www.raspberrypi.org/forums/viewtopic.php?f=78&t=133242&p=890103

[Guillaume Ludwig]

I found a solution.

But I see on my android phone that you replied at the same time :) I will check the links after.

Here is my code :

import os
os.environ['SDL_VIDEODRIVER'] = 'dummy'
import pygame
from pygame.locals import *
pygame.init()
pygame.display.set_mode((1,1))

With that, the event works perfectly on the Pi. Woo ! :)
(and without a second window next to the main one…)

[Guillaume Ludwig]

Slowly, fast opengl python game making for the Pi becomes easier :)

Thank you so much for pi3d Paddy !

I hope to have time this week-end to get things done correctly…

[Paddy]

Really looking forward to seeing it. What app is running in the screen shot; is that your engine?

[Guillaume Ludwig]

No it's Tiled, THE opensource map editor. My goal is to re-use the most useful software for each part of the job.

Strangely, moving sprites is really smooth on the Pi but really laggy on a i7 under Linux.

Le jeu. 9 juin 2016 18:16, Paddy <pat...@eldwick.org.uk> a écrit :

Really looking forward to seeing it. What app is running in the screen shot; is that your engine?

--
You received this message because you are subscribed to a topic in the Google Groups "pi3d" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pi3d/zHdPO_VYj7k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pi3d+uns...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paddy]

I've also noticed that from time to time. 20fps on the Pi sometimes seems much smoother than 60fs on the laptop. I'm not sure exactly why: the fps in pi3d is controlled (assuming the frames_per_second argument has been used) by a time.sleep() in the Display.is_running() method at the start of each loop so it would be odd if the operating system was occasionally diverting effort away to the extent it 'overslept' on the laptop but not on the raspberry pi. I might do a bit more research.

PS what does it look like on the i7 processor if you drop to 30fps or so?

[Guillaume Ludwig]

Yes, I've read your code, and even the time.time() and time.sleep() code of Python. On my i7 desktop (linux/kde5), I found that what I'm doing has an influence on the lag in pi3d. That can be the case (after reading the code) but it seems a bit too much. Maybe it's only the 'state' of my linux distribution (drivers, etc). I'll test on my i5 linux laptop to see if the problem is the same.

On 30fps (i7) the lag is present also.

Strangely, if a have a lot of apps started I even have « leaps ». Not a lag, the opposite, the movement of my sprite is more important than it should. I can't explain that (an ultra faulty time.sleep() ? Which causes a raise of fps for a short amount of time…).

[Guillaume Ludwig]

Findings (on i7 linux) : If I specify the w and h arguments to pi3d.Display.create() the framerate is quite ignored. Using 60, I get 50 most of the time and jumping from 70 to 110. If I don't specify w and h, the framerate is correct. But it still lags a bit.

[Paddy]

That might be something to look into. I might see what happens if python has a lower nice value. I had noticed previously that the frame rate seemed smoother if I kept moving the mouse around over the pi3d x11 window. It might also be something to do with the x11 server (or SDL, I've not checked if I get the same thing if I specify use_pygame?) 

[Paddy]

Guillaume, how's your project, have you overcome all problems?

[Guillaume Ludwig]

Hi Paddy,

I've made little progress, as I was really busy at work and home lately (and my daughter refuses to make large naps :p).
I'm trying to put together some « object » management (multiple collision detection systems, moving, interaction) and graphics, to have a complete « sprite » system as intended in a game.

And I was also wondering how to code some isometric game… (I'm starting with a ortho 2d game, but I don't want to be stalled if I want another choice after, aiming at a _generic_ 2d game engine).

Regarding the laggy effect on the x86 Linux, no real progress unfortunately…

[Paddy]

Hi, I used to know about that (18 and 20y ago) but I think we have genes to ensure that all traces are removed from our memory!

I've started to look at using numpy for collision detection in the ElevationMap see https://github.com/tipam/pi3d/blob/develop/pi3d/shape/ElevationMap.py#L232 Not sure if it helps but this might be a basis for general mesh collision system.

Also not got anywhere with the FPS, I've not been able to reproduce the thing you found when setting w,d - maybe it's somehow related to the GPU or mesa libraries