Minimum Python version support for contributions

[Benjamin Moran]

Hi guys,

New user of pyglet, and potential contributor. I'm not sure if the core developers visit this forum, but I thought it was a good place to start.
Basically, I want to implement support for SDL2 GameController_API style controller mapping, and wanted to knowthe minimum Python version I should target. Python 3 is a given, but is 2.7 old enough on the 2.x branch? Does pyglet still officially support older than that?

In case anyone is not familar with the SDL2 Game Controller API, it's basically a way to have game controllers "just work" on Linux/Max/Windows. It presents an internal layout that mimics the ubiqitous Xbox360 layout. There is also an internal database of dozens of popular controllers that just work out of the box. For controllers that are not recognised yet, there is the ability to load mappings from a file (SDL_GameControllerAddMappingsFromFile), or automatically from an environmental value (SDL_GAMECONTROLLERCONFIG), as is the case when you launch a game under Steam.
SDL Wiki page:  http://wiki.libsdl.org/CategoryGameController
GameController mapping database:  https://github.com/gabomdq/SDL_GameControllerDB

This would be built on top of the existing pyglet joystick API, in much the same way as SDL2 does it.

Any feedback would be appreciated.

-Ben

[Leif Theden]

I think that 2.7 is fine.  The legacy code in pyglet (>2.6) likely exists because it was written a long time ago, and not likely because it written to be compatible with old python. Python 2.6 is nearly 8 years old now and isn't required except on odd linux (redhat, etc). 

I've considered exactly what you are proposing, using SDL2 for input, but I think that you will have a lot more work to do.  AFAIK, you will need to use all of SDL2 (windowing, events, etc).  Pyglet interfaces at the OS level (with ctypes), so you *may* have to re-implement more than just the joystick API.  Something to think about anyway.

[Benjamin Moran]

Thanks for the reply, Leif.

What I'm thinking is much simpler than that. I plan on leaving the existing joystick api in pyglet as-is, and implementing the game controller api as an abstraction over that (how it's done in SDL).

I've already hacked up a quick script that converts the pyglet joystick ids to the SDL GUIDs on Linux. I just need to code up the rest of the abstraction for the game controller buttons and sticks.

[Leif Theden]

Ah, I completely misread your post  :D.  Good luck!  That would be a great addition to pyglet.  Let me know if you want a tester... I've got lost of controllers to play with.

[Benjamin Moran]

I'd definitely appreciate testers once I get some work done. Actually, if you're on Linux, would you mind testing out the bit of code in the attached file? You'll have to open a joystick device first in pyglet, then pass it to the function. It'll should then give you an sdl2 style GUID. (I just did some crude conversion of the available id_*s in pyglet, but I imagine there might be a better way to get these values).

Also, whether or your Windows/Mac/Linux, would you mind sending me the id_bustype, id_vendor, id_product, id_version, and name from your joysticks? I've no idea if these are the same on other platforms, but I will need to figure out how they get converted in any case.

Thanks!

[Rob]

I agree. I currently only test 2.7 and 3.4. I guess that 2.6 and 3.2 are already old enough to not consider for pyglet anymore. 3.3 is also questionable.

Rob

Op woensdag 21 oktober 2015 20:27:26 UTC+2 schreef Leif Theden:

[Benjamin Moran]

Thanks for the reply, Rob. With that in mind, I wouldn't mind cleaning up and modernizing some of the existing modules a bit. I've got my eye on the sprite module for a start. I'm thinking it would be a lot more readable if @property and @<x>.setter decorators were used instead of the current property declarations. If I were to go through and make those changes, would that be something you would consider pulling into master?

-Ben

[Rob]

I am definitely in favor of that. I also use @property and such in the refactoring.

Rob

Op maandag 26 oktober 2015 13:04:46 UTC+1 schreef Benjamin Moran:

[Benjamin Moran]

Great!  I'll send a pull request your way sometime soon.

-Ben

[Leif Theden]

Looking forward to removing legacy code from pyglet!  I did this little exercise in the past...took just a couple hours to get it right.  Make sure that you get the docstrings right so it doesn't affect the auto-generated documentation.  I would offer my code as a PR, but I dropped 2.x support.

[Benjamin Moran]

I've mostly kept the docstrings as-is, so hopefully it won't cause any issues. I'll check again before submitting it. I've tried to follow convention with regards to @property/@x.setter decorator docstrings, but admittedly I'm not an expert.

Ah, it sure would be nice to drop Python 2 support, wouldn't it? I guess having dual compatibility with 2/3 is a good next step in that direction. One day! Python 2 only has a few years left of support after all.

[Leif Theden]

Ubuntu and arch Linux are shipping with Python 3 as the default and probably others as well. For a least a few years though Python 2 needs to be supported.

[Rob van der Most]

That is also the reason I use future for the compatibility. It makes python 3 code run on python 2, so you are backwards compatible instead of the other way around.

Rob

On 27 Oct 2015 10:05, "Leif Theden" <leif....@gmail.com> wrote:

Ubuntu and arch Linux are shipping with Python 3 as the default and probably others as well.  For a least a few years though Python 2 needs to be supported.

--
You received this message because you are subscribed to the Google Groups "pyglet-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users...@googlegroups.com.
To post to this group, send email to pyglet...@googlegroups.com.
Visit this group at http://groups.google.com/group/pyglet-users.
For more options, visit https://groups.google.com/d/optout.

[Benjamin Moran]

I've made a pull request for my refactored sprite module. There aren't really any code changes or docstring changes, so I don't think anything should be affected. If there are any issues I can fix it up.

[Leif Theden]

Looks pretty good to me.  Unless I'm mistaken, it looks like Sprite.position is missing the setter?  I see Sprite.set_position, but according to the old code, Sprite.set_position was also the setter for the position property.  I would resolve this by creating a Position.setter that calls Sprite.set_position.  My other comment is that your PR also changes the triple-quotes around the docstrings.  I don't mean to be pedantic, but since the rest of pyglet uses triple single-qoutes ( ''' ) instead of ( """ ), then I think that should be left unchanged.  Just my 2 cents.  Other than that, it looks correct to me.

[Rob van der Most]

Currently I also mostly replace the quotes. Afaik the double quotes are the preferred ones in python.

Rob

[Benjamin Moran]

Thanks for the feedback.

I left the set_position method there because it's not possible to pass more than one value for a @position.setter. If added, it will require x,y to be passed as a tuple. That's OK, but set_position has to stay for now anyway so it doesn't break any code.

-Ben

[Leif Theden]

> I left the set_position method there because it's not possible to pass more than one value for a @position.setter

Good point. I didn't catch that.

[Leif Theden]

I'm worried that lack of a position setter will break code that relies on it.  I've got two ideas on how to handle it:

1. Leave it as is

2. Use the following:

https://gist.github.com/bitcraft/cbb6dba134fcd6edd96c

I've test the above on 2.7 and 3.4 and it worked fine.

[Rob]

Using a property has my preference, as it makes the whole api consistent. Setting a position as a tuple is quite common. I would make the original set_position private (so rename to _set_position) and then let set_position raise a deprecation warning and call it. We can then remove it a few releases later.

Rob

Op woensdag 28 oktober 2015 18:36:49 UTC+1 schreef Leif Theden:

[Leif Theden]

I was thinking about a depreciation warning as well. 

--
You received this message because you are subscribed to a topic in the Google Groups "pyglet-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pyglet-users/dpoL2M8-b4E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pyglet-users...@googlegroups.com.

[Benjamin Moran]

Sounds good. I'll make those changes and update the pull request.

Thanks guys.

[Benjamin Moran]

Pull request updated. Let me know if I missed anything!

[Rob]

Looks good. Only one minor comment: can you add a deprecation warning to the set_position too? 

E.g.

warnings.warn("Use position property instead.", DeprecationWarning)

Rob

Op donderdag 29 oktober 2015 08:20:55 UTC+1 schreef Benjamin Moran:

[Benjamin Moran]

Done!
I've not used the warnings module before, but It looks straightforward. Let me know if I messed that up somehow and I'll fix it.

-Ben

[Leif Theden]

Merged.  Nice work  =D

[Benjamin Moran]

Thanks Leif, and Rob, for your feedback.

I've been working on my Game Controller implementation as well, but need some opinions on the best place to add it in. It's probably best to start a new thread for that.

[Rob van der Most]

And you thanks for the contribution.

For the place to add it, I am thinking we might need to upgrade the input module to a package. We can then expose the original contents of the input module through __init__.py.

Rob

On 29 Oct 2015 4:40 pm, "Benjamin Moran" <benmo...@gmail.com> wrote:

Thanks Leif, and Rob, for your feedback.

I've been working on my Game Controller implementation as well, but need some opinions on the best place to add it in. It's probably best to start a new thread for that.

--

You received this message because you are subscribed to the Google Groups "pyglet-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users...@googlegroups.com.

[Benjamin Moran]

The input module would make sense as a package.

My issue with the Game Controller implementation is more a logistical one. From a high level, "Game Controllers" are really simple. They are just joystick devices that match an internal configuration database. They then expose a consistent internal button/axis/trigger mapping. So essentially, the code is the same as a Joystick, plus the additional database check and button mapping. With that in mind, I was debating whether or not it would make sense to layer it on top of the Joystick class, or just create a parallel implementation.

Layering it on top of the Joystick class would require a few additions to that class, so maybe it's not a good idea for a first revision. I think I'll do a parallel implementation, even if it does mean duplicating some methods (for example the _create_joystick method in evdev.py).

[Leif Theden]

I would use the existing API, and layer your virtual device on top of that.  Each platform has their own way of getting input from devices, and creating a 'parallel implementation' at the device level would involve writing an implementation for each platform (win32, os x, linux).  By using the existing API, you will basically just be making a mapping layer between real events and the 'normalized' events that fit your universal controller model...if that makes sense.

https://pyglet.readthedocs.org/en/pyglet-1.2-maintenance/programming_guide/input.html#using-joysticks

Here's a couple ideas for using the existing API:

1. Create a subclass of Controls (pyglet.input.base.Control) that map a value from an existing device to your generic model.  Then aggregate the Control objects into a Joystick instance.  This is how pyglet does it (besides the event mapping).  You will get event binding with decorators and basically a drop-in replacement.  I'd recommend this approach first.

2. Create a subclass of EventDispatcher, and listen to all joystick events.  The new class will have a dict mapping with keys of raw input for a specific controller and values of your generic model.  Subscribe to all joystick events, then dispatch events that fit your model.

3. Create a generic class with a set of on_joy_press, on_joyaxis methods that will map inputs.  This class can be customized and events bound/subscribes to a device with joystick.push_handlers

Those are my ideas at the moment, I hope that gives you some direction.  Finally, I hope you decide to use the existing pyglet API, as opposed to hacking evdev or something; your work will work for all platforms without much effort.

If you are really not sure what to do, you can tackle non API things first, like sourcing a database for joystick devices and building a json/xml file that can be loaded in the module to supply the button/axis mapping.  I realize you linked one already, but that will have to be transformed into something python can use, like a dict.

[Benjamin Moran]

Thanks for the good ideas Leif.

For a first attempt, I've decided to implement it in parallel with the existing Joystick class, in the same way as the AppleRemote is done. I think doing it this way will help me understand the requirements better. After that I will go for a simpler solution (as you've suggested), and then ask for opinions on possible inclusion into the main pyglet branch. This is also a way for me to learn the codebase better, as I really like pyglet and want to be able to contribute more in the future.

I've already got an idea for handling the configuration database. It will be basically the same way SDL does things, as I feel this works well.
1.  Firstly, there will be an internal database (list of dictionaries or similar) that contains predefined support for the most popular game controllers.
2.  Secondly, there will be a method from adding additional mappings from a file. SDL has the "SDL_GameControllerAddMappingsFromFile" function, so I want to add something similar.

Regarding #2, I've worked out a crude parser for the sdl2 community database at https://github.com/gabomdq/SDL_GameControllerDB. I want to have support for this format, because it's also the same format that is produced by Steam when you configure a controller in "Big Picture Mode" (the TV interface for Steam). The work has already been done in this regard, so why not use the same format.

[Rob]

BTW, I just noticed I was confused. Input is already a package :-S. Never mind ;-)

Op donderdag 29 oktober 2015 19:57:25 UTC+1 schreef Rob:

And you thanks for the contribution.

For the place to add it, I am thinking we might need to upgrade the input module to a package. We can then expose the original contents of the input module through __init__.py.

Rob

On 29 Oct 2015 4:40 pm, "Benjamin Moran" <benmo...@gmail.com> wrote:

Thanks Leif, and Rob, for your feedback.

I've been working on my Game Controller implementation as well, but need some opinions on the best place to add it in. It's probably best to start a new thread for that.

--
You received this message because you are subscribed to the Google Groups "pyglet-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users+unsubscribe@googlegroups.com.

[Benjamin Moran]

Yeah, I didn't really look either :)

I've been working on my GameController implementation. (Just a first attempt, so it'll be changed later).
So far I've added a parallel GameController class alone-side the existing Joystick and AppleRemote classes, and a corresponding get_game_controllers method that mimics the get_joysticks method. I've also added a separate module:  "input.gamecontroller.py". This module contains a simple internal mapping database, as well as a method to add additional mappings from a file, and some methods to retrieve the mappings (such as get the mapping for button #1, or which button index is "X" on the controller).

My question is about a data structure to store the internal controller mapping database. Right now I'm just using a list of dictionaries. The dictionaries themselves contain entries like: {...'a': 'b1', 'righty': 'a5', dpright': 'h0.2',....} and so on. This is a fairly raw import of the data from the SDL format. I'm thinking there might be a better format to store this data in Python. Some of the data is simply key:value pairs, such as 'name': 'Sony DualShock 4', so a dictionary works well. However, keys really contain three things - the control name, the mapping type (button, axis or hat), and the index. Depending on the type of controller, "dpright" could map to either a hat or a button. It would be better to store these in the database in such a format. Instead of the simple "a5" or "b1" like above, I was thinking something like this would be better:
{"name": "Sony DualShock 4",  "a": ("button", 1),  "dpright": ("hat", 0, 2),...}

It starts to look a bit funny for a dictionary, but it should work. Is there any better structure that could be used for storing this type of data that would be better? Something other than a dictionary? The data is only accessed once when the game controller is instantiated.

Thanks for any comments. I'm learning a bit about the input stack as I hack on this stuff, so it's pretty fun.

-Ben

On Saturday, October 31, 2015 at 4:06:24 AM UTC+9, Rob wrote:

BTW, I just noticed I was confused. Input is already a package :-S. Never mind ;-)

Op donderdag 29 oktober 2015 19:57:25 UTC+1 schreef Rob:

And you thanks for the contribution.

For the place to add it, I am thinking we might need to upgrade the input module to a package. We can then expose the original contents of the input module through __init__.py.

Rob

On 29 Oct 2015 4:40 pm, "Benjamin Moran" <benmo...@gmail.com> wrote:

Thanks Leif, and Rob, for your feedback.

I've been working on my Game Controller implementation as well, but need some opinions on the best place to add it in. It's probably best to start a new thread for that.

--
You received this message because you are subscribed to the Google Groups "pyglet-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pyglet-users...@googlegroups.com.

[Leif Theden]

My opinion, for what it is worth, is to just use dictionaries for as long as you can and use them to build your interface classes.  Named tuples are very useful too.  Text => Dictionaries/NamedTuples => Pyglet Classes.  I wouldn't introduce any 'classes' for the sake of building classes if they are not needed to build the pyglet controller code.  At this point, this is the only part of pyglet that I am aware of that pulls in configuration from a text file, so I don't see an immediate need to engineer new classes.

Some other comments that I have...will the controller database be parsed at startup, or only when game controllers are enumerated?  will it be slow?  would it be better to make a script to transform the community list into json, or python source code?  Will it use lots of memory?  Just my thoughts.

[Benjamin Moran]

Thanks, I'll keep it as a list of dictionaries for now. It's really nothing more than this:
game_controller_db = [{...}, {...}]

I'll see if I can answer your questions:

The mapping DB is only parsed when gamepads are enumerated. First a guid is created for the device, which is checked if it's in the mapping DB. If it is, that specific mapping dict is used to map the buttons/axis correctly. It's fast - basically instantaneous because it's only doing some dict queries.

To clarify, nothing is loaded from disk by default. The internal mapping dict is pre-defined, and will contain a few dozen mappings by default. A helper function, add_mappings_from_file(), can be used to expand the internal DB from an external text file. This is an option for application developers.

Basically we could just pre-define the entire community list, but the mapping dictionary does get loaded into memory when gamepads are being created. I'll have to check just how big it is.

[Benjamin Moran]

As a follow-up, I've switched over to a dictionary with tuples. I'll also paste a sample of what the internal mapping list looks like. It's quite small in the end. I've checked with sys.getsizeof(), and it looks like two mappings are 80 bytes. After adding ALL the mappings from http://github.com/gabomdq/SDL_GameControllerDB, the list only grew to 936 bytes for a total of 93 comple game controller mappings. This means that we can keep an internal list of several dozen popular game controller mappings in under a Kilobyte. If the sys.getsizeof() function is accurate, that sounds quite reasonable to me!

mapping_list = [{'guid': '050000004c050000c405000000010000', 'name': 'PS4 Controller (Bluetooth)',
                 'b': ('button', 2), 'leftshoulder': ('button', 4), 'guide': ('button', 12),
                 'dpdown': ('hat0', 4), 'rightx': ('axis', 2), 'dpup': ('hat0', 1),
                 'dpright': ('hat0', 2), 'rightshoulder': ('button', 5), 'y': ('button', 3),
                 'dpleft': ('hat0', 8), 'righty': ('axis', 5), 'rightstick': ('button', 11),
                 'platform': 'Linux', 'x': ('button', 0), 'lefty': ('axis', 1), 'a': ('button', 1),
                 'lefttrigger': ('axis', 3), 'leftx': ('axis', 0), 'back': ('button', 8),
                 'leftstick': ('button', 10), 'start': ('button', 9), 'righttrigger': ('axis', 4)},
                 {...}, {...}, {...}]