status of trunk vs 1.1-maintenance branch?
Bruce Smith
It seems to me that the most difficult immediate issue for the future of pyglet is what to do about the divergence between the 1.1-maintenance branch (off of which all releases have been made) vs. the trunk (sometimes called the 1.2dev branch). The incompatibility between them can confuse new users and new developers, and leads to more work and possible confusion for all maintainers. Examples of the confusion: what to check out, which branch(es) to fix bugs in, and what to do with code in the experimental subdirectory which only works with the maintenance branch.
It would be great if someone who knows could summarize:
- the goals and specific elements of the refactoring represented by the trunk;
- its status, relative to those goals;
- advice about how to handle this going forward.
I don't know if anyone but Alex understands this fully, but I know at least some other developers know much more about how these branches differ than I do.
I think the near-term highest priority in the code, besides fixing individual bugs, is getting it back to a state of having only one important branch -- that is, either abandoning the trunk (reverting to the 1.1-maintenance branch) or finishing the trunk and abandoning further development in the 1.1-maintenance branch.
- Bruce Smith
Thomas Woelz
related issues remain in this group since the _users_ of pyglet are
the ones that are needed to keep the development of this library.
Also, I think the users should know what is going on with the library.
At least I want to keep track of the changes, even though I'm not
planning on helping the development.
Thomas
Tristam MacDonald
I'd prefer that the whole thread about the future of pyglet and
related issues remain in this group since the _users_ of pyglet are
the ones that are needed to keep the development of this library.
Also, I think the users should know what is going on with the library.
At least I want to keep track of the changes, even though I'm not
planning on helping the development.
--
Tristam MacDonald
http://swiftcoder.wordpress.com/
Bruce Smith
I'd prefer that the whole thread about the future of pyglet and
related issues remain in this group since the _users_ of pyglet are
the ones that are needed to keep the development of this library....
I agree, but this point is unrelated to my post which started this thread, so maybe I should clarify: you are favoring one pyglet mailing list rather than two, whereas my original post is about getting back to one main svn branch in the code repository rather than two.
- Bruce
Alex Holkner
> It seems to me that the most difficult immediate issue for the future of
> pyglet is what to do about the divergence between the 1.1-maintenance branch
> (off of which all releases have been made) vs. the trunk (sometimes called
> the 1.2dev branch). The incompatibility between them can confuse new users
> and new developers, and leads to more work and possible confusion for all
> maintainers. Examples of the confusion: what to check out, which branch(es)
> to fix bugs in, and what to do with code in the experimental subdirectory
> which only works with the maintenance branch.
I'll do my best to provide a quick summary of where I think things are
at. Bear in mind that it's been several months since I worked on
these changes, and I haven't checked my facts.
The 1.2dev branch includes several new features and several unrelated
large refactors of existing features. The scope grew as development
went on, which is why for a while most bug fixing continued in trunk
without being backported to 1.1-maintenance (which started when I
realised how big 1.2 was getting). I have no summary listing of which
bugs have been fixed in trunk but not maintenance; refer to the SVN
logs and the tracker for specific commits. I'm fairly sure there are
no issues fixed in maintenance that have not also been fixed in trunk.
Most changes to trunk were first prototyped in the experimental/
directory (of trunk). There are some notes (.txt files) in there that
may help in understanding the design or limitations; however these
notes should not be read as documentation for the trunk
implementation. The major features/changes are:
Audio refactor:
- Audio decoding and playback occur in separate threads to the main
thread; this is to avoid buffer underruns at low framerates and to
avoid having the application developer manually tune the ideal buffer
size and poll interval.
- PulseAudio replaces ALSA as the Linux audio backend
- Decoding, synchronisation, output and the public interface are more
loosely coupled and interchangeable, with better code reuse between
platforms.
- Audio/video synchronisation is redesigned, giving better
synchronisation and a simpler implementation
- Multithreading in Python turns out to be extremely flakey,
especially on Windows, where frequent process and even operating
system crashes were experienced. I'm not confident that all issues
were resolved -- besides the Vista crashes, Python's multithreaded
lock implementation is not stable and KeyboardInterrupt signals are
not delivered consistently. Interpreter shutdown with daemon threads
is not clean.
- While A/V sync was greatly improved, synchronisation between
multiple audio players was lost.
- In general, the media module needs far more testing and development
for it to replace the existing 1.1 implementation; or reverted.
Video resolution switching
- Mostly complete as far as possible given platform limitations
(Linux support in particular is limited).
- See the new "canvas" module for documentation
Window/input/GL refactor
- Related to the new canvas module, this reduces coupling between
these modules, in preparation for allowing pyglet to interoperate with
other GUI toolkits in various ways (embedding in both directions).
- OpenGL 3.0 is supported
- This refactor is considered complete, however there are probably
device configuration issues lingering
Tablet and joystick support
- Complete as far as scoped for this version. Tested only with the
limited number of tablets and joysticks I have on hand.
- Also, Apple Remote controls are supported (on OS X).
- See the new "input" module for documentation
Platform library refactor
- Shared platform library constants, function prototypes and types
moved to the new "libs" module in order to decrease coupling with
"window" module.
- Stable, but further refactoring is probably possible from other modules
I don't believe the new public APIs have been documented in the
Programming Guide, however the module and function-level documentation
is fairly fleshed out.
I would rate the stability of trunk at somewhere approaching an alpha
release for device and platform testing. For what it's worth, my
latest PyWeek entry used pyglet 1.2dev internally and no issues were
reported (the game had a fairly involved audio component, too).
As for advice on future development, I have a few notes which you are
free to take or ignore at your pleasure:
- The process for creating distributable Windows and OS X packages is
extremely time-consuming and error-prone (it is documented in
doc/internal). I would suggest dropping these distributions for just
the tar.gz/zip packages for developers to drop into their projects.
However the download counts for the installation packages are much
higher than the source/egg packages.
- Speaking of egg packages, these should also be dropped in future;
download counts are low and setuptools/easy_install is extremely
fragile, leading to broken installations that are difficult to repair
if you don't know where to look.
- It may be possible to release just some of the new 1.2 features
incrementally, rather than all of them at once. However the
refactoring of libs and canvas may make this difficult.
- I would be tempted to put audio/video in the "too hard" basket, and
create a C library with light Python bindings for this functionality
(much like AVbin does for media decoding).
- A Py3K conversion would be quite a lot of work.
Regards
Alex.
Tristam MacDonald
I would rate the stability of trunk at somewhere approaching an alpharelease for device and platform testing. For what it's worth, my
latest PyWeek entry used pyglet 1.2dev internally and no issues were
reported (the game had a fairly involved audio component, too).
- The process for creating distributable Windows and OS X packages is
extremely time-consuming and error-prone (it is documented in
doc/internal). I would suggest dropping these distributions for just
the tar.gz/zip packages for developers to drop into their projects.
However the download counts for the installation packages are much
higher than the source/egg packages.
- Speaking of egg packages, these should also be dropped in future;
download counts are low and setuptools/easy_install is extremely
fragile, leading to broken installations that are difficult to repair
if you don't know where to look.
- A Py3K conversion would be quite a lot of work.
Richard Thomas
Cleaning up the issue tracker would be a good first step though, even
going through and giving some of them a status would be useful.
Richard.
On Aug 15, 12:22 pm, Tristam MacDonald <swiftco...@gmail.com> wrote:
Richard Thomas
people were interested in the separation. Not sure if people are yet
but I thought I'd advertise it again.
Richard.
Steve Johnson
1.2dev at all. Could those with knowledge of the new internals and
undocumented code spend some time documenting it so that others can
figure it out more quickly?
As far as public documentation goes, that is one of the areas where I
think I can contribute meaningfully. My svn-fu is weak, so I don't
know how to do any fancy patch making/applying or branching/merging.
More internal documentation will help me do that.
Bruce Smith
....
It sounds to me as if we should focus our efforts on bringing trunk up to a full release, and put 1.2dev on hold for the immediate future.
Alex: thanks very much for your status description, that's just what we need.
Tristam: To clarify a possible misunderstanding: AFAIK, trunk and 1.2dev are the same thing, in the svn repository. That is, the most recent releases have been made from
http://pyglet.googlecode.com/svn/branches/pyglet-1.1-maintenance/
but new features and refactoring intended for "development towards 1.2" (as well as many bugfixes) have been done only in the trunk, i.e. in
http://pyglet.googlecode.com/svn/trunk/
since there is no separate (non-trunk) branch corresponding to 1.2dev. This means that "bring trunk to a full release, but put 1.2dev on hold" is not so simple, since trunk and 1.2dev are two names for the same "branch".
Alex, is my understanding correct?
If it is, then making a release from trunk will require either reverting some work (after copying it into a new separate 1.2dev branch) (figuring out exactly what to revert might be a lot of work and introduce new bugs), or finishing some work (especially in audio, based on Alex's summary), or accepting the possibility of a new release worsening behavior (especially in audio on Windows). (And accepting some loss of audio functionality in any case.) The latter choice (just release, and accept that some old behavior will have new bugs) is easiest, and might be acceptable -- I don't know.
Another possibility is to copy trunk to a new 1.2dev branch, revert trunk to 1.1-maintenance, and backport some bugfixes from the new 1.2dev branch to the new trunk. This is more conservative, but risks relegating a lot of good work in 1.2dev to the back burner for too long.
My sense is that cleaning all this up is unavoidably a significant amount of work, no matter how it's done, so the decision of what strategy to follow should be influenced greatly by whoever steps up to actually do that work (starting with reading the current code to fact-check and update Alex's summary).
Also it will be necessary to warn users that the next release might well break things that worked before.
Bruce Smith
....
Cleaning up the issue tracker would be a good first step though, even
going through and giving some of them a status would be useful.
Definitely. But when fixing a bug, before the future of the trunk/1.2dev vs 1.1-maintenance split is decided, it's important to make it clear in the bug report whether it's only been fixed in the trunk (the usual case, but worth saying explicitly IMHO) or also in the 1.1-maintenance branch.
Tristam MacDonald
On Sat, Aug 15, 2009 at 4:22 AM, Tristam MacDonald <swift...@gmail.com> wrote:....It sounds to me as if we should focus our efforts on bringing trunk up to a full release, and put 1.2dev on hold for the immediate future.
Alex: thanks very much for your status description, that's just what we need.
Tristam: To clarify a possible misunderstanding: AFAIK, trunk and 1.2dev are the same thing, in the svn repository. That is, the most recent releases have been made from
http://pyglet.googlecode.com/svn/branches/pyglet-1.1-maintenance/
but new features and refactoring intended for "development towards 1.2" (as well as many bugfixes) have been done only in the trunk, i.e. in
http://pyglet.googlecode.com/svn/trunk/
since there is no separate (non-trunk) branch corresponding to 1.2dev. This means that "bring trunk to a full release, but put 1.2dev on hold" is not so simple, since trunk and 1.2dev are two names for the same "branch".
Ben Smith
of the bug-fixes have been done in the trunk rather than on the 1.1
maintenance. So, I've made a first pass at identifying which issues
are addressed in the trunk and are not addressed in the maintenance
branch.
I pulled the log messages from revision 2156(where the maintenance
branch was made) to head from the trunk and the 1.1 maintenance
branch. I grabbed the issue numbers out of these logs with grep, so
I'm not claiming that this list entirely comprehensive. Here's the
list of (revision: message) issues I found that were fixed in the
trunk and were not fixed in 1.1 maintenance (according to log
messages):
2459: Issue 418: Added float division in win32 mousewheel event
2458: Issue 424: fix typo in documentation
2457: Issue 416: infinite loop in resource.reindex with win32 paths.
2456: Issue 422: document the requirement for '/' in resource.path
entries.
2455: Issue 414: multiple mouse buttons during drag on OSX (applied
patch from swiftcoder)
2454: Issue 409: putting back pyglet.media.have_avbin
2453: Issue 423: glMultiDrawElements was being given garbage data when
draw an IndexedVertexDomain
2452: Issue 381: migrating sprites lose VertexDomain attributes
2451: Fix for issue 413.
2450: Fix memory leak when loading images using QuickTime, issue 436.
2447: clean up better as per issue 408
2434: code cleanup for app and input, issue #393
2433: code cleanup for font and text, issue #392
2432: apply image cleanup patch for issue #391
2430: Issue 390: various typos addressed
2428: Issue 382: evdev name may be utf-8 encoded.
2425: Issue 368: Remove saved_by_zero font from astraea demo.
2424: Issue 368: fix executable bits of various files, remove shebangs
from non-exec files, fix line-endings on NOTICE.
2423: Issue 379: setup.py package list out of date
2422: Issue 380: OS X input typo
2421: Issue 376, add 2.6 to supported python versions in readmes
2420: Issue 377, scroll wheel broken on win32.
2402: PA driver: hold off on telling time until after write callback
following seek. Fixes backward seek issue.
2344: Issue 294: Clarify Texture.create(rectangle) parameter, add
force_rectangle parameter.
2339: Issue 357: Workaround ATI driver reporting GL 1.5 but not
exporting glGenBuffers.
2338: Issue 353: xlib key symbol mismatch when shift key held.
2337: Issue 361: mouse dx/dy incorrect when window freshly mapped
2326: Issue 355: hex and octal constants parsed by wraptypes.
2325: Issue 358: spelling fix.
2237: Fix double flip problem when manually running event loop on
win32 (e.g. WINDOW_INTIAL_FULLSCREEN test). Possibly fixes issue 335.
2227: Issue 334: check no modifiers present for ESC close window
shortcut
2170: Issue 328: always clean build dir before creating mpkg dist.
2169: Issue 319: unschedule leaving dummy funcs in schedule list
behind
2158: Issue 275: return True from EventDispatcher.dispatch_event if
the event was handled.
These are the issues I've found fixed in both trunk and 1.1
maintenance:
2364: Issue 362: fix generic attributes
2352: Issue 363: fix resource indexing of second level dirs.
2264: Issue 347: text is split into grapheme clusters before rendering
glyphs to texture. Tested on win32.
2262: Issue 345: mouse platform visible after window recreate
2239: Issue 336: WGL error checking requires initial context
2217: Issue 304, fix fix.
2215: Revert previous documentation commit and fix issue 331 as
described
And these are noted from log messages as fixed in maintenance but not
necessarily released:
2168: Issue 329: unschedule leaving dummy funcs in schedule list.
2446: Issue 410: typo fix
-b
On Aug 15, 12:09 pm, Tristam MacDonald <swiftco...@gmail.com> wrote:
> On Sat, Aug 15, 2009 at 2:32 PM, Bruce Smith <ores...@gmail.com> wrote:
Alex Holkner
> My understanding is that the 1.2 development is in one of two places:
> http://code.google.com/p/pyglet/source/browse/trunk/experimental/
> http://code.google.com/p/pyglet/source/browse/branches/enhancements_1_2/
> And that the trunk itself (with the exception of trunk/experimental)
> contains solely patches to the 1.1 development.
This is incorrect. 1.2dev is trunk, as Bruce described.
The enhancements_1_2 branch was a working branch for some of the new
features and can be disposed of now.
The experimental/ directory has always just been a place to put ideas,
whether slated for eventual release or not.
Alex.
Ben Smith
ported many of these issues from the trunk to a temporary branch.
Here's a link to a list of what's left as discrepancies between trunk
and this temporary branch: http://groups.google.com/group/pyglet-users/msg/380d28c8f71361a9?hl=en
-b
On Aug 15, 2:06 pm, Alex Holkner <alex.holk...@gmail.com> wrote: