0.11.4 and release testing
Jonas Borgström
0.11.3 unfortunately contained a few regressions so we need to get
0.11.4 out pretty soon.
Most issues seems to have been fixed in 0.11-stable already but are
there any other bug fixes that need to go into 0.11.4 as well?
And to avoid similar regressions in the future we need to do something
to improve our pre-release testing.
What is the best approach? Pre-release testing checklists, release
candidates or both?
/ Jonas
Remy Blank
> Most issues seems to have been fixed in 0.11-stable already but are
> there any other bug fixes that need to go into 0.11.4 as well?
I'd like to fix the following ticket for 0.11.4:
http://trac.edgewall.org/ticket/8059
> What is the best approach? Pre-release testing checklists, release
> candidates or both?
I would suggest at least one release candidate, and trying to enlist a
few people with various architectures and configurations (OS, database,
web server, front end) for testing. I'm sure we can find a few people on
this list who would be willing to test release candidates on their setups.
A standard testing procedure would be nice, but longer term we should
try to automate the procedure completely. A script that installs Trac
from a tarball or SVN tag, creates a new environment, and performs and
checks various operations on it would be very useful. There is some
overlap with functional tests, maybe they should just be extended to
allow using e.g. other databases, and to operate on an environment
created from scratch.
Not related to 0.11.4, we should also diversify the configurations on
which the automatic unit tests are run. I am in the process of setting
up an OS X machine with Python 2.3 - 2.6, and a Windows VM. Could these
be used as build bots for t.e.o?
-- Remy
Jonas Borgström
> Jonas Borgström wrote:
>> Most issues seems to have been fixed in 0.11-stable already but are
>> there any other bug fixes that need to go into 0.11.4 as well?
>
> I'd like to fix the following ticket for 0.11.4:
>
> http://trac.edgewall.org/ticket/8059
>
>> What is the best approach? Pre-release testing checklists, release
>> candidates or both?
>
> I would suggest at least one release candidate, and trying to enlist a
> few people with various architectures and configurations (OS, database,
> web server, front end) for testing. I'm sure we can find a few people on
> this list who would be willing to test release candidates on their setups.
Good idea, I created a simple wiki page where people interested in
helping us out can sign up:
http://trac.edgewall.org/wiki/TracDev/ReleaseTesting
> A standard testing procedure would be nice, but longer term we should
> try to automate the procedure completely. A script that installs Trac
> from a tarball or SVN tag, creates a new environment, and performs and
> checks various operations on it would be very useful. There is some
> overlap with functional tests, maybe they should just be extended to
> allow using e.g. other databases, and to operate on an environment
> created from scratch.
>
> Not related to 0.11.4, we should also diversify the configurations on
> which the automatic unit tests are run. I am in the process of setting
> up an OS X machine with Python 2.3 - 2.6, and a Windows VM. Could these
> be used as build bots for t.e.o?
Yes definitely. A py23 build bot is especially interesting given our
current 0.11.x track record. A windows build bot is of course also very
interesting.
Let me know if you need some help getting Bitten to work.
/ Jonas
Christian Boos
> Hi all,
>
> 0.11.3 unfortunately contained a few regressions so we need to get
> 0.11.4 out pretty soon.
>
> Most issues seems to have been fixed in 0.11-stable already but are
> there any other bug fixes that need to go into 0.11.4 as well?
>
* Stopped sync in 0.11.2.1
http://trac.edgewall.org/ticket/8067
Not sure the proposed patch really helps, as the rollback should
happen anyway. Clarification of the installation requirements
for MySQL seems to be the really important point there.
* Trac does not send object names in quotes to PostgreSQL
http://trac.edgewall.org/ticket/7600
ecarter fixed it in r7923, independently from the provided patch,
but there's also a contributed unit test on that ticket that could be
checked in as well.
> And to avoid similar regressions in the future we need to do something
> to improve our pre-release testing.
> What is the best approach? Pre-release testing checklists, release
> candidates or both?
>
Besides what has already been proposed so far, I'd say we should also
make official release candidates (tagged as "rc1"), then freeze the
stable branch. If there's no complaints in the next week, which is
usually the time frame during which the issues escaping our own testing
are found, we simply redo the packages without the "rc1" tag. If a
change is needed, the procedure repeats with a "rc2", eventually with a
shorter freeze period, if we're confident the fix was trivial (but as
usual, beware of trivial fixes ;-) ).
-- Christian
anatoly techtonik
>
> * Stopped sync in 0.11.2.1
> http://trac.edgewall.org/ticket/8067
>
> Not sure the proposed patch really helps, as the rollback should
> happen anyway. Clarification of the installation requirements
> for MySQL seems to be the really important point there.
I agree that the patch won't help here much. A word of caution and a
message in log if DB frontend is MySQL and at least one table is
MyISAM. The check in SQL:
select table_name, engine from information_schema.tables where
table_schema=DATABASE();
>> And to avoid similar regressions in the future we need to do something
>> to improve our pre-release testing.
>> What is the best approach? Pre-release testing checklists, release
>> candidates or both?
Both, but also instructions and automation scripts.
http://trac.edgewall.org/wiki/TracDev/ReleaseTesting is not enough.
Ideally a person should be able to execute one test_trac.py script
that will ask a few questions if needed (or use config file) and
execute all tests itself.
In a distant future FitNesse-like framework may help to add/experiment
with tests more interactively.
--
--anatoly t.
Shane Caraveo
if testing is this week, but I've added myself to the wiki page.
Shane
rupert.thurner
mediawiki?
rupert.
Eli
> A standard testing procedure would be nice, but longer term we should
> try to automate the procedure completely. A script that installs Trac
> from a tarball or SVN tag, creates a new environment, and performs and
> checks various operations on it would be very useful. There is some
> overlap with functional tests, maybe they should just be extended to
> allow using e.g. other databases, and to operate on an environment
> created from scratch.
The environment is already created from scratch by the functional tests.
Tim and I have talked about a wider range of DB backends; there's a lot that
could be done there.
> Not related to 0.11.4, we should also diversify the configurations on
> which the automatic unit tests are run. I am in the process of setting
> up an OS X machine with Python 2.3 - 2.6, and a Windows VM. Could these
> be used as build bots for t.e.o?
The wider the variety, the better.
Eli
------------------. "If it ain't broke now,
Eli Carter \ it will be soon." -- crypto-gram
retr...@gmail.com `-------------------------------------------------
John Hampton
> On Sunday 01 March 2009 12:19:12 pm Remy Blank wrote:
>> A standard testing procedure would be nice, but longer term we should
>> try to automate the procedure completely. A script that installs Trac
>> from a tarball or SVN tag, creates a new environment, and performs and
>> checks various operations on it would be very useful. There is some
>> overlap with functional tests, maybe they should just be extended to
>> allow using e.g. other databases, and to operate on an environment
>> created from scratch.
>
> The environment is already created from scratch by the functional tests.
>
> Tim and I have talked about a wider range of DB backends; there's a lot that
> could be done there.
And, extending the testing to more DB backends is on the list for the
PyCon Sprint. Some everyone come to PyCon and help out! ;)
-John
Jonas Borgström
Yes, and I've just updated it to [7938].
Anyway, most or all tickets mentioned in this thread as blockers seems
to be fixed now.
#8067 is still open but as far as I can tell the mysql myisam engine is
the real culprit:
http://trac.edgewall.org/ticket/8067
So unless any new blocker has surfaced I can make Trac 0.11.4-rc1
available for testing sometime tomorrow.
/ Jonas
Christian Boos
> On 3/3/09 8:59 AM, Remy Blank wrote:
>
>> rupert.thurner wrote:
>>
>>> would running -stable on teo be an option, like wikipedia does it with
>>> mediawiki?
>>>
>> I think that's already the case. AFAIK, Jonas is updating t.e.o
>> periodically to 0.11-stable.
>>
>
> Yes, and I've just updated it to [7938].
>
> Anyway, most or all tickets mentioned in this thread as blockers seems
> to be fixed now.
>
> #8067 is still open but as far as I can tell the mysql myisam engine is
> the real culprit:
> http://trac.edgewall.org/ticket/8067
>
"Done". There's eventually #8089 for people caring about MySQL support.
Not a blocker for 0.11.4, though.
> So unless any new blocker has surfaced I can make Trac 0.11.4-rc1
> available for testing sometime tomorrow.
>
Yes, there's one. I unfortunately broke the blockquote and citation
styles when trying to fix #7970. If we don't find a way to fix the
regression, I'll simply revert r7820.
-- Christian
Christian Boos
I think we're now (r7945) in a good shape for a rc1.
For the last ticket I left opened for 0.11.4
(http://trac.edgewall.org/ticket/8128), the patch attached there should
work, but as I'm not able to fully test it from end to end, I think it
needs another pair of eyes. It shouldn't harm if committed, but there's
also no big pressure for it, so it's your take jonas.
-- Christian
Jonas Borgström
> I think we're now (r7945) in a good shape for a rc1.
>
> For the last ticket I left opened for 0.11.4
> (http://trac.edgewall.org/ticket/8128), the patch attached there should
> work, but as I'm not able to fully test it from end to end, I think it
> needs another pair of eyes. It shouldn't harm if committed, but there's
> also no big pressure for it, so it's your take jonas.
I think the patch looks both good and safe.
I have just committed it to make sure it makes it into rc1 which I've
started preparing.
/ Jonas