Roundup issue tracker 2.1.0 (20th anniversary edition) released

[John P. Rouillard]

Hello Everybody:

I'm proud to release version 2.1.0 of the Roundup issue tracker.
This 20th anniversary edition has been possible due to the help of
several contributors. This release is a bugfix and minor feature
release, so make sure to read
https://www.roundup-tracker.org/docs/upgrading.html to bring your
tracker up to date.

2.1.0 builds on the 2.0.0 major release that introduced:

* Python 2 and Python 3 support
* a new REST interface
* updates to jinja2 templates including security improvements

The changes, as usual, include some new features and many bug fixes.

Note that you should run ``roundup-admin ... migrate`` to update the
database schema version. Do this before you use the web, command-line
or mail interface and before any users access the tracker.

You can download it with:

pip download roundup

then unpack and test/install the tarball.

Among the notable improvements from the 2.0.0 release are:

Mysql backend now uses an index to make sure that key values are not
duplicated when two roundup processes run in parallel. (Hence the
need for ``roundup-admin ... migrate``.)

Postgres back end uses a server side cursor. This reduces the memory
use of the roundup process on large queries.

Fix sorting of multilinks in templating code. Sorting by a link
without a value no longer generates a traceback. Sorting now works
as documented by the spec.

If ``roundup-admin security`` finds an invalid property, it exits
with a non-zero status. It can be used as part of a CI/CD pipeline
to validate schema security.

Security fixes for jQuery, markdown handling.

Fixes to markdown handling if using the jinja2 template.

Keyword editing in jinja2 template improved.

Fix a number of tracebacks.

Installation uses setuptools not distutils.

Valid class names are documented and enforced. All class names now
match ``[A-z][A-z0-9_]+[A-z_]``.

Replace a number of deprecated/missing functions in newer pythons.

Fix history showing invalid data when an update is rejected.

The filter command in roundup-admin works transitively and handles
empty values properly.

Password reset documented in user guide.

Admins can set the language used for stemming in the xapian indexer.

Devel and responsive template strings now extracted for translation.

The file CHANGES.txt has a detailed list of feature additions and
bug fixes (58) for each release. The most recent changes from
there are at the end of this announcement. Also see the
information in doc/upgrading.txt.

If you find bugs, please report them to issues AT roundup-tracker.org
or create an account at https://issues.roundup-tracker.org and open a
new ticket. If you have patches to fix the issues they can be attached
to the email or uploaded to the tracker.

Upgrading
=========

If you're upgrading from an older version of Roundup you *must* follow
all the "Software Upgrade" guidelines given in the doc/upgrading.txt
documentation.

Note that you should run ``roundup-admin ... migrate`` to update the
database schema version. Do this before you use the web, command-line
or mail interface and before any users access the tracker.

Roundup requires Python 2 newer than version 2.7.2 or Python 3 newer
than or equal to version 3.4 for correct operation.

To give Roundup a try, just download (see below), unpack and run::

python demo.py

then open the url printed by the demo app.

Release info and download page:
https://pypi.org/project/roundup
Source and documentation is available at the website:
https://roundup-tracker.org/
Mailing lists - the place to ask questions:
https://sourceforge.net/p/roundup/mailman/


About Roundup
=============

Roundup is a simple-to-use and install issue-tracking system with
command-line, web and e-mail interfaces. It is based on the winning design
from Ka-Ping Yee in the Software Carpentry "Track" design competition.

Note: Ping is not responsible for this project. The contact for this
project is rouilj at users.sourceforge.net. Use this address for
security or other sensitive issues. Development discussions occur on
the roundup-devel at lists.sourceforge.net mailing list. Tickets can
be opened at https://issues.roundup-tracker.org.

Roundup manages a number of issues (with flexible properties such as
"description", "priority", and so on) and provides the ability to:

(a) submit new issues,
(b) find and edit existing issues, and
(c) discuss issues with other participants.

The system facilitates communication among the participants by managing
discussions and notifying interested parties when issues are edited. One of
the major design goals for Roundup that it be simple to get going. Roundup
is therefore usable "out of the box" with any Python 2.7.2+ (or 3.4+)
installation. It doesn't even need to be "installed" to be operational,
though an install script is provided.

It comes with five basic issue tracker templates

* a classic bug/feature tracker
* a more extensive devel tracker for bug/features etc.
* a responsive version of the devel tracker
* a jinja2 version of the devel template (work in progress)
* a minimal skeleton

and supports four database back-ends (anydbm, sqlite, mysql and postgresql).

Recent Changes
==============

>From 2.0.0 to 2.1.0.

Fixed:

- Reverse multilink to *the same class* would trigger a traceback about
a modified dictionary on iteration (Ralf Schlatterbeck)
- issue2551086 - Valid class names not documented. Should follow
``[A-z][A-z0-9_]+[A-z_]``. This was never documented or enforced, but
we get obscure errors if the rules are not followed. (Tom
Ekberg tests by John Rouilard)
- issue2550564 - Roundup sets "Precedence: bulk" on all outgoing mail,
which seems wrong. Handle Auto-Submitted header on *inbound* email
like we do precedence bulk. This is part of this issue.
- roundup-admin filter calls find() not filter when using -s -c -S
(John Rouillard)
- When requesting transitive properties via ``@fields`` in the REST-API,
an empty link in the transitive property (e.g. author.username when
requesting message properties) would result in a 404 error. Now we're
returning a JSON 'null' value. for an empty link (e.g. empty author in
the example). (John Rouillard)
- sphinxcontrib.cheeseshop is unmaintained and using old http
url. Attempts to override cheeseshop_url failed. Replace call to
cheeseshop in docs with raw html and remove references to
cheeseshop. (John Rouillard)
- issue2551093 - return plain text if markdown formatter throws exception
(reported by Cedric Krier, fix by John Rouillard)
- issue2551094 - make simplemde handle line breaks the same as the
backend markdown formatters. (report: Cedric Krier, patch: Christof
Meerwald)
- issue2551092 - fix crash bug by aligning
``roundup.anypy.email_.decode_header`` with stdlib ``email.header`` and
convert string to bytes for python 3. (Cedric Krier)
- issue2551097 - fix underlying bug in use of fenced codeblocks with
markdown2. Fix for issue2551093 to prevent exception trigger.
(patch: Cedric Krier)
- issue2551099 - disable processing of data url's in markdown. Display
as plain text. (John Rouillard)
- issue2551100 - old jquery has security issues, upgrade it and fix
user.help.html (John Rouillard)
- replace deprecated base64.decodestring with base64.b64decode in
roundup_server.py and roundup_xlmrpc_server.py (reported by
lmsteffan in irc)
- removed run_tests.py. Newer pytest doesn't support generating
stand alone testing bundles. Python 3.9 generates errors running
the current run_tests.py. (reported by lmsteffan in irc)
- issue2551104 - fix issue with markdown autolink next to punctuation (ced)
- removed support for old style trackers that use dbinit.py and
config.py. Also remove all uses of deprecated imp module. (John Rouillard)
- removed support for setting database type using
<database>/backend_name. (John Rouillard)
- fixed some issues when generating translations. Use mappings and
named format parameters so translators can move substituted tokens
in translations. (John Rouillard)
- in rest interface, fix uncaught exceptions when parsing invalid
Content-Type and Accept headers. Document response formats more
fully in doc/rest.txt. (John Rouillard)
- in filter, filter_iter and _materialize_multilinks, use named cursor
with postgresql. This turns of client-side cursor handling and avoids
*large* roundup process (or wsgi process) in case of large results.
Fixes issue2551114. (Ralf Schlatterbeck)
- issue2551108 - fix handling of designator links when formatted
as markdown links. (Reported by Cedric Krier; John Rouillard)
- Fix filename created from mail attachments, fixes issue2551118
- Call verifyPassword even if user does not exist. Address timing
attack to discover valid account names. Useful where anonymous user
is not allowed access. (John Rouillard)
- issue2551126 - AttributeError: 'str' object has no attribute
'local'. Fix traceback caused by DateHTMLProperty.pretty() called
on a string value due to error in some other field. (Reported by
reda, fix: John Rouillard)
- issue2550899 - Migrate setup.py to setuptools; fixes:
issue2550866 'pip install --editable .' fails; et al.
this now requires that setuptools be installed. (Patch by John
Kristensen (jerrykan); additional doc changes (upgrade.txt,
RELEASE.txt) John Rouillard)
- issue2551128 - Impossible to validate a user with unknown timezone
Raise KeyError when an unrecognized timezones is passed to
pytz. (patch Cedric Krier, test John Rouillard)
- issue2551129 - Template not found return 500
Handle traceback caused when requested @template is not found.
Return 400 error in this condition. (patch Cedric Krier,
additional change and test John Rouillard)
- issue2551062: roundup-admin security now exits status 1 when
it finds an invalid property. It no longer tries to print the rest
of the security properties. (John Rouillard)
- issue2551078 - Fix traceback caused when putting two id's into a
Link html field. A ValueError is raised. Handle exception and return
value. hyperdb.py now reports 'you may only enter ID values for
property ...' to the user. (John Rouillard)
- issue2551120 - The sorted method of MultilinkHTMLProperty crashes,
if the given property is unset for an element of the list. Crash
fixed. New feature NoneFirst added to method to make unset values
sort at start or end of sorted list. (John Rouillard)
- issue2550648 - keyword boolean search. Issue has multiple problems.
Fix issue where saving the keyword boolean search would remove the
link to open the editor. (John Rouillard)
- issue2551136 - timezone extention crash on Python 3.8. cgi.escape
is used in some template to provide a select box of timezones. It
uses cgi.escape that is deprecated and removed from 3.8 and newer.
Use html.escape with fallback to cgi.escape. (Cedric Krier)
- roundup-server can act as an SSL server. Usually SSL is provided by
a front-end server like nginx, hiawatha, apache. The SSL parameters
have been upgraded to TLS 1.1. Cert is RSA 2048 bytes with SHA512
signature. Without these upgrades, ssl mode won't start. Note this
exposes other issue with roundup-server operating as an SSL
endpoint. See issue2551138 and issue2551137. (John Rouillard)
- issue2551122 - sorted method of MultilinkHTMLProperty does a string
sort even if the property is an integer. Fixed so that the orderprop
for the linked class is used. (John Rouillard, reported by Nagy Gabor)
- issue2550964 - History can (temporarily) show incorrect value when a
change is rejected. Fix history function to always use the database
values and ignore the current setting in the form. (John Rouillard)
- Fix find() with anydbm. Using protected properties raised KeyError.
Add shortcut fast return. Both changes come from rdbms_common.py's
find(). (John Rouillard)
- Fix traceback caused by calling history() with arguments in a
non-item context. (John Rouillard)
- issue2551141 - roundup-admin returns no such class when restoring
item with duplicate key. Fix incorrect error message when using
roundup-admin to restore a user when the username is already in use.
(John Rouillard)
- issue2551142 - Import of retired node with username after active
node is imported raises unique constraint failure. (Reported by Ganesh
Sittampalam/Heffalump on irc. John Rouillard)
- *** Must run roundup-admin migrate ***
Increment rdbms version from 5 to 6. Mysql rdbms classes were
missing unique key constraint. Found during fix for issue2551142.
See upgrading.txt. (John Rouillard)
- ignore blank lines in CSV class editing. (John Rouillard)
- issue2551122 - fixing order by a link/multilink broke other props
should be final change for that ticket. (John Rouillard)

Features:

- issue2550522 - Add 'filter' command to command-line
interface. Filter command was actually added in 2.0.0, but this
issue requested transitive searching. So that::

roundup-admin -i . filter issue assignedto.username=Admin

will work. This also fixes a bug. If assignedto.username had no
matches, all issues would be returned. This is also fixed.
(John Rouillard)
- issue2550716 - Email address displayed after password reset request.
This fix actually made it into 1.6 release. However this release
documents how password reset works in user_guide.txt. (John Rouillard)
- issue2551094 - add new markdown config.ini setting to allow embedded
newlines to cause a linebreak same as GitHub Flavored Markdown.
(Patch: Cedric Krier; Doc change/checkin John Rouillard)
- issue2551096 - enable markdown autolink for email and bare url's.
Modify raw markdown adding appropriate link markers on the fly.
(Cedric Krier)
- issue2551098 - add rel="nofollow" for links generated by markdown2
backend and rel="nofollow noopener" for mistune and markdown
backends. Prevents link spam. noopener prevents security issue when
available. (John Rouillard)
- Added explanation for modifying Fileclass content files to
customizing.txt. Result of mailing list question. (John Rouillard)
- issue2551109 - Improve keyword editing in jinja2 template. (Cedric Krier)
- issue2551117 - Add example systemd config
- Allow admin to configure language used for stemming in xapian
indexer. (John Rouillard request by Nagy Gabor)
- Move memorydb from test to roundup/test to allow regression-testing in
tracker instances without copying code. Also move the test-detectors in
tx_Source_detector.py to roundup/test for two reasons: It's used in the
memorydb convenience functions and it may be useful in other tests. Make
the prefix a parameter of the convenience functions to be usable in other
tests. (Ralf Schlatterbeck)
- pytest suite now starts the server under wsgi and loads the home
page. This test is skipped if the requests module is not installed.
- extract translatable strings from devel and responsive templates. Merge
translations from https://sourceforge.net/p/roundup/code/merge-requests/3/
(John Rouillard. DE translations by Tobias Herp.)
- send_message now allows setting authid to set source of email.
(John Rouillard)
- issue2550837 - New option for web auth (also http header passing).
Allow admin to configure authentication header replacing the default
REMOTE_USER. Also allow arbitrary headers to be passed to the
tracker when using roundup-server behind a proxy. This code is
experimental see upgrading.txt admin_guide.txt. (John Rouillard)
- add image/svg-xml as valid mime type to serve. Was being served as
octet-stream. (John Rouillard)
- improve customizing.txt documentation on use of Special Form
Variables. Added example html inputs to illustrate the doc.
Fix position of designator in doc example. It occurs before
@link@ or other edit command. (John Rouillard)

--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.