Announcing Fritter: It's not spam

[Peter Law]

Hi,

At the moment the process of sending mailshots to the competitors
requires a lot of effort, some from those reviewing the email, but
mostly from the one person actually sending the email. In order to
stay within the rate-limiting restrictions of our accounts, the list
of emails must be manually chopped up (into groups of ~50 iirc) for
sending and the actual email sent several times. This takes a lot of
time, and yet each recipient still gets what is clearly a bulk email.
We can do better.

This issue was first raised four years ago in the form of ticket #635,
and is now hopefully approaching being resolved. With you help we can
reduce the effort required while improving the experience for everyone
involved.

A little while ago I created a system called Fritter [1] which reduces
the human interactions to the following:
* An email template is submitted for review (to Gerrit)
* Review occurs, using the same tooling and process as for code submissions
* The patchset is merged

See submission g:1949 for an idea of how this looks in action.
Note: only templates submitted via patches will be converted to
emails. Direct pushes are ignored.

Fritter is built on top of the existing template to email pieces from
nemesis, though these have been extracted and expanded into a new
library (libfritter [2]) which now includes validation and previewing
of the templates. It also wraps our existing gerrit listener
(gerritbot) in order to receieve events from Gerrit when the
submissions occur.

In the medium term nemesis should be moved over to using libfritter
(#2611), and doing so would provide a good introduction to both. (If
someone would like to pick this up I'd be very happy to provide
guidance towards getting set up and about what changes need to be
made.)

With Jeremy's help over the last week this has now been deployed onto
saffron via puppet. There are however still some configuration issues
we're working through. Specifically there's an issue where fritter
can't see the email addresses in LDAP, which g:1965 resolves.

During this testing phase feedback on the system both code [1] and
user experience are highly desirable.

The puppet config is a little limited in that it doesn't have any
means to interact with Gerrit. As a result, the following manual steps
are needed:
* Initial login to gerrit to create an admin account (the first user
to login becomes the admin)
* Login to gerrit as fritter, add the public key from dummy-secrets.git/fritter
* Login to gerrit as admin, create a group for the fritter user to be
part of, then grant that group streamEvents permissions

The fritter service logs to standard out, which ends up in
`/var/log/messages`. It's got pretty good logging of the actions being
performed, and errors do end up there too.

However, it should also be possible to set up a local instance which
can run the tests without actually having a full saffron clone running
(see the repo README for details).

Please therefore have a look at it and let us know what you think. If
ll goes well, we'll publicise this further (ie on srobo@) in time for
the next mailshot.

Thanks,
Peter

[1] https://www.studentrobotics.org/cgit/fritter.git
[2] https://www.studentrobotics.org/cgit/libfritter.git
#635 https://www.studentrobotics.org/trac/ticket/635
#2611 https://www.studentrobotics.org/trac/ticket/2611/
g:1949 https://www.studentrobotics.org/gerrit/#/c/1949
g:1965 https://www.studentrobotics.org/gerrit/#/c/1965

[Peter Law]

Hi,

> A little while ago I created a system called Fritter [1] which reduces
> the human interactions to the following:
> * An email template is submitted for review (to Gerrit)
> * Review occurs, using the same tooling and process as for code submissions
> * The patchset is merged

> With Jeremy's help over the last week this has now been deployed onto
> saffron via puppet. There are however still some configuration issues
> we're working through.

I'm pleased to say that these issues have now been (mostly) resolved.
The issue where you need to apply puppet twice from a clean image
unfortunately remains, as no apparent solution has appeared.

While there are no general usage docs, the classes themselves are
fairly well commented. As a result, I've connected up Read The Docs to
my github clone of libfritter. The docs for this library are therefore
now visible online at http://libfritter.readthedocs.org/en/latest/.
I may do something similar for the fritter portion at some point.

Since the system now appears to be working I'll post a general
announcement to the main list shortly.

Peter