Email bug reporting
Florian Schmitz
We put a lot of thought into this feature recently, but so far we have
been unsuccessful finding a suitable implementation. So unless someone
on this list has a great new idea for its implementation, we'll most
likely drop this feature (note that it is still possible to do it with
Jabber though).
The base idea is to have a dedicated IMAP/POP3 account where everyone
can send emails to and Flyspray checks it on a regular basis for new
mails, to make it work without any difficult server configuration.
Now we thoguht about these possibilities:
1. We trust the senders
This is quite simple to implemenet, but it's obviously insecure (also
consider spam). I don't think that we want to have such a feature.
2. Moderated emails
The list of available emails is somwhere shown in the admin panel /
project management area, and there it is decided which emails are
processed and which are not. This is slightly more secure (you could
still fake emails though), but it doesn't really increative productivity
and lacks speed.
3. Authentication
We could require an authentication token within the email (maybe
user+password, or rather a special MD5 hash). That would be quite
secure, but then again you could just use the web interface.
4. Encryption
Use PGP to encrypt and sign mails. Sounds good, but is a whole (too) lot
of work since no libraries available for this functionality.
Comments appreciated.
Regards,
Flo
Krisztian VASAS
On Sun Jun 03, 2007 at 10:55:17 +0200, Florian Schmitz wrote:
>The base idea is to have a dedicated IMAP/POP3 account where everyone
>can send emails to and Flyspray checks it on a regular basis for new
>mails, to make it work without any difficult server configuration.
This is a good idea.
>Now we thoguht about these possibilities:
>
>1. We trust the senders
>This is quite simple to implemenet, but it's obviously insecure (also
>consider spam). I don't think that we want to have such a feature.
>
>2. Moderated emails
>The list of available emails is somwhere shown in the admin panel /
>project management area, and there it is decided which emails are
>processed and which are not. This is slightly more secure (you could
>still fake emails though), but it doesn't really increative productivity
>and lacks speed.
>
>3. Authentication
>We could require an authentication token within the email (maybe
>user+password, or rather a special MD5 hash). That would be quite
>secure, but then again you could just use the web interface.
As the mail is plaintext, this is not the best solution.
>4. Encryption
>Use PGP to encrypt and sign mails. Sounds good, but is a whole (too) lot
>of work since no libraries available for this functionality.
Maybe this can help:
http://www.php.net/manual/en/ref.gnupg.php
IroNiQ
--
Member of Frugalware Developer Team
Web: http://www.ironiq.hu
LinuxCounter: #331532
Florian Schmitz
> http://www.php.net/manual/en/ref.gnupg.php
I fear that virtually no one has this extension installed. Then we'd
invest a lot of time coding a feature no one can use.
Regards,
Flo
Martin Marconcini
On Jun 3, 2007, at 10:55 AM, Florian Schmitz wrote:
Hi,
>
> The base idea is to have a dedicated IMAP/POP3 account where everyone
> can send emails to and Flyspray checks it on a regular basis for new
> mails, to make it work without any difficult server configuration.
>
I like it.
>
> 1. We trust the senders
This won't work :)
>
> 2. Moderated emails
> The list of available emails is somwhere shown in the admin panel /
THis is the way to go (to keep the whole thing simple).
One way to "moderate" spam is to limit the number of Bug reports/hr
(for example) to prevent a MASS email entering the BTS.
How about a "TMDA" like mechanism? Like when you send a bug report by
email, the system returns you another one which you have to reply in
order to make sure you're not "a bot".
99% of the spam stuff are sent with mailers which do not handle TMDA ok.
http://tmda.net/
When you auth yourself (by replying) you go to the white list (else
your email gets either deleted or blacklisted after N amount of days
without a reply).
I have used this in the past and i can say that I almost never had a
spam pass thought it. I removed it cuz "users" didn't fully
understand that I needed for them to hit reply xD and I had to
manually read the list of "awaiting aproval emails" to see if there
was something good waiting for "user stupidity". ;)
But in Flyspray's case, no "regular" users will have to send emails
here. And then again, if they have to, this could be explained to
them. In TMDA, if i don't recall wrong, when you were whitelisted, it
was permanent. It could be based upon a timing here (N number of days
of inactivity?) dunno. Something like that.
> 3. Authentication
> We could require an authentication token within the email (maybe
> user+password, or rather a special MD5 hash). That would be quite
> secure, but then again you could just use the web interface.
>
Aye, but emails are plaintext... but it's not bad anyways to have a
special "email hash password" in order to be able to post. If
somebody wants to fake you, and he's sniffing your traffic, you have
worse problems methinks. :)
> 4. Encryption
> Use PGP to encrypt and sign mails. Sounds good, but is a whole
> (too) lot
> of work since no libraries available for this functionality.
Possibly, cuz it will add a new req. (the gnupgp libraries) and all
the coding involved.
My 2cents. :)
--
Martin Marconcini
"A complex part of programming doesn't even involve writing code. I
am referring to the interaction between computer programs and people.
User interaction is a complex subject that has been the subject of
many books." Annonymous.
Jonathan Oxer
> We put a lot of thought into this feature recently, but so far we have
> been unsuccessful finding a suitable implementation.
I think this would be a very useful feature, but IMHO it shouldn't be
implemented as an internal feature within FS itself: this is a perfect
example of something that could be done as a helper using a SOAP or
XML-RPC interface.
This has been discussed a few times over the years. Here are a couple of
examples from the list archives:
http://article.gmane.org/gmane.comp.bug-tracking.flyspray.devel/286/
http://article.gmane.org/gmane.comp.bug-tracking.flyspray.devel/538/
http://article.gmane.org/gmane.comp.bug-tracking.flyspray.devel/551/
http://article.gmane.org/gmane.comp.bug-tracking.flyspray.devel/403/
http://article.gmane.org/gmane.comp.bug-tracking.flyspray.devel/1522/
If FS provided a consistent API for things like adding tasks and
comments then there's no need to bloat it with extra code for things
like handling incoming email. Features like that are a natural to be
separated out of the main codebase: just add a separate script that does
the mail processing and connects to the FS API to add the actual task.
As long as the API is consistent (and/or versioned) the core FS
developers can remain focused on FS internals, and other developers can
focus on helper scripts such as an email interface without getting in
the way of core development. That's a great way to "segment" the
development of a project like FS so that as it grows different people
can contribute / maintain sub-sections independently.
Personally I'd love to see an API re-introduced so that I can resurrect
my SVN integration scripts and rewrite them in a version-independent
way. At the moment my scripts can do things like open and close tasks
and add comments on SVN commits, but because of changes to FS internals
they won't work with current FS version unless I change my code to match
FS' current structure. If instead FS provided an API that I could code
to then they would keep working regardless of the version.
So my plea is for anyone willing to add an API to FS that will allow
external scripts to perform actions such as:
* Add task
* Add comment
* Close task
* Add user
* Update password
If those things were covered it would be a cinch to write external
scripts for a bunch of things, including email->FS, SVN->FS, custom web
form->FS, or whatever. You could even write an Asterisk->FS gateway for
logging voicemail messages as tasks with attachments if you really
wanted to, and the core FS developers wouldn't have to do a single thing
to support it - or care what you're doing, as long as the API does its
job.
Cheers :-)
--
Jonathan Oxer
Ph +61 3 9723 9399
Internet Vision Technologies: <www.ivt.com.au>
"Ubuntu Hacks": <www.ubuntuhacks.com>
"How To Build A Website And Stay Sane": <www.stay-sane.com>
Cristian Rodriguez
> >4. Encryption
> >Use PGP to encrypt and sign mails. Sounds good, but is a whole (too) lot
> >of work since no libraries available for this functionality.
>
> Maybe this can help:
> http://www.php.net/manual/en/ref.gnupg.php
>
hell no, trying to kill ants with a tank now :-)
--
"I write my code for anyone to use, and 'anyone' includes evil
megacorporate monopolists pretty much by definition. I wouldn't change
those terms retroactively if I could, because I think empowering
everyone is a far more powerful statement than empowering only those I
agree with.." -ESR
Cristian Rodriguez
> So my plea is for anyone willing to add an API to FS that will allow
> external scripts to perform actions such as:
>
The API you request is already being worked on by me, as a "plugin" (
not part of flyspray( , it is a REST based service and due to several
reasons, it only runs with PHP5 ( fundamentally because the libraries
requires it)
I'll publish it when it reaches a decent quality.
Jonathan Oxer
> The API you request is already being worked on by me, as a "plugin" (
> not part of flyspray( , it is a REST based service and due to several
> reasons, it only runs with PHP5 ( fundamentally because the libraries
> requires it)
>
> I'll publish it when it reaches a decent quality.
Christian, you're a champion.
Florian Schmitz
> developers can remain focused on FS internals, and other developers can
> focus on helper scripts such as an email interface without getting in
> the way of core development.
Now here's the problem: There are no such "other" developers currently.
And even if someone contributed an XML API, we'd probably end up with
having to maintain it on our own (as it was the case in 0.9.8) since
long-term contributions happen even more rarely.
And btw, such an API is not a solution for the email problem. It doesn't
help us with the main hurdle, which is to verify the incoming emails.
Regards,
Flo
Cristian Rodriguez
> And btw, such an API is not a solution for the email problem. It doesn't
> help us with the main hurdle, which is to verify the incoming emails.
It solves the issue partially ;). whatever rare way to access flyspray
or perform flyspray tasks from a third party, non-web app can be done
with this API, the only need for the third party code is to be able to
talk the HTTP protocol.
Jonathan Oxer
> Now here's the problem: There are no such "other" developers currently.
But there *could* be, and this is one way to encourage it. In fact it's
for exactly the same reason that I asked for a publicly accessible 1.0
test system to be set up: the way to encourage participation is to lower
barriers to it. The lack of feedback on proposed 1.0 features was
because it was too much work for typical admin-level users of FS to
install the current dev version just so they could comment on new
features. Immediately after the demo system was set up there were a
bunch of comments on specific features, which is exactly what you and
Christian wanted and totally proved the point.
This situation may appear to be different but it's actually exactly the
same, and I'll pick on myself as an example. I have a FS installation
with a bunch of integration work applied to it for linking into
Subversion, into package build scripts, and for release management. I've
talked about it at a bunch of conferences and if you're interested there
are slides from one such talk here:
http://jon.oxer.com.au/talks/id/46
Flyspray forms the core of the system in that talk, with a bunch of
other things hanging off it that trigger/react to external events. But
the integration scripts are all tightly bound to FS 0.9.8 because they
have to go behind Flyspray's back to talk straight to its database, and
if the DB schema changes everything else has to change with it. So the
internal FS at my company is still at 0.9.8, and that's not going to
change any time soon because everything else would have to be re-written
to suit.
But if those same integration scripts could instead be written against
an API they would continue to work even if the FS internals are
completely rearranged, and they could usefully be provided to other FS
users who might want to deploy them.
The point is this: if you want other people to help, make it easy to do
so. Right now it's too much work for me to bother rewriting my external
scripts to suit newer FS internals. But if there was an API it would be
a totally different story. Make it easy for people to help, and they
will. *I* will, at least. Most potential developers won't want to become
core FS developers or care about all the internals because all they want
to do is fix their particular little problem or add their personal
feature.
> And even if someone contributed an XML API, we'd probably end up with
> having to maintain it on our own (as it was the case in 0.9.8) since
> long-term contributions happen even more rarely.
Probably true, but I posit that it would be far less work for you to
maintain a single API implementation with about 6 external methods than
to maintain a big chunk of email interface management code which does
everything the API would do plus more.
> And btw, such an API is not a solution for the email problem. It doesn't
> help us with the main hurdle, which is to verify the incoming emails.
Sure it could: just provide a "validate user" method in the API.
Florian Schmitz
> core FS developers or care about all the internals because all they want
> to do is fix their particular little problem or add their personal
> feature.
Well, addon-like contributions that build upon XML-RPC APIs or similar
are nice, but actually a few more core developers would be quite
important. We saw how fast a development team of about 5 people can
fall apart, right now we are just 2.
> But if those same integration scripts could instead be written against
> an API they would continue to work even if the FS internals are
> completely rearranged, and they could usefully be provided to other FS
> users who might want to deploy them.
Even an XML-RPC will change over time though as long as Flyspray is
undergoing major changes. It might be somewhat easier to update your
scripts for XML changes, but it will be necessary once in a while.
> Probably true, but I posit that it would be far less work for you to
> maintain a single API implementation with about 6 external methods than
> to maintain a big chunk of email interface management code which does
> everything the API would do plus more.
That's not quite right. You know, we already have some nice backend
functions which we could simply use for the email stuff without adding
a lot of code. Actually we use those for the Jabber interface already.
So for Flyspray internal features, such an interface would not be
neccessary.
Maybe it would be a good idea though to write an RPC wrapper for
exactly those functions, as far as I can see it shouldn't require a
lot of work or code. I guess I'll look into that after I am done with
SVN integration.
> Sure it could: just provide a "validate user" method in the API.
How would that method work? That's the only problem actually.
Regards,
Flo
Julien Nigay
Hi everyone.
We put a lot of thought into this feature recently, but so far we have
been unsuccessful finding a suitable implementation. So unless someone
on this list has a great new idea for its implementation, we'll most
likely drop this feature (note that it is still possible to do it with
Jabber though).
The base idea is to have a dedicated IMAP/POP3 account where everyone
can send emails to and Flyspray checks it on a regular basis for new
mails, to make it work without any difficult server configuration.
Now we thoguht about these possibilities:
1. We trust the senders
This is quite simple to implemenet, but it's obviously insecure (also
consider spam). I don't think that we want to have such a feature.
2. Moderated emails
The list of available emails is somwhere shown in the admin panel /
project management area, and there it is decided which emails are
processed and which are not. This is slightly more secure (you could
still fake emails though), but it doesn't really increative productivity
and lacks speed.
3. Authentication
We could require an authentication token within the email (maybe
user+password, or rather a special MD5 hash). That would be quite
secure, but then again you could just use the web interface.
4. Encryption
Use PGP to encrypt and sign mails. Sounds good, but is a whole (too) lot
of work since no libraries available for this functionality.
Martin Marconcini
On Jun 4, 2007, at 10:57 AM, Julien Nigay wrote:
>
>
> Maybe you could take a look to some blog systems such as
> Serendipity or WordPress, I don't remember wich one, that provide a
> way to post your entries by email. It could be interesting to see
> how they do authentification.
>
It is WordPress.
Florian Schmitz
> so it means the user is > valid.
Unfortunately, due to shortcomings in the technology called "email",
this is not possible. Everyone can easily fake the from-address. We
can only use this information to determine from which user the email
comes, but we cannot *verify* that information.
Regards,
Flo
Julien Nigay
Martin Marconcini
On Jun 4, 2007, at 11:23 AM, Julien Nigay wrote:
> That's why I suggest to see how WordPress work, they probably spent
> some time on that question before implementing this functionnality.
> I hope they use something more secure than just email
> authentification :)
AFAIK, they don't.
They let you enter an "ofuscated" email account. Like:
post123to_my_Blog_...@yourdomain.com
If an email comes from that email, it gets posted. This is how it
used to work "a few months back when I saw it". Dunno if they changed
it.
--
Julien Nigay
Florian Schmitz
> How about a "TMDA" like mechanism? Like when you send a bug report by
> email, the system returns you another one which you have to reply in
> order to make sure you're not "a bot".
Is this desirable? When doing a lot of stuff with Flyspray emails it
might be somewhat annoying to confirm them all (just one is not
enough).
If it's not (I suppose so), we just drop this feature and focus on
APIs, so that anyone might code the script they need to match just
their own security requirements.
Regards,
Flo
Martin Marconcini
On Jun 4, 2007, at 9:39 PM, Florian Schmitz wrote:
>
> Now to get back to the topic a little...
>
>> How about a "TMDA" like mechanism? Like when you send a bug report by
>> email, the system returns you another one which you have to reply in
>> order to make sure you're not "a bot".
>
> Is this desirable? When doing a lot of stuff with Flyspray emails it
> might be somewhat annoying to confirm them all (just one is not
> enough).
NOno, in TMDA when you confirm "one", you're whitelisted. xD
Florian Schmitz
As I said before, that's not enough ;)
Regards,
Flo
Fry, Joseph
Every user is able to provide a list of 7 "verification words"... These words corrispond to each day of the week.
When a user submits via email, they must put todays "verification word" in the subject line (overlap the days by an hour or so for late night users). If they provide the correct word, the email is posted... If they provide the incorrect word, they are blacklisted for the rest of that day. They can un-blacklist themselves via the web interface. And they can change the words of the day at anytime.
Also, I would suggest that the email submission feature only remain active if the user submits at least one email per 10 day period... Then can always re-activate it via the web interface. This way, only the users that take advantage of the feature, and will notice if someone has hijacked their account, will be able to submit via email.
As I see it, email submission is a convienance for active users... Not for an occasional submitter... My ideas, while odd, will not greatly inconvienance the target user... But make it much more difficult for spammers to spam the system via email.
Joe
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.472 / Virus Database: 269.8.7/830 - Release
> Date: 6/3/2007 12:47 PM
>
>
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.472 / Virus Database: 269.8.11/836 - Release Date: 6/6/2007 1:10 PM
Cristian Rodriguez
>
> I've got a bit of a hack that would work for verification... See if you can follow.
First, this feature has been dropped.
>
> Every user is able to provide a list of 7 "verification words"... These words corrispond to >each day of the week.
"verification words" are just like passwords, if users has internet
connection they can already submit a tasks visiting the BTS webpage.
also this idea of verification words does not really work, rememeber,
email is transmitted in plain text, and passes by many hosts..also
this "verification words" has to be public for the users to know..
hence publsihed via the internet or any other insecure channel..
We have really tried to find a good solution, no avail. your idea is
just one of possible hacks.
tig...@aspid.com
totally decide that you will not be implementing it.
It would be very nice to be able to reply to comments using email (and
generate comments). I don't care about opening new bug reports via
email.
I do not care about security and spam. We have RBL installed and thus
non-existent spam. For those who do care about security and spam this
feature could be made optional.
Florian Schmitz
> non-existent spam. For those who do care about security and spam this
> feature could be made optional.
It's not that simple. I'd say that most of our users *do* care about
security, and if we implement it only for those who don't care, then
we have done a lot of effort for only a very few users. Our current
development team size won't allow such an approach.
Regards,
Flo
Cristian Rodriguez
> I do not care about security and spam. We have RBL installed and thus
> non-existent spam. For those who do care about security and spam this
> feature could be made optional.
It would not work properly anyway, spam is a seriuos issue, is such a
problem that even though we have not recieved reports about bots
spamming 0.9.9.x releases, we have added(contrary to our wishes I must
say) captcha support to avoid possible future web spam attacks.
thinking about a feature that reads emails directly from an inbox
without thinking about spam is just pure insanity.