Lack of replies

[Steven Robbins]

Antonio Russo wrote:

> As someone who would like to participate more in the development of Debian,
> my personal experience is that making contributions is like dropping a
> message in a bottle into the sea. It feels like a complete crap-shot
> whether I'll even receive a comment on any code contribution (including
> debian-devel RFS, salsa MR, or BTS patch).

For salsa and BTS, one reason for lack of reply can be that the maintainer is
never notified. I get surprised sometimes that salsa has merge requests
waiting for me. I suspect that either the email notification is broken or off
by default and I never saw instructions to enable it.

In the case of the BTS: it used to email me but that broke a couple years ago
and apparently it is hard to fix. So currently a class of us don't get email
from any bug reports.

My suggestion, therefore, is: if the patch is important to you, do also CC
directly the maintainer address.

-Steve

[Andrey Rakhmatullin]

On Fri, Dec 29, 2023 at 02:18:41PM -0600, Steven Robbins wrote:
> Antonio Russo wrote:
>
> > As someone who would like to participate more in the development of Debian,
> > my personal experience is that making contributions is like dropping a
> > message in a bottle into the sea. It feels like a complete crap-shot
> > whether I'll even receive a comment on any code contribution (including
> > debian-devel RFS, salsa MR, or BTS patch).
>
> For salsa and BTS, one reason for lack of reply can be that the maintainer is
> never notified. I get surprised sometimes that salsa has merge requests
> waiting for me. I suspect that either the email notification is broken or off
> by default and I never saw instructions to enable it.

It is off by default unless that was changed recently-ish.

> In the case of the BTS: it used to email me but that broke a couple years ago
> and apparently it is hard to fix. So currently a class of us don't get email
> from any bug reports.

What is that class?

[Steven Robbins]

On Friday, December 29, 2023 2:18:41 P.M. CST Steven Robbins wrote:


> In the case of the BTS: it used to email me but that broke a couple years
> ago and apparently it is hard to fix. So currently a class of us don't get
> email from any bug reports.

Andrey Rakhmatullin asked "what is that class?".

It's unclear to me. From the last discussion [1], it seems like it should be
everyone. Maybe it is everyone whose mail infrastructure looks at DMARC [2]?

I think Sébastien Noel's 2021 note to the bug report is germane to this
discussion:

"""
Same observation here:
DMARC aggregate reports notifies me that emails sends to the BTS
are not delivered to the final recipient.

I should not be surprised anymore if bugreports are left un-answered,
maintainers are simply not getting notification...

Since the last comment of this bug, 4 months have passed and no
reaction.
I suspect that nobody recieved the last email, and my guts tell me
that i'm writing to /dev/null rigth now :(

Without a working BTS, i'm wondering :
Is the project still interested by users' feedback ?
"""


[1] https://lists.debian.org/debian-devel/2022/09/msg00273.html
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=754809

[Scott Kitterman]

Alternatively, BTS users that are interested in others getting their emails might be better off posting from a domain that doesn't have a DMARC policy that's designed to be used for domains that send only transactional email (i.e. no human users). It's working fine for me.

Scott K

[Sam Hartman]

>>>>> "Scott" == Scott Kitterman <deb...@kitterman.com> writes:


Scott> Alternatively, BTS users that are interested in others
Scott> getting their emails might be better off posting from a
Scott> domain that doesn't have a DMARC policy that's designed to be
Scott> used for domains that send only transactional email (i.e. no
Scott> human users). It's working fine for me.

Would including a warning about the domain you are sending from be more
tractable as an improvement than SRS-style header rewriting?
(This directed more at people who know the BTS code).

At least people could be warned that because of the domain they send
from their mail might not get through.

--Sam

[Michael Neuffer]

By far most users in all likelihood do not know that something like even DMARC exists.

Do not discount them just because you know and can act accordingly.

Cheers
    Mike

[Scott Kitterman]

I agree. I do, however, think it's at least as reasonable to assign blame for DMARC breaking all sorts of traditional email usage on users that use domains with restrictive DMARC policies as it is to blame the BTS.

If someone wants to change the Debian BTS to be more DMARC friendly, then they should probably work with the BTS maintainers to implement it. Inferring that the lack of progress is indicative of not wanting feedback is nonsense.

In the meantime, users who want to successfully interact with the BTS have options (use a different email provider). BTS works fine with non-broken email systems.

Scott K

[Scott Kitterman]

My guess is that such a warning email (which is the only way we'd have to do it) would also cause a lot of complaints. Ultimately, I think we (for some value of we that doesn't include me doing the work) will need to have the BTS send all emails from bugs.debian.org role addresses and not use the sender's email in From anymore.

Scott K

[Colin Watson]

On Wed, Jan 03, 2024 at 05:10:43PM +0000, Scott Kitterman wrote:
> My guess is that such a warning email (which is the only way we'd have
> to do it) would also cause a lot of complaints. Ultimately, I think
> we (for some value of we that doesn't include me doing the work) will
> need to have the BTS send all emails from bugs.debian.org role
> addresses and not use the sender's email in From anymore.

Yeah, this is what we ended up doing in Launchpad for similar reasons in
2020: mail from bug #NNNNNNN resulting from an action by a user whose
display name is "Some User" is sent with "From: Some User
<NNN...@bugs.launchpad.net>". The switch was a bit nerve-wracking, but
it largely seems to be working OK there and I think is basically the
only way to make things work reliably in the modern email environment.

That said, debbugs has much higher expectations of everything being done
through email than Launchpad does, so it'd need some care to make sure
that e.g. From/Reply-To are exactly right in all cases, and it's quite
likely that people would be somewhat discombobulated by the change. It
would make it harder to take a BTS conversation to private email, for
instance (but perhaps that's a good thing since new users often end up
doing this by accident?), and we'd have to think about how it interacted
with things like X-Debbugs-Cc to mailing lists.

[Disclaimer: I no longer work on Launchpad, and although I should still
have permissions to work on debbugs it's been a long time since I
actually did so. Maybe later this year ...]

--
Colin Watson (he/him) [cjwa...@debian.org]

[Daniel Gröber]

Hi Scott,

On Wed, Jan 03, 2024 at 05:10:43PM +0000, Scott Kitterman wrote:

> >At least people could be warned that because of the domain they send
> >from their mail might not get through.
> >
> My guess is that such a warning email (which is the only way we'd have to
> do it) would also cause a lot of complaints.

> I think we [...] will need to have the BTS send all emails from

> bugs.debian.org role addresses and not use the sender's email in From
> anymore.

Just to make sure I understand the constraints: we can determine at sending
time whether a particular domain is going to cause trouble or not, right?
If so could this rewrite scheme be applied only for recipients where it's
absolutely necessary?

That way DDs, who are likeley to care more about their BTS email workflow
than the average user, don't have to deal with the negative consequences of
the address rewriting if they're already behind a polite mailserver.

Further if this discrimination is possible I wonder if it might not also be
possible to accomodate the subset of BTS users who are behind broken mail
providers but use sensible mail clients (mutt and such).

Specifically I think when you embedd an message/rfc822 part mutt allows me
to autoview the message inline, see the (pretty set) of headers, and reply
to this message instead of the "envelope".

So when BTS sees a broken domain it could generate the usual message with
address rewriting applied, but also attach in an multipart/alternative the
untouched version for this set of users to use.

Not sure that all works out, just a crazy idea,
--Daniel

[Scott Kitterman]

That's consistent with things I've seen proposed and implemented for mitigation of DMARC issues with mailing lists. I don't know if the multipart solution has ever been implemented. From/Mail From definitely have been.

If I were designing it, I would lean heavily on Colin Watson's experience with DMARC mitigation in Launchpad. I've done some consulting work in this area, but I have never implemented it.

Scott K

[Jeremy Stanley]

On 2024-01-04 15:54:28 +0100 (+0100), Daniel Gröber wrote:
[...]

> Just to make sure I understand the constraints: we can determine
> at sending time whether a particular domain is going to cause
> trouble or not, right? If so could this rewrite scheme be applied
> only for recipients where it's absolutely necessary?

[...]

Unfortunately no. It *used* to be a popular assumption that you
could look at the published DKIM/DMARC policies for a given address
and determine whether you can safely put them in the From header,
but that changed recently when Gmail decided to treat messages from
its own users more strictly than the policy it publishes for them in
DNS. And since Gmail does custom domain hosting too, you can't
simply limit the workaround to treating their well-known domain
specially. Given its popularity (near ubiquity) as a freemail
provider these days, telling users they'll have to get an address
somewhere else to interact with the BTS is unlikely to end well
either.
--
Jeremy Stanley

[Colin Watson]

On Thu, Jan 04, 2024 at 03:54:28PM +0100, Daniel Gröber wrote:
> On Wed, Jan 03, 2024 at 05:10:43PM +0000, Scott Kitterman wrote:
> > >At least people could be warned that because of the domain they send
> > >from their mail might not get through.
> > >
> > My guess is that such a warning email (which is the only way we'd have to
> > do it) would also cause a lot of complaints.
>
> > I think we [...] will need to have the BTS send all emails from
> > bugs.debian.org role addresses and not use the sender's email in From
> > anymore.
>
> Just to make sure I understand the constraints: we can determine at sending
> time whether a particular domain is going to cause trouble or not, right?
> If so could this rewrite scheme be applied only for recipients where it's
> absolutely necessary?

I haven't looked into this in a lot of detail, but my concern would be
that that would end up being flaky and confusing in practice. I think
people want the behaviour of the BTS to be easily predictable without
having to get an advanced degree in MTA debugging first.

[Scott Kitterman]

I agree that inconsistent behavior is concerning. IETF mailing lists selectively rewrite From based on domain DMARC and it seems to be mostly okay. It may be that the typical IETF participant knows more about email than the typical BTS user.

Personally, I worry more about the added complexity and maintainability. It's timely that this
Niklaus Wirth quote from 2008: A Brief History of Software Engineering by Niklaus Wirth showed up in my Mastodon feed yesterday:

“A primary effort must be education toward a sense of quality. Programmers must become engaged crusaders against home-made complexity. The cancerous growth of complexity is not a thing to be admired; it must be fought wherever possible.”

I think it's definitely possible to avoid this complexity, so we probably should.

Scott K

[Daniel Gröber]

Hi Jeremy,

On Thu, Jan 04, 2024 at 03:11:59PM +0000, Jeremy Stanley wrote:
> On 2024-01-04 15:54:28 +0100 (+0100), Daniel Gröber wrote:
> > could this rewrite scheme be applied only for recipients where it's
> > absolutely necessary?
>

> Unfortunately no. It *used* to be a popular assumption that you could

> look at the published DKIM/DMARC policies [...] but [...] Gmail decided

> to treat messages from its own users more strictly than the policy it
> publishes for them in DNS. And since Gmail does custom domain hosting
> too, you can't simply limit the workaround to treating their well-known
> domain specially. Given its popularity (near ubiquity) as a freemail
> provider these days,

Any good reason we cannot look at the MX domain (or in the worst case) ASN
associated with mailserver IP to special case particularly offensive
implementations such as this if looking at the DMARC policy works in the
average case?

> telling users they'll have to get an address somewhere else to interact
> with the BTS is unlikely to end well either.

That's certainly not something I'd advocate for. I want us to minimize the
PITA for the technically literate without sacrifising general usability.

--Daniel

[Jeremy Stanley]

On 2024-01-04 16:52:58 +0100 (+0100), Daniel Gröber wrote:
[...]

> Any good reason we cannot look at the MX domain (or in the worst case) ASN
> associated with mailserver IP to special case particularly offensive
> implementations such as this if looking at the DMARC policy works in the
> average case?

[...]

Unfortunately not. An example I know of is Red Hat's corporate
E-mail, they use a third-party filter (Mimecast) as their MX which
then forwards to their custom domain on Gmail.

As a mailing list admin in some popular open source communities,
I've become all too familiar with these challenges. It doesn't help
that the situation changes month-to-month, so what you decide today
may suddenly stop working any moment and you're back to square one.

The cynic in me says that the mass freemail providers slipped these
policy frameworks in as a sort of Trojan Horse, promising to fix
"the spam problem" when their real goal was poisoning the well we're
all drinking from and then setting themselves up as the only safe
supply of drinking water. They're all too happy to propose standards
when they want other operators to do something, but then willfully
ignore those same standards whenever it suits their purposes.
--
Jeremy Stanley

[Colin Watson]

On Thu, Jan 04, 2024 at 04:52:58PM +0100, Daniel Gröber wrote:
> That's certainly not something I'd advocate for. I want us to minimize the
> PITA for the technically literate without sacrifising general usability.

To be honest, I think that address rewriting might actually be part of
improving usability of the BTS - but it would have to be done carefully
and with an eye to various other features that have evolved as
workarounds for the current situation. It may be that some other
changes need to happen first.

The original design had a collection of addresses for each bug, all of
which were filed in the system itself, and some of which are also sent
on to others. As seen on https://www.debian.org/Bugs/Developer:

* n...@bugs.debian.org — such messages are also sent to the package
maintainer and forwarded to debian-bugs-dist, but not to the
submitter;
* nnn-su...@bugs.debian.org — these are also sent to the submitter
and forwarded to debian-bugs-dist, but not to the package maintainer;
* nnn-ma...@bugs.debian.org — these are only sent to the package
maintainer, not to the submitter or debian-bugs-dist;
* nnn-...@bugs.debian.org — these are only filed in the bug tracking
system (as are all the above), not sent to anyone else.

This has been a mostly functional but slightly problematic design for as
long as I've been involved with Debian. The classic problem is that
people email a followup to a bug to nnn@bugs and (without much thinking
about it) expect it to be seen by everyone who's interested in that bug.
But in fact this doesn't automatically go to the submitter, nor
necessarily to anyone else who's been involved in the discussion on the
bug.

In practice, if you're receiving the bug discussion by email, then you
can reply-all and it'll usually be more or less OK - but if the email
thread is at all non-linear then people may well end up being left out
by accident, people can easily forget to reply-all rather than just
reply, and it's not exactly obvious how to participate in this way if
you weren't already CCed. ("bts --mbox show nnn" exists and is OK for
experts, but it's not exactly obvious and probably only works with some
MUAs.)

At some point debbugs gained a "subscribe" feature: you can email
nnn-subscribe@bugs to subscribe to a bug, or nnn-unsubscribe@bugs to
unsubscribe, with a confirmation message in each case. So far so good,
though clunky. But submitters aren't auto-subscribed, and you can't
really tell who's subscribed so you have no way to know in advance if
your message is going to reach the right people. (The implementation is
also kind of weird: IIRC it's done via lists.debian.org, so even the BTS
itself doesn't really know who's going to get the message.) As a
result, while this helped with certain use cases, it didn't really solve
the problem above.

What I always thought would be a better model would be for each bug to
be a "nosy list" (the term comes from roundup, I think). That is, bugs
would have a list of addresses notified of changes, by default filing a
bug or sending a message to it would cause you to be added to the list,
and subscribing to or unsubscribing from any given bug would be easy.

Now, such a change would certainly require some people to adjust a bit,
and it would be easier if the BTS had an optional authenticated web
interface to allow you to (un)subscribe more easily. I'm not saying it
would be straightforward. But if things worked this way, then I think
rewriting addresses to nnn@bugs would ultimately be less controversial -
it would be the most convenient default, as the address that's most
likely to reach everyone you probably want it to reach.