Format=flowed

[Mike Easter]

format=flowed a la RFC 2646 and 3676 is allegedly supported in Tb/SM/gecko

http://kb.mozillazine.org/Plain_text_e-mail_(Thunderbird)#Flowed_format
By default, support for flowed plain-text format is enabled.

There have been a great many bug reports filed and subsequently
integrated to be found in searches in the bug reports on strings f=f or
format=flowed, most particularly 456053 for your attention
https://bugzilla.mozilla.org/show_bug.cgi?id=456053 Quoted text
receives hard line breaks in f=f plain-text replies

That report says that what was previously working in Tb 2.0.0.x and I
presume gecko 1.9.1 and SM 2.0 is no longer working ever since the
numerous releases of gecko, Tb, and SM thereafter up to and including
the present.

3 years ago, in 2008 Sep Andriy Tkachuk posted a patch in bug 456053
which has been shown to work to solve the problem way back then and also
as recently as the newest Tb/SM.

I don't understand why so many versions of gecko, Tb, and SM have been
released with the partially broken f=f regression problem unfixed when
there is a patch which must surely elucidate the nature of the problem
to the gecko/Tb moz developers.

Is there something that can be done about this before the next f=f
partially broken release of gecko/Tb/SM? If not, why not?


--
Mike Easter

[Blake Winton]

On 11-09-26 18:46 , Mike Easter wrote:
> 3 years ago, in 2008 Sep Andriy Tkachuk posted a patch in bug 456053
> which has been shown to work to solve the problem way back then and also
> as recently as the newest Tb/SM.
>
> I don't understand why so many versions of gecko, Tb, and SM have been
> released with the partially broken f=f regression problem unfixed when
> there is a patch which must surely elucidate the nature of the problem
> to the gecko/Tb moz developers.

The problem is that that's a surprisingly tricky bit of code (as you can
tell because of the fact that this was regressed in the first place),
and while that patch might help show the source of this particular
problem, it doesn't help find the other problems that that patch might
cause. And for that reason, any patch that fixes this will need a lot
of automated tests.

> Is there something that can be done about this before the next f=f
> partially broken release of gecko/Tb/SM? If not, why not?

Asking someone to fix it would help. Writing down the big list of
inputs and outputs for the various formatting modes would help. Turning
that big list into actual automated tests would be more helpful.
Posting to this newsgroup also helped, since now I'm aware of the bug,
even if I don't have any idea how many people it impacts. :)

On the down side, the rapid release schedule means that it's already too
late for the next version of Thunderbird. On the up side, there will be
a new version every six weeks, so once the fix gets it, it should be
available much quicker than it would otherwise.

Later,
Blake.

[Mike Easter]

Blake Winton wrote:

> And for that reason, any patch that fixes this will need a lot
> of automated tests.

I don't believe that 'automation' of some things is a practical idea,
but then perhaps I have a limited imagination about automating something.

As an example of the 'automation' problem; when I am composing/editng a
message with the currently broken versions, and that composition
contains quoted text which was previously properly f=f, the
*composition* shows the presence of the trailing spaces in the quoted
text...

... but, when the message is _posted_, the trailing spaces for the
quoted text are/ have been/ stripped away, but the trailing spaces for
the new/original text are not/ have not been/ stripped away.

How does one devise a 'series of automated tests' to detect something
like that working or not working. Eyeballs work, if employed in the
correct way.

It seems to me that it - this problem and the patch solution - just
keeps getting pushed aside. It also seems that if a gecko problem
doesn't relate to html email, some people don't think it is worth
bothering with, that plaintext and news have fallen off the development
interest list.

--
Mike Easter

[Blake Winton]

On 11-09-26 19:37 , Mike Easter wrote:
>> And for that reason, any patch that fixes this will need a lot of
>> automated tests.

> As an example of the 'automation' problem; when I am composing/editng a
> message with the currently broken versions, and that composition
> contains quoted text which was previously properly f=f, the
> *composition* shows the presence of the trailing spaces in the quoted
> text...
>
> ... but, when the message is _posted_, the trailing spaces for the
> quoted text are/ have been/ stripped away, but the trailing spaces for
> the new/original text are not/ have not been/ stripped away.
>
> How does one devise a 'series of automated tests' to detect something
> like that working or not working. Eyeballs work, if employed in the
> correct way.

As was mentioned previously, you create a set of inputs, and the outputs
that should be produced, and then we write code to make sure we're doing
that. In this case, as I understand it, the input is some lines that
start with ">" and end with " ", and the output is lines that start with
">" and end with " ". :)

If you aren't up to creating the automated tests (and that's perfectly
reasonable! Not everyone should need to be an expert in everything),
then just creating the list of inputs and outputs for the various
formats would still be enormously useful. I mention it because that is
a task I believe you can do, since you seem to have spent a fair bit of
time learning about f=f, and how it should work.

> It seems to me that it - this problem and the patch solution - just
> keeps getting pushed aside.

I can see why you would feel that way, but while there is a small change
that may fix the problem, we need to know that it doesn't cause more
problems. I would imagine that you would be sympathetic to Mozilla's
position here, since it was the lack of tests which allowed this
regression to happen in the first place.

> It also seems that if a gecko problem doesn't relate to html email,
> some people don't think it is worth bothering with, that plaintext
> and news have fallen off the development interest list.

Perhaps some people do, but for me, I'm not a fan of HTML mail, and use
plaintext for the vast majority of my conversations. Heck, I'm even
using the newsgroups to send this message, instead of the mailing-list
equivalents. :)

Thanks,
Blake.

[Mike Easter]

Blake Winton wrote:
> Mike Easter wrote:

>> 3 years ago, in 2008 Sep Andriy Tkachuk posted a patch in bug 456053

> Posting to this newsgroup also helped, since now I'm aware of the

> bug, even if I don't have any idea how many people it impacts. :)

I forgot to say thank you for your attention. It is appreciated.

--
Mike Easter

[Mike Easter]

Blake Winton wrote:

> As was mentioned previously, you create a set of inputs, and the outputs
> that should be produced, and then we write code to make sure we're doing
> that. In this case, as I understand it, the input is some lines that
> start with ">" and end with " ", and the output is lines that start with
> ">" and end with " ". :)

But in this case, we know a patch that fixes the problem. The unsolvable
hangup/ bottleneck/ chokepoint/ seems to be about a concern that the fix
will break some unknown and unforeseen something in miles of code.

That is, we don't actually need an input/output scheme to diagnose that
the problem is fixed, but instead we need to reassure someone that an
applied fix has not broken some unknown mysterious thing that we have no
clue as to the input/output scheme to find.


--
Mike Easter

[Jim]

On 09/26/2011 07:11 PM, Mike Easter wrote:
> That is, we don't actually need an input/output scheme to diagnose that
> the problem is fixed, but instead we need to reassure someone that an
> applied fix has not broken some unknown mysterious thing that we have no
> clue as to the input/output scheme to find.

Part of the reason Mozilla requires tests is to ensure that fixes don't
regress in the future, e.g. when someone refactors the affected code two
years down the line.

It would also be instructive to see where this code is used in Mozilla
so that other consumers of this code (if there indeed are any) won't be
negatively affected by this change. It's certainly possible to figure
out what this code is used for and from there identify possible use
cases that this change might break.

More generally, it's a good policy to ensure that any patch leaves the
code in a better state (cleanliness, tests, etc) than it was before the
patch.

- Jim

[Axel Hecht]

Just to pile on this thread, my personal pet bug with everybody else is
that they're sending mails that look like f=f, but are not. Read, zimbra
and android. I wonder, can we just hack around them? As they're
apparently not able to fix those bugs :-/,
https://bugzilla.zimbra.com/show_bug.cgi?id=22951 is at least out there
for zimbra. Can't find an android bug, though of course there's still
annoying http://code.google.com/p/android/issues/detail?id=2629, but
unrelated.

Axel

[Matej Cepl]

Dne 27.9.2011 00:46, Mike Easter napsal(a):

> There have been a great many bug reports filed and subsequently
> integrated to be found in searches in the bug reports on strings f=f or
> format=flowed, most particularly 456053 for your attention
> https://bugzilla.mozilla.org/show_bug.cgi?id=456053 Quoted text receives
> hard line breaks in f=f plain-text replies

Just to mention https://bugzilla.mozilla.org/show_bug.cgi?id=571502 here.

Matěj

[Mike Easter]

There are 115 bugs currently found by searching format=flowed and 11 by
searching f=f. I didn't count how many are in both groups.

Naturally I think the one I'm talking about is far and away the most
important which should be fixed first :-)

I would also like to see the rewrap function result in f=f instead of
resulting in the rewrapped content having its trailing spaces stripped
or 'de-f=f/ed' and/or there being a similar function to rewrap being
'make into f=f' to convert existing non-f=f into f=f. Currently it has
to be done manually.

Naturally doing it manually only works in Tb2 because all other v/s
strip the trailing spaces in quoted material whenever they are there.


--
Mike Easter

[Jonathan Protzenko]

While we're talking about f=f, the quick reply feature of Thunderbird
Conversations tries to send f=f by default, and also keeps f=f in the
quoted text. I'd be happy if you could give it a try and tell me if
there's something I'm doing wrong or not.

Cheers,

jonathan

[Mike Easter]

Jonathan Protzenko wrote:
> While we're talking about f=f, the quick reply feature of Thunderbird
> Conversations tries to send f=f by default, and also keeps f=f in the
> quoted text. I'd be happy if you could give it a try and tell me if
> there's something I'm doing wrong or not.

I haven't tried it yet, I'm still working on getting a preliminary grip
on what it does.

It is compared to gmail conversations, which says this:

http://mail.google.com/support/bin/answer.py?answer=5900 Gmail groups
all replies with their original message, creating a single conversation
or thread. In other email systems, responses appear as separate messages
in your inbox,

Of course, Tbird can arrange mail or news messages grouped by their
Reference number by default, so I'm still working on understanding what
is up with Tbird conversations, but I see that it also has these
'unrelated' functions...

In addition to threading enhancements, Thunderbird Conversations adds
many features to Thunderbird, obsoleting several other addons -
CompactHeader - RSS Linkify Subject - Slideshow - Displays a slideshow
of picture attachments.

FLOSS says all this
http://en.flossmanuals.net/thunderbird/thunderbird-conversations/

It looks like a very ambitious project/add-on.


--
Mike Easter

[Mike Easter]

Jonathan Protzenko wrote:
> While we're talking about f=f, the quick reply feature of Thunderbird
> Conversations tries to send f=f by default, and also keeps f=f in the
> quoted text. I'd be happy if you could give it a try and tell me if
> there's something I'm doing wrong or not.

Your headers:

User-Agent: Mozilla/5.0 (X11; Linux x86_64;
rv:9.0a1) Gecko/20110912 Thunderbird/9.0a1
Content-Type: text/plain; charset=UTF-8; format=flowed
X-Mailman-Version: 2.1.14

For what it is worth, the message you posted here with your Tb9.0a1 and
likely your add-on shows that its quoted text such as that of mine below
f=f compliantly maintained its trailing spaces and flowability.

I'm not 100% sure there isn't some kind of difference between the
behavior of trailing space stripping for mailing list messages here
compared to 'pure' nntp news messages, but I haven't researched that yet.

>> I would also like to see the rewrap function result in f=f instead of
>> resulting in the rewrapped content having its trailing spaces stripped
>> or 'de-f=f/ed' and/or there being a similar function to rewrap being
>> 'make into f=f' to convert existing non-f=f into f=f. Currently it has
>> to be done manually.

--
Mike Easter