[Dillo-dev] image overlaps text

[Alexander Voigt]

I noticed, that in current Dillo (tip) there are cases where images
overlap surrounding text, making in unreadable. Here is an example page:

https://netzpolitik.org/2016/massenbeschlagnahme-von-mobiltelefonen-in-leipzig-fast-alle-verfahren-eingestellt/

Best regards,
Alex

_______________________________________________
Dillo-dev mailing list
Dill...@dillo.org
http://lists.dillo.org/cgi-bin/mailman/listinfo/dillo-dev

[Sebastian Geerken]

On Fr, Apr 29, 2016, Alexander Voigt wrote:
> I noticed, that in current Dillo (tip) there are cases where images
> overlap surrounding text, making in unreadable. Here is an example page:
>
> https://netzpolitik.org/2016/massenbeschlagnahme-von-mobiltelefonen-in-leipzig-fast-alle-verfahren-eingestellt/

Yes, I noticed this already several times on netzpolitik.org.
Unfortunately, these cases are not always reproducable. The highest
percentage I've found so far on this page:

http://www.nachdenkseiten.de/?p=33180

It's at the top on my todo list.

Sebastian

[Nick Warne]

On 29/04/16 16:46, Alexander Voigt wrote:
> I noticed, that in current Dillo (tip) there are cases where images
> overlap surrounding text, making in unreadable. Here is an example page:
>
> https://netzpolitik.org/2016/massenbeschlagnahme-von-mobiltelefonen-in-leipzig-fast-alle-verfahren-eingestellt/

Strange, renders here OK (latest hg pull).

Nick
--
Gosh that takes me back... or is it forward? That's the trouble with
time travel, you never can tell."
-- Doctor Who "Androids of Tara"

[Sebastian Geerken]

On Fri, Apr 29, 2016, Nick Warne wrote:
>
>
> On 29/04/16 16:46, Alexander Voigt wrote:
> >I noticed, that in current Dillo (tip) there are cases where images
> >overlap surrounding text, making in unreadable. Here is an example page:
> >
> > https://netzpolitik.org/2016/massenbeschlagnahme-von-mobiltelefonen-in-leipzig-fast-alle-verfahren-eingestellt/
>
> Strange, renders here OK (latest hg pull).

As I wrote, this is hard to reproduce.

Sebastian

[Jorge Arellano Cid]

Hi Alexander,

On Fri, Apr 29, 2016 at 05:46:20PM +0200, Alexander Voigt wrote:
> I noticed, that in current Dillo (tip) there are cases where images
> overlap surrounding text, making in unreadable. Here is an example page:
>
> https://netzpolitik.org/2016/massenbeschlagnahme-von-mobiltelefonen-in-leipzig-fast-alle-verfahren-eingestellt/

It works here.

Does going Back and Forward fix the problem?
(I've noticed that several times with the old and newest dillo).

--
Cheers
Jorge.-

[higuita]

On Fri, 29 Apr 2016 13:41:16 -0300, Jorge Arellano Cid <jc...@dillo.org>
wrote:

> Does going Back and Forward fix the problem?

Yes, it solves the problem. Reload also solves this.

For me, i can reproduce this ~70% of the attempts.
it seems just a race drawing the page and getting the resources
from the internet.

Best regards
higuita
--
Naturally the common people don't want war... but after all it is the
leaders of a country who determine the policy, and it is always a
simple matter to drag the people along, whether it is a democracy, or
a fascist dictatorship, or a parliament, or a communist dictatorship.
Voice or no voice, the people can always be brought to the bidding of
the leaders. That is easy. All you have to do is tell them they are
being attacked, and denounce the pacifists for lack of patriotism and
exposing the country to danger. It works the same in every country.
-- Hermann Goering, Nazi and war criminal, 1883-1946

[Jorge Arellano Cid]

Yes.

(I can't reproduce it with your best URL :)

Fortunately, today I managed to find&reduce a case that works all the time
here. Attached it goes. Just resize horizontally to see the overlap.

I hope it works there!

--
Cheers
Jorge.-

PD: subliminal message in the example. ;)

[Jorge Arellano Cid]

On Fri, Apr 29, 2016 at 06:26:50PM -0300, Jorge Arellano Cid wrote:
> [...]

>
> Fortunately, today I managed to find&reduce a case that works all the time
> here. Attached it goes. Just resize horizontally to see the overlap.
>
> I hope it works there!

Ooops, I forgot to attach the image.
Here it goes.



--
Cheers
Jorge.-

[Jorge Arellano Cid]

On Fri, Apr 29, 2016 at 10:11:37PM +0100, higuita wrote:
> On Fri, 29 Apr 2016 13:41:16 -0300, Jorge Arellano Cid <jc...@dillo.org>
> wrote:
> > Does going Back and Forward fix the problem?
>
> Yes, it solves the problem. Reload also solves this.
>
> For me, i can reproduce this ~70% of the attempts.
> it seems just a race drawing the page and getting the resources
> from the internet.

That's my point. The rendering algorithm seems to make a decision on
incomplete data (sometimes depending on available data) and, also sometimes,
it can't recover when all data is got.

Now, if you feed it the whole of the data, it seldom fails.

I hope the reduced test case fails because it gets to that point in
the algorithm and not because it always fails.

[Sebastian Geerken]

Thanks, Jorge, for the perfect testcase. So far, I've tracked down the
problem, but I'm not yet sure how to fix it. Hopefully, the fix will
also solve the other issues with overlapping floats.

Sebastian

[Jorge Arellano Cid]

On Sat, Apr 30, 2016 at 01:31:48PM +0200, Sebastian Geerken wrote:
> On Fr, Apr 29, 2016, Jorge Arellano Cid wrote:
> > On Fri, Apr 29, 2016 at 06:41:32PM +0200, Sebastian Geerken wrote:
> > > On Fri, Apr 29, 2016, Nick Warne wrote:
> > > >
> > > >
> > > > On 29/04/16 16:46, Alexander Voigt wrote:
> > > > >I noticed, that in current Dillo (tip) there are cases where images
> > > > >overlap surrounding text, making in unreadable. Here is an example page:
> > > > >
> > > > > https://netzpolitik.org/2016/massenbeschlagnahme-von-mobiltelefonen-in-leipzig-fast-alle-verfahren-eingestellt/
> > > >
> > > > Strange, renders here OK (latest hg pull).
> > >
> > > As I wrote, this is hard to reproduce.
> >
> > Yes.
> >
> > (I can't reproduce it with your best URL :)
> >
> > Fortunately, today I managed to find&reduce a case that works all the time
> > here. Attached it goes. Just resize horizontally to see the overlap.
> >
> > I hope it works there!
>
> Thanks, Jorge, for the perfect testcase. So far, I've tracked down the
> problem,

Great! :) :)

> but I'm not yet sure how to fix it. Hopefully, the fix will
> also solve the other issues with overlapping floats.

Yes, I also hope in the same direction!

FWIW, yesternight, without knowing whether the example would
tackle it, I came with a different approach to testing (just in case).
Do you remember our slow.cgi testing program? It can be made to serve
adjustable chunks of data with adjustable delays, this way:

slow.cgi?2048:1:/tmp/testcase.html

for 2048 bytes in 1 second (for instance).


This could help to reproduce the bug in the network dependent cases
by finding the combination that triggers it. Just let me know if you
need it as plan B.

HTH.


--
Cheers
Jorge.-

[Sebastian Geerken]

Hi Alexander and others,

On Mo, Mai 09, 2016, Alexander Voigt wrote:
> Dear Sebastian,
>
> I think one of your recent commits (probably 4543:3cb86de1861a)
> introduced a problem with hyphenation. See for example the rendering
> of
>
> debian.org

This commit fixes a problem which has hidden another problem which now
shows up. I'm working on it.

If you want a halfway usable recent version, stick to revision 4542.

Sebastian

[Jorge Arellano Cid]

Hi Sebastian,

The attached tetscase is a slight variation of the previous one.

Comparing rendering behaviour with Firefox here shows some interesting
facts/bugs/design-decisions:

1.- FF links the olive div's width to viewport width only.
Dillo links it to viewport and its own child width.
2.- When splitting the first sentence (in olive div), by narrowing the
viewport, text in the teal div is also split!
3.- After 2.-, when widening the viewport, text in olive widens, but text
in teal not! (reload, does the trick).

As usual, since a few years, following FF in these corner cases is the
safe chioce.

I'd like to dig into it with a view to learn more about rendering floats.
Would you please explain a bit what you've found, which of the above points
you'd recommend me to start with, and spill some suggestions on what parts
of the code/design are most probably responsible for it.


--
Cheers
Jorge.-

[Sebastian Geerken]

Hi Jorge,

On Fr, Mai 20, 2016, Jorge Arellano Cid wrote:
> The attached tetscase is a slight variation of the previous one.
>
> Comparing rendering behaviour with Firefox here shows some interesting
> facts/bugs/design-decisions:
>
> 1.- FF links the olive div's width to viewport width only.
> Dillo links it to viewport and its own child width.

Dillo has a rather (over-)simple definition as what do draw with the
background color. I regard this as a low-priority issue, which can be
solved after the next release.

> 2.- When splitting the first sentence (in olive div), by narrowing the
> viewport, text in the teal div is also split!
> 3.- After 2.-, when widening the viewport, text in olive widens, but text
> in teal not! (reload, does the trick).

I'll write more on this later.

Sebastian

[Sebastian Geerken]

I've looked a bit at this, and it seems to look like another case I
was working on, so it may be best to fix the latter, and hopefully the
former is fixed, too.

I'll describe my analysis so far, which may give you a better
understanding of the code.

The test case is attached. Notice that there is a float within a
float, with no sizes explicitly defined. Dillo (and also FF) will use
the maximal width as size, as long as it fits into the available width
(which is the case here).

It should look like (and looks like in FF):

abcdefghijkl float
mnopqrstuvwx

It actually is drawn by dillo like this:

float
abcdefghijkl mnopqrstuvwx

There are two problems:

1. Why is the word "abcdefghijkl" in the second line?
2. Why is the float positioned too far right?

Let's examine problem 1. I'm using RTFL (<http://home.gna.org/rtfl/>;
use the latest version from SVN, not the release, and make sure you
have the graphviz library installed). To enable the RTFL messages, run
the dillo configure script with "--enable-rtfl".

Run:

$ src/dillo n33180d.html | rtfl-objview -OM -a "*" -A draw

(I'm also adding "-v 'emacsclient -n %n %p'", and you may use the
script rtfl-filter-out-classes.)

The relevant textblock is the lower right one (some guessing, and
examine the attributes). Go to the attribute "lines/0/firstWord",
select the last (which is the relevant) assignment and press Ctrl+A.
This way, we can focus on what has happened before the first line is
eventually created.

Then examine "words/0/badnessAndPenalty" (again last entry); you'll
see "... vs. -31" here, which means that when calculating the badness
(used for line breaking), an ideal width of -31 was assumed, which is
obviously an icorrect value.

The "stack trace" function (Ctrl+T) leads to "accumulateWordData", in
which "calcLineBreak" is used to calculate this value. Press Ctrl+A
again to hide everything behind the last command within
"calcLineBreak" ("leave").

What values go into the return value?

- lineBreakWidth = 194, which looks plausible. (You can examine this
further: this is calculates by "getAvailWidth", which actually
returns the maximal width.)
- leftInnerPadding = 0 (ok)
- leftBorder = 0 (ok)
- rightBorder = 225, which is obviously too large.

"rightBorder" goes back to the attribute "newLineRightBorder", which
is set in "calcBorders". If you look at the messages here, you'll
notice that the result of "getGeneratorRest" is -194, which is too
small (it should be 0 here). Further examination shows:

- getGeneratorWidth (this widget) = 194: ok
- getGeneratorWidth (container) = 0: too small

Examining the latter shows than the width of the child (194) is not
yet regarded here.

I'm not yet sure how to fix this best; most probably a size
recalculation should be queued under certain circumstances.

Sebastian

[Jorge Arellano Cid]

Hi Sebastian,

On Sat, May 21, 2016 at 05:21:07PM +0200, Sebastian Geerken wrote:
> On Fri, May 20, 2016, Sebastian Geerken wrote:
> > Hi Jorge,
> >
> > On Fr, Mai 20, 2016, Jorge Arellano Cid wrote:
> > > The attached tetscase is a slight variation of the previous one.
> > >
> > > Comparing rendering behaviour with Firefox here shows some interesting
> > > facts/bugs/design-decisions:
> > >
> > > 1.- FF links the olive div's width to viewport width only.
> > > Dillo links it to viewport and its own child width.
> >
> > Dillo has a rather (over-)simple definition as what do draw with the
> > background color. I regard this as a low-priority issue, which can be
> > solved after the next release.

I was thinking more in the line of the outmost div's width side-effects on
its children rendering.

Anyway, this may be a different bug, read on...

> > > 2.- When splitting the first sentence (in olive div), by narrowing the
> > > viewport, text in the teal div is also split!
> > > 3.- After 2.-, when widening the viewport, text in olive widens, but text
> > > in teal not! (reload, does the trick).
> >
> > I'll write more on this later.
>
> I've looked a bit at this, and it seems to look like another case I
> was working on, so it may be best to fix the latter, and hopefully the
> former is fixed, too.
>
> I'll describe my analysis so far, which may give you a better
> understanding of the code.

Good news on this one! (i.e. the overlapped image/text in o4.html).

After reading your description of the other problem (n33180d.html),
especially the part where you wonder "why", it became clear that the code
was meant to handle it, so I turned to try to find what may possible be
going astray there.

Short story: I have a working patch.

It solves the overlap (not 1.- 2.- and 3.-). It seems there're
a few different bugs here.

It's not as clean as I'd like, but it already is a proof of concept.
I'll try to understand the issues better and clean it up.

AFAIS it can have an effect on general rendering of floats. Currently,
I dont know if it's related to n33180d.html.

I'll try to share my findings this monday (unless you'd like to
look at it "as is" now).


--
Cheers
Jorge.-

[Jorge Arellano Cid]

Hi Sebastian,

On Sun, May 22, 2016 at 11:29:34AM -0400, Jorge Arellano Cid wrote:
> [...]

> Short story: I have a working patch.
>
> It solves the overlap (not 1.- 2.- and 3.-). It seems there're
> a few different bugs here.
>
> It's not as clean as I'd like, but it already is a proof of concept.
> I'll try to understand the issues better and clean it up.
>
> AFAIS it can have an effect on general rendering of floats. Currently,
> I dont know if it's related to n33180d.html.
>
> I'll try to share my findings this monday (unless you'd like to
> look at it "as is" now).

OK, here is what I've found so far. Two patches attached plus some
testcases.

The first patch is the fix for o4.html. The second one also tackles
o5.html and o6.html (progressive patches).

AFAIU the gist of the problem is in getGeneratorX() not returning
the correct value on some cases. It is called in four places inside
ooffloatsmgr.cc so other functions are sometimes making calculations
with a wrong x value.

The second problem is getFloatsSize() returning the widest among
a list of floats instead of a cumulative sum of widths inside a
cretain range (the second patch just adds to ilustrate the point).

Some tests:
o4.html o5.html o6.html some-sites [1] [2]
First patch: OK Wrong Wrong OK (1) (2)
Second patch: OK OK OK +/- (3) (4)


(1), (2): No images over text. As it was before.
(3) : Some images overlap text.
(4) : Different background color! some rendering diffs.


[1] http://www.pravdareport.com/}
[2] http://tinyurl.com/j2uzps9

It looks like fixing getGeneratorX() paves the way for fixing hidden
bugs. Also fixing getFloatsSize() can shed light for a patch for
n33180d.html.

Please get back to me when you've processed this so we can coordinate.

HTH

--
Cheers
Jorge.-

[Sebastian Geerken]

On Sat, May 21, 2016, Sebastian Geerken wrote:
> [test case n33180d.html ...]

>
> "rightBorder" goes back to the attribute "newLineRightBorder", which
> is set in "calcBorders". If you look at the messages here, you'll
> notice that the result of "getGeneratorRest" is -194, which is too
> small (it should be 0 here). Further examination shows:
>
> - getGeneratorWidth (this widget) = 194: ok
> - getGeneratorWidth (container) = 0: too small
>
> Examining the latter shows than the width of the child (194) is not
> yet regarded here.
>
> I'm not yet sure how to fix this best; most probably a size
> recalculation should be queued under certain circumstances.

At least this one has been fixed, see recent change.

Jorge, I'll look at your test cases now.

Sebastian