Re: [Blink/SPv2] Need inputs for implementing css3-multicol as property nodes

[Chris Harrelson]

On Mon, Jan 18, 2016 at 1:22 AM Morten Stenshorne <mste...@opera.com> wrote:

Chris Harrelson <chri...@chromium.org> writes:

> 2. Fragmentation should not be allowed to break replaced (videos, iframes,
> images) or scrollable content.

Unfortunately, we're only supposed to AVOID breaking inside replaced
content and scrollables (and lines) (and writing mode roots). If the
columns are too short to hold the piece of content, there's no way
around splitting it into multiple fragments. See
https://code.google.com/p/chromium/issues/detail?id=367651 , which I
finally realized that should be WONTFIXed not so long ago.

I don't think we should do this, and should change the spec to disallow breaking these elements. Or failing that, just ignore the recommendation to break them sometimes.

Even for images, they may be animated, and eventually we will want to paint their frames in the compositor, and we don't want to have to teach the compositor how to draw fragmented frames. In my (not necessarily exhaustive) testing of Firefox and Edge last week, I don't think they do break them even if the column is too short. Have you tested and found otherwise in some cases?

Certainly the image striping UX in those examples from crbug 367651 doesn't look useful to me...instead developers should lay out their content so that it doesn't have this problem, and the User Agent should fall back to overflow in cases where it fails so that the user can at least see the content in the shape it was intended and read it.

 

> 3. Fragmentation should stop at replaced content and scrollable content
> boundaries.

Yes, we should not examine such objects to find suitable break points
inside. They should just get brutally fragmented if columns are too
short.

--
---- Morten Stenshorne, developer, Opera Software ASA ----
------------------ http://www.opera.com/ -----------------

[Tab Atkins Jr.]

The decision to require breaks when there isn't enough room in a
single column is relatively recent (from a f2f like a year ago iirc?).
It was made with implementors in the room, so they presumably thought
it was reasonable at the time. If you'd like to object and get the
spec changed, send email to the CSSWG list?

(I don't have a strong opinion on this.)

~TJ

[Morten Stenshorne]

Chris Harrelson <chri...@chromium.org> writes:

> On Mon, Jan 18, 2016 at 1:22 AM Morten Stenshorne <mste...@opera.com>
> wrote:
>
>> Chris Harrelson <chri...@chromium.org> writes:
>>
>> > 2. Fragmentation should not be allowed to break replaced (videos,
>> iframes,
>> > images) or scrollable content.
>>
>> Unfortunately, we're only supposed to AVOID breaking inside replaced
>> content and scrollables (and lines) (and writing mode roots). If the
>> columns are too short to hold the piece of content, there's no way
>> around splitting it into multiple fragments. See
>> https://code.google.com/p/chromium/issues/detail?id=367651 , which I
>> finally realized that should be WONTFIXed not so long ago.
>>
>
> I don't think we should do this, and should change the spec to disallow
> breaking these elements. Or failing that, just ignore the recommendation to
> break them sometimes.
>
> Even for images, they may be animated, and eventually we will want to paint
> their frames in the compositor, and we don't want to have to teach the
> compositor how to draw fragmented frames. In my (not necessarily
> exhaustive) testing of Firefox and Edge last week, I don't think they do
> break them even if the column is too short. Have you tested and found
> otherwise in some cases?

No, as I wrote in the bug report, at least per my testing, Blink is the
only one that follows the spec here.

> Certainly the image striping UX in those examples from crbug 367651 doesn't
> look useful to me...instead developers should lay out their content so that
> it doesn't have this problem, and the User Agent should fall back to
> overflow in cases where it fails so that the user can at least see the
> content in the shape it was intended and read it.

I don't have any strong opinions regarding the spec. For multicol, it
may be prettier to just overflow the column. That's not an option for
pagination (printing or overflow:paged), though. Then you definitely
want the striping, or you'd lose content.

[Chris Harrelson]

Ok thanks for clarifying.

 

> Certainly the image striping UX in those examples from crbug 367651 doesn't
> look useful to me...instead developers should lay out their content so that
> it doesn't have this problem, and the User Agent should fall back to
> overflow in cases where it fails so that the user can at least see the
> content in the shape it was intended and read it.

I don't have any strong opinions regarding the spec. For multicol, it
may be prettier to just overflow the column. That's not an option for
pagination (printing or overflow:paged), though. Then you definitely
want the striping, or you'd lose content.

Wouldn't printing just flow the image into the next page? Prininting doesn't use multicol code right now. Also, is overflow:paged a thing you want to implement?

[Morten Stenshorne]

Chris Harrelson <chri...@chromium.org> writes:

> On Tue, Jan 19, 2016 at 10:43 AM Morten Stenshorne <mste...@opera.com>
> wrote:
>
>> I don't have any strong opinions regarding the spec. For multicol, it
>> may be prettier to just overflow the column. That's not an option for
>> pagination (printing or overflow:paged), though. Then you definitely
>> want the striping, or you'd lose content.
>>
>
> Wouldn't printing just flow the image into the next page?

And *stripe* it over multiple pages if it's too large? :)

> Prininting doesn't use multicol code right now.

A lot of the fragmentation code is still shared between multicol and
pagination for printing, though. But I didn't really understand what you
meant by that remark. The ugliness of supporting both striping and
overflowing, depending on fragmentation context type?

> Also, is overflow:paged a thing you want to implement?

It was implemented a long time ago, with -webkit- prefixes.

[Chris Harrelson]

On Tue, Jan 19, 2016 at 11:12 AM Morten Stenshorne <mste...@opera.com> wrote:

Chris Harrelson <chri...@chromium.org> writes:

> On Tue, Jan 19, 2016 at 10:43 AM Morten Stenshorne <mste...@opera.com>
> wrote:
>
>> I don't have any strong opinions regarding the spec. For multicol, it
>> may be prettier to just overflow the column. That's not an option for
>> pagination (printing or overflow:paged), though. Then you definitely
>> want the striping, or you'd lose content.
>>
>
> Wouldn't printing just flow the image into the next page?

And *stripe* it over multiple pages if it's too large? :)

Yes...I just haven't thought about it, so was fishing for information. :)

 

> Prininting doesn't use multicol code right now.

A lot of the fragmentation code is still shared between multicol and
pagination for printing, though. But I didn't really understand what you
meant by that remark. The ugliness of supporting both striping and
overflowing, depending on fragmentation context type?

I think maybe I was just wrong in that remark.

 

> Also, is overflow:paged a thing you want to implement?

It was implemented a long time ago, with -webkit- prefixes.

Ok right.

[Tien-Ren Chen]

Okay, let's sum up what the spec says (to my best knowledge):

1. There are both layout-level and visual-level fragmentation.
2. The layout-level fragmentation adds appropriate padding (i.e.
breaks) in the block progression. Monolithic contents (replaced
elements, overflow clips, inline blocks) are not subject to
layout-level fragmentation, and does not propagate fragmentation
context to its descendants.
3. The visual-level fragmentation (i.e. "the separation and transfer
of page boxes") applies after raster, position:relative, and
transform. As specified in
https://drafts.csswg.org/css-break/#transforms

Here is Chris's argument:
> 1. All paint properties (i.e. effects represented in the property trees)
> should be applied *after* fragmentation

> 2. Fragmentation should not be allowed to break replaced (videos, iframes,
> images) or scrollable content.

> 3. Fragmentation should stop at replaced content and scrollable content
> boundaries.

I think Chris is arguing that we should not have visual-level
fragmentation, so we won't need to impose the complexity of
fragmentation on our compositor. I like this idea too, especially most
of time it doesn't make sense to paint inline overflow (e.g. accent
marks) or visual decoration (e.g. box shadow) to a different column /
page. However this will require a spec change.

IMO visual-level fragmentation isn't all that difficult to implement
either as a filter or a non-linear transform. I personally prefer it
to be done in the transform tree, since hit-testing will be done in
the transform tree and we want the complexity to be contained in a
single tree.

[Chris Harrelson]

By non-linear/transform implementation, do you mean something like what pdr suggested? The problem I can see with that approach is that it is not invertible, meaning the compositor is not able to apply property tree effects post-transform. This would lead to not being able to properly composited-animate elements which would otherwise be animatable, such as transformed elements within a multicolumn container, since the compositor wouldn't be able to understand how to reverse-engineer the new transforms (and clips) necessary after animation update.

Chris

 

[Chris Harrelson]

Sorry this sentence doesn't parse I think. I meant to say that the compositor is not able to reverse-engineer the effect of adjusting a transform on the related fragments, because it could have non-linear effects.

[Tab Atkins Jr.]

Oh wait, the "stripe images if they're too big" thing is just a MAY,
for UAs that can do so and want to avoid data loss.

If we can't easily do it, we're allowed to ignore that and just let it overflow.

~TJ

[Tien-Ren Chen]

BTW I made another jsbin to illustrate my interpretation about the
spec: http://jsbin.com/kepefi/

Let me rephrase Chris's comment:

Suppose we have a transform tree that looks like:
root <--- xformA <--- multicol <--- xformB
Assuming that we rastered the element (based on space B) in the space
A directly, but animated xformB, we won't be able to reuse the texture
rastered in space A easily. Is that your concern?

I would say it is indeed a difficult case, but the difficulty does not
lie on the representation, but animating stuff in a fragmented space
is intrinsically hard. IMO we should raster in space B, and perform
draw-time fragmentation in this case.

On Tue, Jan 19, 2016 at 2:56 PM, Tab Atkins Jr. <jacka...@gmail.com> wrote:
> Oh wait, the "stripe images if they're too big" thing is just a MAY,
> for UAs that can do so and want to avoid data loss.
>
> If we can't easily do it, we're allowed to ignore that and just let it overflow.

But we already have the slicing behavior for printing. I don't think
we can simply unship that.

[Tab Atkins Jr.]

On Tue, Jan 19, 2016 at 3:43 PM, Tien-Ren Chen <trc...@chromium.org> wrote:
> On Tue, Jan 19, 2016 at 2:56 PM, Tab Atkins Jr. <jacka...@gmail.com> wrote:
>> Oh wait, the "stripe images if they're too big" thing is just a MAY,
>> for UAs that can do so and want to avoid data loss.
>>
>> If we can't easily do it, we're allowed to ignore that and just let it overflow.
>
> But we already have the slicing behavior for printing. I don't think
> we can simply unship that.

Slicing on print isn't a problem, if I'm understanding correctly -
there's no animation to deal with. (I may have missed something.)

~TJ

[Chris Harrelson]

Tien-Ren/Morten, do you think my proposal is unworkable or a bad idea?

Let's try to get resolution on this issue now, since it's impacting Slimming Paint v2 work, and also multicolumn is unprefixing soon, so the sooner we make any breaking changes the better.

Chris

[Morten Stenshorne]

Preventing slicing on a regular basis isn't going to be easy, since our
fragmentation implementation is all about slicing the container that
establishes fragmentation, and then just pushing the contents a little
(that's pagination struts) at fragmentainer breaks, so that we don't
split stuff that we shouldn't split (e.g. lines or images). So, letting
unsplittable content overhang a fragmentainer (column) rather than being
sliced when we fail to find enough room for it, isn't straight-forward.

And the current behavior is also correct, according to the spec.

I'd rather say you go ahead and avoid slicing where it's unimplementable
(or at least hard) for Slimming Paint v2, and treat it as a bug. Slicing
unbreakable content, such as images or videos (or having them overflow
the fragmentainer in the block progression direction, for that matter),
is an unideal situation that authors ought to avoid, anyway.

[Chris Harrelson]

Hi Morten,

Sneaking in a very delayed update just before you are going on leave, sorry about that. :)

We'd still like to move forward with a general plan of never cutting up replaced or otherwise unsplittable content. I think what you are suggesting is that we add paint code which ignores fragments 2+ for a PaintLayer which according to the output of layout fragmentation has multiple fragments, but is nevertheless unsplittable? I'm ok with approaches like that and treating suboptimal layout as a bug, as long as it's relatively straightforward.

When I spoke with Levi a while ago he said it seemed feasible to modify the multicol layout code to avoid fragmenting children which are unsplittable. But it sounds from your reply that the multicol layout code doesn't actually know anything about the children, it just blindly cuts up the pagination container according to its "naive" layout bounds?

Chris

--
You received this message because you are subscribed to the Google Groups "paint-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to paint-dev+...@chromium.org.
To post to this group, send email to pain...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/paint-dev/87r3gtgagg.fsf%40aeneas.oslo.osa.

[Morten Stenshorne]

Chris Harrelson <chri...@chromium.org> writes:

> Hi Morten,
>
> Sneaking in a very delayed update just before you are going on leave, sorry
> about that. :)
>
> We'd still like to move forward with a general plan of never cutting up
> replaced or otherwise unsplittable content. I think what you are suggesting
> is that we add paint code which ignores fragments 2+ for a PaintLayer which
> according to the output of layout fragmentation has multiple fragments, but
> is nevertheless unsplittable? I'm ok with approaches like that and treating
> suboptimal layout as a bug, as long as it's relatively straightforward.
>
> When I spoke with Levi a while ago he said it seemed feasible to modify the
> multicol layout code to avoid fragmenting children which are unsplittable.
> But it sounds from your reply that the multicol layout code doesn't
> actually know anything about the children, it just blindly cuts up the
> pagination container according to its "naive" layout bounds?

Exactly. It just cuts somewhere, and tries its best to push overflowing
content to the next column (pagination struts). But if the columns are
too short to fit some tall object, it's going to be sliced anyway. It's
the PaintLayer code that does the slicing, so if we want to avoid this
for certain types of PaintLayer objects (for implementation complexity /
performance reasons), we need to add code that prevents the slicing. I'm
thinking that we want just one PaintLayerFragment for such layers, with
the correct translation for the first column that the object occurs in,
and tall enough to fit the whole thing.

[Chris Harrelson]

Yes, one PaintLayerFragment that is sized to the entire layout object, and positioned to the first column in which it appears. What's the easiest place to put that code?

[Morten Stenshorne]

Probably in PaintLayer::collectFragments(). Either need to add special
code there, or add some parameter to
LayoutFlowThread::collectLayerFragments() and all the way down to
MultiColumnFragmentainerGroup::collectLayerFragments(), so that it
doesn't slice.

[Chris Harrelson]

Patch in progress: https://codereview.chromium.org/2028893002.