The expectation of pointerevent_after_target_removed.html does not match the spec

[masayuki-nakano]

Test List

• pointerevent_after_target_removed.html?mouse
• pointerevent_pointerout_no_pointer_movement.html

Rationale

I'm trying to fix the pointer event dispatching issue of pointerevent_after_target_removed.html?mouse in Firefox.

The test expects that pointerout and pointerleave are fired when_an element removed from the DOM tree when the pointer is under the element. One of the failure reason Firefox is, Firefox does not synthesize Pointer Events at changing element under the pointer.

Once I fix it, pointerevent_pointerout_no_pointer_movement.html starts failing in Firefox. This test checks that pointer events won't be fired when the visibility of an element under the point is toggled.

So, the former expects synthesized pointer events but the latter expects no synthesized pointer events. I think that the former behavior is better for web apps which want to update something when element under the pointer is changed without pointer state changes, and corresponding mouse events are fired since old browser versions. Therefore, the latter behavior blocks web apps to migrate mouse event listeners to pointer event listeners, but I'm not so familiar with Pointer Events, therefore, I'm not sure which test should be changed.

The test expects that pointerout and pointerleave are fired on the click target parent when the pointer moves out from the parent after removing the target from the DOM tree.

The pointerout and pointerleave are not defined as they should not be fired on orphan nodes. Therefore, it may be reasonable that those events are fired on the removed node (Safari works as so). However, it seems odd to dispatch at least pointerout on the (ex-)parent.

[masayuki-nakano]

Oh, the latter came from Mozilla, written by @EdgarChen, and reviewed by @smaug---- in bug 1529904.

(The former came from Chrome.)

[EdgarChen]

Right, the pointerevent_pointerout_no_pointer_movement.html was added for w3c/pointerevents#457 which is for the cases where the pointer doesn't move, not triggering a pointerout event is the expected behavior per spec and w3c/pointerevents#457.

However, pointerevent_after_target_removed.html is different, it moves the pointer to another element after the original element has been removed.

[masayuki-nakano]

Right, the pointerevent_pointerout_no_pointer_movement.html was added for w3c/pointerevents#457 which is for the cases where the pointer doesn't move, not triggering a pointerout event is the expected behavior per spec and w3c/pointerevents#457.

Thank you, Edgar!

However, pointerevent_after_target_removed.html is different, it moves the pointer to another element after the original element has been removed.

Oh, good point. I'll check the things and close this issue if I misunderstand something.

[masayuki-nakano]

I updated the first comment. So, the tests are not incongruous. However, I still think that the expectation of pointerevent_after_target_removed.html is not reasonable.

The test listens to pointerenter, pointerover, pointerout and pointerleave on the parent of the clicking element. The spec does not say that stop dispatching the events on the orphan nodes (after removed from the DOM tree). Therefore, dispatching pointerout and pointerleave on the removed element is fine and Safari does it. However, browsers may have already lost the ex-ancestor information of the orphan nodes. Therefore, it's hard to dispatch pointerleave on ex-ancestors.

I think that the spec should define the behavior in this case first.

[EdgarChen]

The test expects to get pointerenter, pointerover, pointerout and pointerleave dispatched on the div#parent.

Test generates following pointer actions in https://github.com/web-platform-tests/wpt/blob/8f3e51e7eaf38d99a7b6316b010cd673eb25d0ff/pointerevents/pointerevent_after_target_removed.html#L68-L75

let actions = new test_driver.Actions()
    .addPointer("TestPointer", pointer_type)
    .pointerMove(0, 0, {origin: parent})

Pointer is moved into the hit test boundaries of div#child, per spec, pointerenter is dispatched on both div#child and div#parent. And pointerover is dispatched on div#child only.

    .pointerDown()
    .pointerUp()

Now div#child is removed (either in pointerdown or pointerup event handler), but the pointer isn't moved, so no pointer event should be dispatched per w3c/pointerevents#457.

    .pointerMove(0, 0, {origin: done})

Now pointer moves from div#parent to div#other, given that the pointer has moved out of the hit test boundaries of div#parent, should a pointerout or pointerleave event be dispatched to it?

• It could seems a bit odd to dispatch a pointerout event to div#parent without a preceding pointerover event.
• It may make sense that a pointerleave event is dispatched to div#parent.

In any case, it would be good if spec can define the behavior more clearly.

    .pointerDown()
    .pointerUp();

div#other receives click event, test end.

[jgraham]

@nt1m Do you have an option on this from the WebKit point of view?
@flackr Do you have an opinion from the Chromium point of view?

[nt1m]

cc @aprotyas

[flackr]

I think the spec is pretty unclear about how to track whether a pointer is "entering" or "leaving" an element after a dom change. Each browser seems to have subtly different behaviors. I've put together a few cases in a demo:

1. The green element is reparented to the body on click.
2. The red element is reordered within the same parent on click.
3. The purple element is removed on click.

In chrome, it seems that we keep tracking the node the pointer was over even if it was removed and readded and dispatch boundary events when the pointer moves for all changes from that node to the node the pointer is now over.
In firefox, when the node is removed it seems we no longer consider the pointer over it. Reparenting doesn't dispatch any leave events but the next move re-enters all elements.
Safari is similar to chrome, except still fires an event to the removed node.

All browsers have an experience where the enter and leave events can be mismatched as leaving relating to structural changes is invisible to the application. This is the issue being raised in crbug.com/1147998. One reasonable way we could fix this is when the node the pointer is over is removed, consider the still attached ancestor to be the node the pointer is now over.

As for whether or not we should fire events at removed nodes, while we do have precedent for this with touch events (demo) we don't for pointerevents.

So my proposal would be a behavior that no browser currently has for pointer events (as far as I can tell), but would ensure that the count of pointers would be correct. That being, when a node is detached, we track the nearest still attached ancestor of the previous node the pointer was over so that on the next move we don't re-enter any of the still attached ancestors if we're still over them.

[mustaqahmed]

In chrome, it seems that we keep tracking the node the pointer was over even if it was removed and readded and dispatch boundary events when the pointer moves for all changes from that node to the node the pointer is now over. In firefox, when the node is removed it seems we no longer consider the pointer over it. Reparenting doesn't dispatch any leave events but the next move re-enters all elements. Safari is similar to chrome, except still fires an event to the removed node.

Thanks for the concise summary of the differences. To add some details, Chrome remembers only the node under the pointer and handles all boundary event dispatches here (ignoring PE vs ME complications for sake of ease). Firefox diff seems to be just what you mentioned: forget a moved/removed node then re-hittest.

So my proposal would be a behavior that no browser currently has for pointer events (as far as I can tell), but would ensure that the count of pointers would be correct. That being, when a node is detached, we track the nearest still attached ancestor of the previous node the pointer was over so that on the next move we don't re-enter any of the still attached ancestors if we're still over them.

The big comment in the Chrome code above reminded me that in the past we found that "dynamically tracking the nearest attached ancestor" is not without its own issues: if we reorder (or even just reattach at the same point) an ancestor of the node under pointer, it will cause firing a sequence of redundant leave/enter events that we don't see in Chrome today. We concluded that the non-trivial cost of dynamic tracking in a deep DOM doesn't worth it: we would need to remember the whole ancestor chain and update it on every change in kIsConnectedFlag.

Let's discuss this in tomorrow's PEWG meeting.

[flackr]

Firefox diff seems to be just what you mentioned: forget a moved/removed node then re-hittest.

To be clear, every browser re-hittests on move, the difference is that in firefox there is no previous node so it re-enters every ancestor.

The big comment in the Chrome code above reminded me that in the past we found that "dynamically tracking the nearest attached ancestor" is not without its own issues: if we reorder (or even just reattach at the same point) an ancestor of the node under pointer, it will cause firing a sequence of redundant leave/enter events that we don't see in Chrome today.

Which is similar to, but fewer events than the current behavior on firefox.

We concluded that the non-trivial cost of dynamic tracking in a deep DOM doesn't worth it: we would need to remember the whole ancestor chain and update it on every change in kIsConnectedFlag.

I think it's actually much more trivial than this. mouse_press_node_ in blink already has this behavior that when the current mouse_press_node_ is removed it automatically switches it to track the nearest still attached ancestor (see MouseEventManager::NodeWillBeRemoved). We could do the same without needing to remember the whole ancestor chain because any changes to that chain will fire NodeWillBeRemoved so it can use the node's ancestor chain.

[smaug----]

when a node is detached, we track the nearest still attached ancestor of the previous node the pointer was over

I think that could work. Need to be careful of course with shadow DOM, ancestor here should mean the same thing as it does when constructing DOM Event path.

[mustaqahmed]

Let's discuss during today's PEWG meeting the following suggested changes to the test, now that we seem to have reached consensus on @flackr's proposal above:

1. In the current test, the pointer enters a node (so also enters its parent) then deletes the node then exits the parent. So the current assumption is correct that at the parent element, both (a) 'enter' vs 'leave' events are paired and (b) 'over' and 'out' events are paired. Right?

2. The test currently also assumes a relative order (at the parent) between 'enter/leave' and 'over/out' events. This matches corresponding UIEvents WPTs (mousemove-across.html and mousemove-between.html). But PointerEvent spec does not specify this (nor does UIEvents spec IIUC). Is it okay to remove this assumption from this particular test?

3. We need to add an assertion on boundary events received at the deleted element. Currently this is missing.

[smaug----]

One related thing to think about is whether changing the slot attribute of a slotted element so that it gets to a different place in the flattened tree has similar issues. (or changing the name attribute of a slot element)

[mustaqahmed]

As per PEWG discussion today:

• @flackr suggested adding a test where the pointer remains stationary after deletion.
• Update to the second point in my last comment above: this test should assert the relative event order as per this UI Event Spec section, which is (over, enter, out, leave).

[flackr]

• @flackr suggested adding a test where the pointer remains stationary after deletion.

To clarify, I meant we should have a test where the pointer moves, but remains over the parent (rather than moving to another element). If the pointer never moves no subsequent events should be fired.

[masayuki-nakano]

Oh, I realized that there are these paragraphs in 4.1.3 Firing events using the PointerEvent interface:

Fire the event to the determined target. The user agent SHOULD treat the target as if the pointing device has moved over it from the previous target for the purpose of ensuring event ordering [UIEVENTS]. If the needs over event flag is set, an over event is needed even if the target element is the same.

Save the determined target as the previous target for the given pointer, and reset the needs over event flag to false. If the previous target at any point will no longer be connected [DOM], update the previous target to the nearest still connected [DOM] parent following the event path corresponding to dispatching events to the previous target, and set the needs over event flag to true.

These paragraphs explain the Chrome's behavior is right one. However, it's different from the conclusion from w3c/pointerevents#477.

For the DOM mutation performance under a pointer, not dispatching any events makes browsers faster. However, I'm not sure whether it's so important or not.

@smaug---- @mustaqahmed @flackr

[masayuki-nakano]

Well, currently, I think that the paragraphs should be dropped from the spec because it's odd to dispatch pointer boundary events only when the target is removed. With this definition, when an element is appended and its region becomes underneath of the pointer, pointer boundary events are not fired. So, I feel they are inconsistent behavior from developers point of view.

[mustaqahmed]

Oh, I realized that there are these paragraphs in 4.1.3 Firing events using the PointerEvent interface:

...

These paragraphs explain the Chrome's behavior is right one. However, it's different from the conclusion from w3c/pointerevents#477.

Thanks @masayuki-nakano for paying close attention to this. To properly fix any gap, please elaborate any difference you spotted. Sorry if it is obvious, I am a bit lost between various boundary event discussions!

For the DOM mutation performance under a pointer, not dispatching any events makes browsers faster. However, I'm not sure whether it's so important or not.

I completely agree that not dispatching pointer event during DOM mutation is a good idea. The spec seems to support this in my opinion because 4.1.3 is about what happens during dispatch. What do you think?

With this definition, when an element is appended and its region becomes underneath of the pointer, pointer boundary events are not fired.

My interpretation is that boundary events are correctly dispatched for appended elements because of the same reason above: the section is about what happens during dispatch. If DOM has been modified since previous event dispatch, "previous target" would be different from "current target" (the element appended underneath the pointer), forcing boundary event firing.

[flackr]

Well, currently, I think that the paragraphs should be dropped from the spec because it's odd to dispatch pointer boundary events only when the target is removed. With this definition, when an element is appended and its region becomes underneath of the pointer, pointer boundary events are not fired.

When an element is appended and becomes underneath the pointer for the next event dispatch, the previous target will differ from the target of the next dispatched event and we will dispatch boundary events to enter the new element.

[masayuki-nakano]

@mustaqahmed

Oh, I realized that there are these paragraphs in 4.1.3 Firing events using the PointerEvent interface:
...
These paragraphs explain the Chrome's behavior is right one. However, it's different from the conclusion from w3c/pointerevents#477.

Thanks @masayuki-nakano for paying close attention to this. To properly fix any gap, please elaborate any difference you spotted. Sorry if it is obvious, I am a bit lost between various boundary event discussions!

Sorry, I'm also lost. I'm a new for Pointer Events, therefore, I'm not familiar with discussions before.

Initially, I was thinking that pointer boundary events should be fired same time as mouse boundary events for making migration from mouse events to pointer events easier for developers (and avoid misunderstanding the new API). However, each event type definition of pointer boundary event defines they should be fired only when a pointer move occurs and w3c/pointerevents#457 and test for it considered as same as the event type definitions. Therefore, I understood as so. However, I found the paragraphs yesterday when I was investigating another issue. Therefore, I'm now confused.

For the DOM mutation performance under a pointer, not dispatching any events makes browsers faster. However, I'm not sure whether it's so important or not.

I completely agree that not dispatching pointer event during DOM mutation is a good idea. The spec seems to support this in my opinion because 4.1.3 is about what happens during dispatch. What do you think?

Ah, indeed, dispatching pointer boundary events at next input makes sense from the performance point of view. However, it means that if user stops moving pointer for a while, pointer boundary event listeners cannot change something visually immediately after the mutation but will do it at next input suddenly. It maybe looks odd.

With this definition, when an element is appended and its region becomes underneath of the pointer, pointer boundary events are not fired.

My interpretation is that boundary events are correctly dispatched for appended elements because of the same reason above: the section is about what happens during dispatch. If DOM has been modified since previous event dispatch, "previous target" would be different from "current target" (the element appended underneath the pointer), forcing boundary event firing.

I don't see any definition about "forcing boundary event firing" in current spec. Which part does make you think so?

@flackr

Well, currently, I think that the paragraphs should be dropped from the spec because it's odd to dispatch pointer boundary events only when the target is removed. With this definition, when an element is appended and its region becomes underneath of the pointer, pointer boundary events are not fired.

When an element is appended and becomes underneath the pointer for the next event dispatch, the previous target will differ from the target of the next dispatched event and we will dispatch boundary events to enter the new element.

Well, if dispatching pointer boundary events is better for Pointer Events users, I don't disagree with doing that. However, the problem is, I don't find the definition in current spec, and the each event type documentation is unclear about mutation cases. Therefore, at least they should be defined clearer.

[flackr]

Well, if dispatching pointer boundary events is better for Pointer Events users, I don't disagree with doing that. However, the problem is, I don't find the definition in current spec, and the each event type documentation is unclear about mutation cases. Therefore, at least they should be defined clearer.

I agree! This was exactly the problem with the ui-events spec as well. It doesn't say anything about the mutation case which resulted in different removal behavior between the browsers.

The problem that the text you are concerned about is fixing is a real one, described in https://issues.chromium.org/u/0/issues/40156858 . The TLDR is that when the element the cursor is over is removed, browsers did not dispatch pointerleave / pointerout to any of the ancestors because on the next event (which results in dispatching boundary events), those nodes no longer were in the ancestor chain of the removed node.

The updated text from w3c/pointerevents#491 changes this so that the previous element that we dispatch boundary events from tracks the still connected ancestor, so that from the perspective of ancestor listeners the cursor movement makes sense (e.g. we don't send enter to a node the cursor had previously entered and not seen a leave sent to yet).

To go back to the specific example, let's say you have a single node:

<div id="a"></div>

The cursor moves over this node, so we dispatch boundary events and set it as the |previous target|. Then when a new node is moved under the cursor or added under the cursor the next event dispatch will dispatch boundary events from that |previous target| to the new current target. This behavior is not new - it is intended to be the same as the implied behavior from ui events per https://www.w3.org/TR/uievents/#events-mouseevent-event-order

I believe there may be browser differences as to whether these boundary events occur at the next movement or earlier, but the change in the spec text was only to make sure that when you have a removed node we still know which node the cursor was over per the previous dispatched boundary events.

[mustaqahmed]

Didn't we already agree above that the spec requires firing boundary events before down and up events too? For convenience, here is the relevant spec text (link):

Fire the event to the determined target. The user agent SHOULD treat the target as if the pointing device has moved over it from the previous target for the purpose of ensuring event ordering. If the needs over event flag is set, an over event is needed even if the target element is the same.

If so, the last two WPT changes merged above go against the spec in my opinion and should be reverted:

• this one just landed, and
• this one landed a while ago.

In case the source of the confusion is the following question (which I didn't directly answer, sorry):

I don't see any definition about "forcing boundary event firing" in current spec. Which part does make you think so?

Please see the paragraph right after the quoted spec text above: the needs over flag is there just for this.

@masayuki-nakano: Are we on the same page?

[masayuki-nakano]

Didn't we already agree above that the spec requires firing boundary events before down and up events too? For convenience, here is the relevant spec text (link):

Fire the event to the determined target. The user agent SHOULD treat the target as if the pointing device has moved over it from the previous target for the purpose of ensuring event ordering. If the needs over event flag is set, an over event is needed even if the target element is the same.

Ah, yeah, the text seems reasonable for new inserted element under the pointer.

If so, the last two WPT changes merged above go against the spec in my opinion and should be reverted:

* [this one](https://github.com/web-platform-tests/wpt/pull/45192) just landed, and

* [this one](https://github.com/web-platform-tests/wpt/pull/44199) landed a while ago.

Well, for the latter, yes, it should be reverted. I'm still not 100% sure for the former, but I guess it's so. (ccing @EdgarChen)

[mustaqahmed]

Changes to both the spec and the WPT landed yesterday.

[masayuki-nakano]

Changes to both the spec and the WPT landed yesterday.

Thank you very much, and sorry for I couldn't do that because I have some urgent bug fixes...

[mustaqahmed]

No worries. Let us know if Sec4.1.3 still needs any update.