feat(web): longpress restoration, enhancements 🐵 #9698
Conversation
User Test ResultsTest specification and instructions User tests are not required Test Artifacts
|
keymanapp-test-bot
bot
changed the title
|
Notes from today's team meeting demo:
Personal note / question: So, key de-selection without dropping the menu?
|
Yes. A user may move their key back towards the longpress to select a key again |
jahorton
force-pushed
the
feat/web/longpress-restoration
branch
from
a8887c7 to
e7fa856
Compare
|
Two new updates:
Fixed. Cancellation still occurs if the finger goes too far off the top or sides of both the actual keyboard and the subkey menu. Aside from that, selection is only maintained near subkeys and is cleared when wandering far while still within the keyboard's bounds. I also upped the permitted fudge factor (to maintain selection) a bit.
Alternative idea: once roaming has begun, just... disable the up-flick shortcut. After a new base key has been reached, the up-flick shortcut is no longer viable for that keystroke. It still works before any roaming from one base key to another occurs. I opted to go this way partially because I did find it unintuitive when the neighboring key's menu got displayed on quick input attempts - like if I started near the right edge of 'g' and went up-right, sometimes the subkey menu for 'h' would get displayed instead if I actually went right (instead of up-right) when trying to input the shortcut. Now, the shortcut just won't trigger... rather than triggering for the wrong key. "Two birds, one stone" - assuming my perspective on this is reasonable. (Granted, the latter problem is largely due to using a mouse to emulate touch input... I think.) |
jahorton
changed the base branch from
feat/web/key-specific-gesture-integration
to
fix/web/integration-test-fixes
This has since been fixed as well; turns out, I'd missed a regression in previous PRs that resulted in the |
| targetRoot: this.element, | ||
| inputStartBounds: vkbd.element, | ||
| maxRoamingBounds: sustainBounding, | ||
| itemIdentifier: (coord, target) => { |
There was a problem hiding this comment.
Essentially, this is findNearestKey & its helper, but for subkeys.
|
I know it's been quite a while since the original requests, but I'd still like to ping @eddieantonio (submitted #1886) and @MattGyverLee (submitted #1025) on this to see if the changes seem satisfactory. While the build product won't be directly accessible due to bulid-agent permissions, I can provide y'all a few screen recordings based on the PR's current state. Note: I've set them to be visible via drop-down when clicking the entries below. Having lots of animated gifs next to each other can be distracting, after all. Gigantic longpress menu - two rows, second with one less key
Going beneath the longpress's base key (rather than the subkey menu) will cancel selection, allowing the user to abort the longpress if they release the touch in such a location. Fitts's law - loose selection range
I do realize that I did return to the actual key to end the longpresses in those two gifs... but it's not necessary, as the selected key will trigger regardless. It's just hard to override my "old habit" in the heat of the moment while trying to take a concise recording. Swift longpresses - working with muscle memory
The 's' longpress is there for comparison on the standard longpress wait timing, while the rest trigger and execute far more quickly. As long as the longpress-shortcut flick is in a generally upward direction - within roughly 67.5 degrees of a purely-upward direction - and the user doesn't shoot extremely far, the shortcut will trigger early. Accordingly, if you know where the subkey you want is (which I admittedly didn't model terribly well here, noting the massive 'f' overshoot), it becomes possible to quickly use such subkeys by muscle memory. Of course, this is probably trickier when there are many subkeys: (1) some could lie too far off the side, outside the accepted angle range, and (2) if there are two rows, gotta be careful to land on the side of the correct row. On that last one, though - we'll probably have the shortcut functionality disabled on any key with defined upward flicks. Too much risk of getting the wrong interaction if the user's off by a bit in such a case. But, with no flicks defined... well, you get something that's almost a pseudo-flick if your subkey menus are reasonably compact and well-designed. Fitts's law for system keyboards
Since subkey menus can't go out of bounds on iOS, gotta make sure we get overshoot range if the menu goes beneath the base key! I used a lightly-modified version of Web for this one in order to match the iOS keyboard's constraints. If there are any other usage patterns you'd like to see a recording for, let me know. |
| const subkeyStyle = this.subkeys[0].style; | ||
| const subkeyHeight = Number.parseInt(subkeyStyle.height, 10); | ||
| const basePadding = -0.666 * subkeyHeight; // extends bounds by the absolute value. | ||
|
|
||
| const bottomDistance = underlyingKeyBounding.bottom - baseBounding.bottom; | ||
|
|
||
| const roamBounding = new PaddedZoneSource(this.element, [ | ||
| // top | ||
| basePadding, | ||
| // left, right | ||
| basePadding, | ||
| // bottom: ensure the recognition zone includes the row of the base key. | ||
| // basePadding is already negative, but bottomDistance isn't. | ||
| -bottomDistance < basePadding ? -bottomDistance : basePadding | ||
| ]); |
There was a problem hiding this comment.
This is the section that defines the 'roaming bounds' that enable and disable key selection. Note the basePadding entry - that value in particular is what defines the maximum overshoot we're allowing. We can easily tweak it to make it "more" or "less" permissive if needed.
Note: values here are negative for 'external padding' - essentially, CSS 'margin', but positive for 'internal padding' - essentially, CSS 'padding'.
|
I've noted a few TODO items on #9752 that likely belong as a direct followup to this work... or possibly even as a component of this PR:
Given the size of this PR at present... I'll probably do a followup to this one but before #9752 as a remedy. |
|
@jahorton This looks GREAT! Why does the O jump up before the menus shows? It's interesting that the original "o" of the base key disappears while the longpress is active. Is that intentional or a styling mismatch of white text on a white background? |
|
One small thing, the popup is typically centered over the key if space allows. When there is an odd number of columns, one choice will always be preselected if you don't move your finger. That's great and repeatable. Most of my keys have only one popup, which happens to be an odd number. Unfortunately, when there are an even number of columns, it's ambiguous/fidgety which one will be selected. I don't know how many of us can regularly hit the left or right side of a key.
What do you think about shifting the popup display half a key left or right when there is an even number of columns so that one "default" key is unambiguously selected? If possible, I suggest it could be shifted a half-key left when the base key is on the right side of the screen (O) and a half-key right when the base key is on the left side of the screen (E). Gboard does this realignment:
|
|
This looks like a fantastic update! Good GIFs. @MattGyverLee's suggestion about popup display placement is worth looking into. Apart from that, I'm afraid it's been far too long since I've seen Keyman code to comment on the actual code. But this will be a nice addition when it lands! EDIT: and, yes, it will fix #1886! |
If you're talking about the touch-position 'o', that's because of a little shortcut we've long had for longpresses - an upward motion of sufficient distance during the longpress can trigger the subkey menu to appear early. It does require moving a distance about equal to 1/4 a key's height, mostly in a vertical direction, so you're probably noticing the movement that's triggering the early display.
About that - we recently also had this PR go through: #9496. It will be possible to specify a 'default' subkey that will be selected before any movement is registered. While this likely won't "play nice" with the quick-menu up-flick shortcut, if you just hold position, the specified 'default' key will be the pre-selected one. There's also a bit of "wiggle room" / "noise cancellation" before the default is unselected. If no default is selected, but the base key has a direct match as a subkey, that one gets picked as default instead. So...
... you'd only run into this issue if there's no default subkey specified in KMW 17.0. That realignment idea is something we could look at, though due to limited time we may have to defer that specific idea to a later version; we still have to get multitaps and flicks working correctly and ensure those have good visual feedback, and those two currently have higher priority. |
This seems like a very low priority feature for now. |
Fixes #1025.
Fixes #1886.
This PR reworks the old longpress handling methods and classes and connects them to the new gesture engine; they're now operable at this stage of integration. While I was at it, I took the opportunity to vet some related features baked into the engine; subkeys are now selectable by a "nearest key" heuristic, rather than requiring the touchpoint to be directly inside them.
From the referenced issues:
This behavior is now 100% supported in the exact manner suggested by the image. Once the subkey menu is displayed, horizontal movement will select the key in the same column at the touchpoint in the nearest subkey row.
(The recording above is in a "before" state, obviously.)
Similarly for the left edge and the top edge - for all three cases, the nearest key will remain selected until the touch location wanders "too far" from the subkey menu - approximately 1/4 to 1/3 a key's height. It's a decent, but not extreme, fudge factor.
[Edit: this has since been approximately doubled.]
Now...
@keymanapp-test-bot skip