Navigating the StoryRiver using KeyboardShortcuts

[BurningTreeC]

In #4782 we've discussed broadly the possibility of navigating the story-river using keyboard shortcuts. I'd like to move the discussion here, so that we can gather ideas for 5.1.24 or one of the next releases to tackle this

[BurningTreeC]

So ... I'm now at the point where I'd like to tackle this 😸

I'll try to create a summary of our previous discussions here

[pmario]

Before you start coding please talk to me

[BurningTreeC]

Hi @pmario, I'm all ears 😸

[BurningTreeC]

@pmario - what are your ideas, how would you implement this?

[pmario]

@BurningTreeC ... Sorry for the delay.

The last time I did try to tackle the problem, it was top-down from the "tm-navigate" message down to the storyview. It didn't work out too well.

So now let's reduce it to the keyboard handling and see, where we need to make changes. .. At the moment the interface between the different js-code elements are not well defined and that makes it difficult to change the code. .. That's the reason, why all the existing plugins, that deal with the story "visibility" handling are very hacky.

I think for the keyboard handling, the only important tiddlers are $:/StoryList and $:/HistoryList but ... (more to follow)

[pmario]

Since TW 5.1.23 we use -[prefix[$:/StoryList]] -[prefix[$:/HistoryList]] in the "saver" filters, which will allow us to manage several independent stories. That's a big win, since the last try

[pmario]

So the keyboard handling will need to know about the "active" story, that should be addressed with a keyboard shortcut. So every event needs to know, from which "story-source-context" it is activated and which "story-target-context" it needs to modify

[BurningTreeC]

Yes, that's what I was thinking, too.

Hi @pmario btw

[pmario]

There are 16 messages, that are handled by the navigator widget.

https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/widgets/navigator.js#L19-L37

I did create a "test-framework" that will allow us to test the TW code interactively, to make sure, that we don't break things. https://wikilabs.github.io/editions/trigger/

[BurningTreeC]

Hi @pmario, what are you doing in your test framework? Could you explain it a bit?

[pmario]

I basically click the "Next" button, and at the end there should be "no errors" in the output field. This should allow us to check, if we messed something up in the navigator widget

[BurningTreeC]

Yeah, but what are the messages the "Next" button is sending? What does it do?

[pmario]

From my point of view it sends the tm-navigate, with some NEW info about the "target-story-context"

[pmario]

tm-navigate can be "intercepted" by the linkcatcher-widget, which allows end-users to "re-traget" links to different stories, using wikitext

[pmario]

As you can see:

https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/widgets/navigator.js#L153

the tm-navigate message uses this.addToStory();, which is defined in navigator.js. this.addToStory() looks like this:

https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/widgets/navigator.js#L131

which is defined in story.js

https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/story.js#L37

---

This dataflow is not as consistent as here, for all of the messages. .. We will need to fix this

[pmario]

You know about these functions, since you helped to make the "story" be part of the navigator widget. So we only need to instantiate the code once

[pmario]

I think story.js should be to ONLY place that modifies $:/StoryList??? and $:/HistoryList??? tiddlers. We need to clean up the API here

[BurningTreeC]

right, I know about the functions in the navigator widget

[pmario]

As soon as the program hits this function: https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/story.js#L82-L91

The "animation-chane" is triggered with a "store change event"

[pmario]

may be just a reminder

The startup module $:/core/modules/startup/story.js

"https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/startup/story.js#L21

contains several places, where it deals with the StoryList tiddlers. .. BUT only the default ones are active atm. So we need to think about startup-handling with several stories active at the same tim.

[BurningTreeC]

That startup module deals with the main story - do we want to make the main story switchable? Like storing the current story in a config tiddler and using that as the main story dynamically?

[pmario]

If 2 or more story rivers are visible at the same time, it may be a "feature request" by users to init both of them properly

[pmario]

As written in: #4782 (comment) The HistoryList change triggers the .handleHistoryChange() function in the list-widget.refresh()

https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/widgets/list.js#L166

which calls storyview.navigateTo() function.

which is implemented with 5 different module-tpye: storyview views atm (3 core and 2 plugins)

The default one is title: $:/core/modules/storyviews/classic.js

The storyviews are essential, because the do the add/remove/navigate animations, which are also needed by the keyboard handlers

[pmario]

Storyviews are the only elements, which can be used to tell us about the tiddler positions, which is needed to trigger "prev/next" tiddler actions.

At the moment those views don't have an API to give proper feedback about these infos

[pmario]

this line: https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/storyviews/classic.js#L37

Is important too. Since at the moment every function that uses positions to scroll a tiddler into view are extreamly "hacky", since the storyviews have no proper API to give feedback where the real top of the page coordinates are

[pmario]

In the whole navigation process, there are several widgets involved:

• list-widget, which defines HistoryList, StoryViews, ViewTemplate and EditTemplate. Changes to HistoryList tirgger the refresh mechanism
• navigator-widget
• link-widget
• linkcatcher-widget
• keyboard-widget

IMO we must keep an eye on the great context. Otherwise the implementation will be hacky and will make it even more difficult to fix the navigation mess.

[pmario]

I think, all the infos, that are needed by the global kb-handling should be stored in the History-tiddlers and may be an new $:/StoryContext or $:/StoryInfo or similar named tiddler, that tells the global handling, which story to manipulate

[BurningTreeC]

I believe recently the modal mechanism was "wrapped" somehow into a navigator widget but I don't recall how

[pmario]

Do you think, modals should be usable with global kb-handling? ... Or should this be a "second" step? ... Can it be a "second step"?

[BurningTreeC]

We can already trigger modals using global kb shortcuts - what do we want to handle?

[pmario]

A modal can contain a "story". So it may be needed to navigate it

[pmario]

The linkcatcher is a lightweight (memory / code) widget, that was designed to "trick" the navigator widget. From my point of view it should be used to send the right information to the navigator widget, so it can do its job

[pmario]

IMO the same is true for the NEW "keyboard" handlers

[pmario]

I do have some prototype code for "link" and "linkcatcher" that sends more info to the navigator which can be used to define the "target-story-context" ... But we need to define the information, that we need first

[pmario]

@BurningTreeC .. You may think, that all of this is "overkill". ... I know, but I think it's the big picture, that we have to keep in mind. ...

Because if the new StoryKeyboard handling is a hack, IMO it will make it impossible to fix the existing navigation API issues.

[pmario]

I don't know the keyboard handlers very well. I was happy, that you did all the heavy lifting for the "input / dropdown" navigation.

I'll have to dig deeper into the code and the new wikitext, which looks super "complex" to me atm.

[pmario]

$:/HistoryList

• has a text field and
• current-tiddler field atm
  • current-tiddler is written here: Story.prototype.addToHistory = function(navigateTo,navigateFromClientRect) {
  • https://github.com/Jermolene/TiddlyWiki5/blob/5832002feb2b662d65a8d0a38c1c81a390e03317/core/modules/story.js#L100

We probably need more info here

[pmario]

Proposal

I suggest, that there should be a possibility, that plugins and new keyboard handler functions should be able to add arbitrary fields here, that will trigger a "history-refresh" in the list-widget.

[pmario]

eg: with a "selected-tiddler" or "active-tiddler" it will be easy to implement some extra highlighting in the TOC, or the "story river".

[pmario]

$:/StoryList

• has the list field, which can probably be used to define "next / previews" function

[pmario]

• The $:/StoryList widget is highly connected with the "navigator-widget".
• The $:/HistoryList and StoryViews are highly connected with the "list-widget"

[BurningTreeC]

What do you mean with "StoryList widget"?

[pmario]

uups ... fixed the text above. It should be $:/StoryList tiddler

[pmario]

Active Story

• IMO the global keyboard handling needs some info, which story is the "active" one, that should be manipulated

[pmario]

Yea, but there may be a unique "qualified" name attached eg: $:/config/StoryList-189435334534

May be:

title: $:/config/StoryList-189435334534
text: #NavigatorWidget:NavigatorWidget ActionNavigateWidget ListWidget [[Draft of 'New Tiddler']] $:/StoryList

using the same format as a TW permalink: https://tiddlywiki.com/#NavigatorWidget:NavigatorWidget%20ActionNavigateWidget%20ListWidget%20%5B%5BDraft%20of%20'New%20Tiddler'%5D%5D%20%24%3A%2FStoryList but without the URL escape chars

[pmario]

... and $:/config/HistoryList respectively

I'm not sure, if we need s config for HistoryList. What do you want to store there?

[BurningTreeC]

I'm not sure, if we need s config for HistoryList. What do you want to store there?

The title of the HistoryList corresponding with the current storyList

[pmario]

May be a data tiddler for the state:

title: $:/state/active/Story

history: $:/HistoryList-1
story: $:/StoryList-1

[BurningTreeC]

Hmm. That is something to decide another day. But we have common sense that both need to be stored somewhere

[BurningTreeC]

@pmario - I DON'T think this is an overkill but I'm missing an overview about the changes you have in mind

It would be helpful if you could provide some starting points like "the current story should be stored in a config or state tiddler", "the currently active/selected tiddler should be stored in the $:/HistoryList tiddler". "The problem that arises from that is that ..."

I currently find it a bit confusing knowing that you have a clearer picture in your mind than you are depicting here

[pmario]

... I currently find it a bit confusing knowing that you have a clearer picture in your mind than you are depicting here

Yes. I think every list is a story. ... But the current UI doesn't use it that way, because we don't have proper "multi story" handling in the core.

Once we have that, I'm sure our UI wikitext can be much simpler and much more powerful

[pmario]

Navigate multiple Stories

The navigator-widget has the following parameters:

Attribute	Description
story	Name of the tiddler containing the story list to be manipulated
history	Name of the tiddler containing the history list to be manipulated
openLinkFromInsideRiver	Determines the location for opening new tiddlers from links within the story river: at the top or bottom of the story river, or above or below the current tiddler
openLinkFromOutsideRiver	Determines the location for opening new tiddlers from linkes outside the story river: at the top or bottom of the story river

story and history are defined whith the widget call in wikitext.

The other params should be configurable within the control panel or a different UI

@BurningTreeC ... "The other params" may also be important for the global kb-handling

[pmario]

IMO

Attribute	Description
relinkOnRename	Determines if tags in other tiddlers are renamed if the tiddler title changes. yes or no. default: no

should be global ONLY

[BurningTreeC]

please @pmario create a SUMMARY of your ideas

[pmario]

I'll try. But I think #5262 (comment) is the most important one atm, that may be written into an actionable "proposal issue"

[Jermolene]

Thanks @pmario it would indeed be useful to have a clear summary of the proposal, and perhaps also to enumerate the requirements the proposal is trying to meet.

I'm concerned about the attention to multiple story rivers. That's definitely a feature that we should add to the core, but I'm not sure that we need to be so concerned with it here. This is about maintaining a visible focus which can be moved via the keyboard or mouse, and keyboard actions that apply to the focused element. We definitely need to be able to track the focus moving to elements outside the main story river (eg elements in the menu bar or side bar), but I don't think that those elements would necessarily be tiddlers (or "tiddler frames" to be clearer), nor would they necessarily be in a river.

So, there wouldn't be a keyboard shortcut for "edit tiddler", there'd be a keyboard shortcut for "edit current thing" which would adapt according to the element that has the focus.

We will need keyboard shortcuts that affect the current story river: the <$keyboard> widget would wrap the story river, and trap the required shortcuts, just as we do for tiddlers.

For me, the underpinning for all of this is the browser implementation of focus. As we've discussed before, I think we should enhance the refresh cycle so that it preserves the focussed element, and tracks the focussed element via state tiddlers.

When we introduce multiple story rivers we will certainly need state tiddlers to keep track of things like which story river should be used to open new tiddlers, but I'd be hoping we wouldn't need to worry about that yet.

[saqimtiaz]

Would you mind elaborating a bit (for my understanding) what keeping track of the focus element and restoring its focus after refresh means in this context? In which cases is it needed? How does the refresh cycle affect the currently focused element?

@BurningTreeC imagine a story river with the tiddlers:

• A
• B
• C

Imagine that B is the tiddler that has focus, and now you click a button in that tiddler that makes tiddler B get refreshed in the story. This means the DOM node for the tiddler content will be destroyed and re-created to refresh it. After this refresh, tiddler B will no longer have focus, the focus was lost when the DOM node was destroyed before being re-created.

So as a simple example we need to remember which tiddler had focus, and then restore that after refresh. Ideally we want to take it a step further and restore focus to the exact element which had focus, which might be a button or an input field in a tiddler.

As background note how the transclude widget calls refreshSelf() when the tiddler being transcluded has changed:
https://github.com/Jermolene/TiddlyWiki5/blob/master/core/modules/widgets/transclude.js#L107

And then note how refreshSelf() destroys the widgets DOM nodes and then calls render() to recreate them:
https://github.com/Jermolene/TiddlyWiki5/blob/master/core/modules/widgets/widget.js#L455

[Jermolene]

I think @pmario is already thinking about users that add their own stories, but I agree that we can think about that later.
Would you mind elaborating a bit (for my understanding) what keeping track of the focus element and restoring its focus after refresh means in this context? In which cases is it needed? How does the refresh cycle affect the currently focused element?

The "refresh cycle" is the process that updates a rendering when there are changes in the tiddler store. It involves recursively calling the refresh() method of each widget in the tree, giving them a chance to respond to changes in the tiddler store. In doing so, they will create and delete DOM nodes as required. The problem comes if one of those DOM nodes had the focus; when it is deleted the browser will follow it's own rules to assign the focus elsewhere, but it won't automatically re-assign the focus to any replacement element.

So the gist of what we need to do is to update the refresh cycle so that first we somehow record which widget has the focus, and then after the refresh cycle we restore it. As I say, the complication is that some or all of the widget and DOM trees might be destroyed and recreated in the course of the refresh. That means that we can't store state data in the DOM tree or the widget tree.

[BurningTreeC]

Hi @Jermolene ,

Thanks, you made it all clear!

[BurningTreeC]

@Jermolene @saqimtiaz ,

Thank you both for your clear explanations!

[BurningTreeC]

@saqimtiaz

I am going to be tied up for another two weeks or so, but if you @BurningTreeC like I can work with you on this after that, much as we did for the keyboard-driven-inputs.

I'd like to do so very much. In the meantime I'm looking into the code to understand the inner workings and to (maybe) fiddle around with it

[pmario]

I think it's much easier than you think. The only thing the global story handling needs to konw is, which is the "target story context" that should be modified. Then it needs to modify the $:/History-xxx tiddler and all the magic should happen automatically.

The only difficulties I see, is to combine mouse and keyboard actions. Eg. The user may want to scroll down with the mouse wheel. Then eg: use PageUp key to scroll ti the top if the the "active" tiddler if it is a long tiddler. If the tiddler is short, PageUp should scroll to the top of the previous tiddler. ... and so on.

If the mouse wheel is scrolled down or up in the meantime, the focus of the tiddler needs to change.

I think ALL if this can be done already with no problems, for the hardcoded single story we have. BUT it gets extreamly hacky as soon as we start to introduce several stories.

I'll try to create a prototype, to show what I mean.

[Jermolene]

I think it's much easier than you think. The only thing the global story handling needs to konw is, which is the "target story context" that should be modified. Then it needs to modify the $:/History-xxx tiddler and all the magic should happen automatically.

I'm having trouble following your meaning here. What is a target story context? What should modify it? What is the "it" that modifies the history list tiddler? What magic happens?

The only difficulties I see, is to combine mouse and keyboard actions. Eg. The user may want to scroll down with the mouse wheel. Then eg: use PageUp key to scroll ti the top if the the "active" tiddler if it is a long tiddler. If the tiddler is short, PageUp should scroll to the top of the previous tiddler. ... and so on.

If the mouse wheel is scrolled down or up in the meantime, the focus of the tiddler needs to change.

We shouldn't re-invent how web apps work. I don't think that scrolling shouldn't change the keyboard focus.

I think ALL if this can be done already with no problems, for the hardcoded single story we have. BUT it gets extreamly hacky as soon as we start to introduce several stories.

I'll try to create a prototype, to show what I mean.

I suspect that it might be a better use of time to try to document your thinking better so that others can follow along. The implementation of this is likely to be complex and hard to follow. Given that at this point I don't fully understand the goals of your design I would find reviewing the code nearly impossible.

[pmario]

At the moment we do have several parameters that can be used to configure the behaviour of the story: #5262 (comment)

• At the moment these parameters belong to, what I call a "Story Context": story, history, openLinkFromInsideRiver and openLinkFromOutsideRiver
• Those elements are defined using the <$navigator widget.
• navigator.js internally defines: tv-story-list and tv-history-list .. So they are also part of the "Story Context"

[pmario]

@Jermolene

I'm having trouble following your meaning here. .... What should modify it?

The "story handling" should modify the $:/HistoryList-xxxx tiddler. Story handling can be done by mouse-wheel, mouse-clicks, global keyboard shortcuts and local shortcuts (inside keyboard-widget body)

... What is the "it" that modifies the history list tiddler?

"story handling"

... What magic happens?

I did some debugging some time ago and I was sure, that the $:/HistoryList "change" event will trigger the story "scroll into view" animation. ... which I called "magic". ... At the moment that's not the case. So no magic happens. ... I'll need to do a bit more code investigation here

[BurningTreeC]

I have a stupid question regarding the preservation of the focused element:

I guess we want to do this using document.activeElement ... Now, to preserve focus over a refresh cycle, should a) the information about the active element be stored in a state tiddler or b) can we store it in a global variable? And if a) - don't we need a unique identifier for each widget?

[BurningTreeC]

Hi @Jermolene , I now have a working prototype that preserves focus over refresh cycles like you proposed. My next question would be regarding how the <$keyboard> widget should know about the tiddler that has focus or focus within.

[Jermolene]

Hi @Jermolene , I now have a working prototype that preserves focus over refresh cycles like you proposed. My next question would be regarding how the <$keyboard> widget should know about the tiddler that has focus or focus within.

I don't think it needs to. The keyboard handlers for the current tiddler would be wrapped around the tiddler frame, as we do in $:/core/ui/EditTemplate, and the keyboard handlers for the current story would be wrapped around the story river.

[BurningTreeC]

Hi @Jermolene , I now have a working prototype that preserves focus over refresh cycles like you proposed. My next question would be regarding how the <$keyboard> widget should know about the tiddler that has focus or focus within.

I don't think it needs to. The keyboard handlers for the current tiddler would be wrapped around the tiddler frame, as we do in $:/core/ui/EditTemplate, and the keyboard handlers for the current story would be wrapped around the story river.

Oh, right! Thanks!

Another thing that could be problematic with this implementation: If I click the new-tiddler button, focus is on the button and will be restored to it. The new tiddler that was then created will not have focus in the title field right after that. There are more subtleties this implementation brings with it

[Jermolene]

Another thing that could be problematic with this implementation: If I click the new-tiddler button, focus is on the button and will be restored to it. The new tiddler that was then created will not have focus in the title field right after that. There are more subtleties this implementation brings with it

Yes, it's an interesting case! At the moment, the way we do this relies on the fact that if a new element is added to the DOM that has the "autofocus" attribute set then browsers automatically move the focus to it.

One could imagine giving widgets a method for them to call during render or refresh to indicate that they want to be given the focus at the end of the current render/refresh cycle. We'd need to have some mechanism for arbitrating between competing claims for the focus.

[BurningTreeC]

One could imagine giving widgets a method for them to call during render or refresh to indicate that they want to be given the focus at the end of the current render/refresh cycle. We'd need to have some mechanism for arbitrating between competing claims for the focus.

That brings me back to the point where I want to edit all affected widgets and detect within their render method if they should focus their domNode again

That would then force me to generate the renderTreeFootprint within each widget but as you mentioned - that shouldn't be the way to go

[BurningTreeC]

A little summary of my current state:

• before the refresh cycle begins I get the widget that owns the currently focused domNode
• then I generate that widgets' renderTreeFootprint that's an array of indexes
• I also store the index of the domNode in the widgets' domNodes array
• after the refresh cycle I walk the widget tree from the rootWidget using the stored indexes to identify the widget that owns the domNode that should be focused again
• but in the meantime the widget can be totally destroyed and also its parent widgets, so that walking the tree results in an undefined widget
  • there seems to be the need of using wikitext to focus elements in certain cases
  • for example when editing a tiddler and saving it, the focus will not automatically be restored to an element within the tiddler or the tiddler-frame itself
  • I haven't yet tried if it's possible to set focus to the then-to-be-created tiddler in view mode through an action on the save-tiddler button but I doubt that it works

What I'm thinking of now is how the mechanism should know which tiddler or element to focus again after the refresh if at the renderTreeFootprint there's no widget anymore

[Jermolene]

Hi @BurningTreeC that's great, it sounds like it might be worth pulling together a draft PR at this point so that we can review?

[BurningTreeC]

Hi @Jermolene , I'll create a draft PR this weekend.

[BurningTreeC]

@Jermolene - I've put together a Draft PR in #5414