Converting documents to tutorials or templates through the UI

[fflorent]

It's possible though very technical to convert a document to a template or a tutorial: you have to make Rest API calls (PATCH /docs/{docId} with this body: {"type": "template"} , {"type": "tutorial"} or {"type": ""} if you want to convert back to a regular document)

It could be much more convenient, more user-friendly and more discoverable to allow changing the document type through the settings panel of the document.

[fflorent]

Here is a mockup proposal for the "Nature of the document":

Some notes:

• this field would be located at the "Document settings" pane;
• it should be localized;
• for the mockup, I used a regular <select> HTML element for simplicity, but it should in fact reuse the same components as the other fields above;
• When the value is changed, the document should be reloaded automatically so the changes take effet;
• I am not sure about the wordings;

[vviers]

Are there specific conditions that need to be met in order for a document to be turned into a tutorial or template ? e.g. can you make a tutorial if there is no GristDocTour table in your document ?

If so, we might need to think about helpful error messages :)

[fflorent]

e.g. can you make a tutorial if there is no GristDocTour table in your document ?

Hmm, I think so, so I would say no need for handling error or exceptional cases.

[lusebille]

As a user I would think it's a bit hidden to put that in the settings, or at least it's not enough , I may think common user journey could be. :

1. I know I want the file as a tutorial from the beginning and I want to choose that option when I'm creating the doc so when I click to 'Add new' I will have on the dropdown list ' Create a new tutorial ' or 'Create a new template '
2. I don't know firstly that my document could be a tutorial or a template but I realized later it could be useful (because I'm a Grist Genius whaou ) for me or my colleagues and I may want to share it so I. suggest adding on the dropdown list clicking on the share icon : 'Convert to a tutorial / Convert into template or Copy as a tutorial / Copy as a template [because I suppose in that case you may want to copy it'
3. I will add than me (Lucie) as a new user of Grist I was looking after a 'file' button with saving/export options but for now the closest iseems to be the sharing button

[fflorent]

Thanks for your feedback @lusebille!

I know I want the file as a tutorial from the beginning and I want to choose that option when I'm creating the doc so when I click to 'Add new' I will have on the dropdown list ' Create a new tutorial ' or 'Create a new template '

Unfortunately, at least as of today, there is currently a subtlety: you must first create a regular document and prepare your template or tutorial. Only once it is ready, you may convert it to a tutorial or a template. Also if you want to modify a tutorial or a template, you should first convert it back to a regular doc.

I know I want the file as a tutorial from the beginning and I want to choose that option when I'm creating the doc so when I click to 'Add new' I will have on the dropdown list ' Create a new tutorial ' or 'Create a new template '

Interesting. I would suggest doing that either in replacement of this issue or as a complement in another issue then, but it would be out of scope of the need as expressed at the beginning. Do you have any thoughts?

I will add than me (Lucie) as a new user of Grist I was looking after a 'file' button with saving/export options but for now the closest iseems to be the sharing button

Yeah, you make a valid point! If I understand correctly, you would have liked to find the "export" as a menu item of a "file" button.
There is an UX improvement to make, these issues are related: #754, #951, and maybe #540

Maybe we could create a meta-issue to track that? But also out of scope here (which does not mean it is not interesting!)

[dsagal]

BTW, it may be worth keeping in mind, as far as what's the common user journey, that very few documents are templates, and even fewer are tutorials. And few users would be creators of templates or tutorials, compared to all users creating Grist docs. On the other hand, these numbers will increase if this is made more convenient (but it would still be relatively uncommon).

[fflorent]

Working on this with @hexaltation

[hexaltation]

@dsagal @paulfitz @lusebille @vviers
https://github.com/gristlabs/grist-core/assets/31125573/ec9e08d0-a31b-451a-ad5f-d2cf3acd3f8c
Does the demo behind the link makes sense for you?
If so I will make a MR according this behavior.

[lusebille]

@dsagal @paulfitz @lusebille @vviers https://github.com/gristlabs/grist-core/assets/31125573/ec9e08d0-a31b-451a-ad5f-d2cf3acd3f8c Does the demo behind the link makes sense for you? If so I will make a MR according this behavior.

I understand what is happening but could it be possible to have an anchor to the dropdown and not have this 'jump effect' when you're choosing a value ?

[dsagal]

Hi @hexaltation ! The demo makes sense. As far as the UI, I'd like to request a change. "Document Type" is not self-explanatory, and many users wouldn't know what it means. It would even be hard to figure out through experimentation.

So.... to make it clearer, I suggest instead of a dropdown, a button that opens a larger modal, with each option explained, like for Formula Timer, with the radio buttons in the modal. I suggest calling option "Template mode", and the button could be "Change type".

The explanations could be:

• Regular document: Regular document behavior, all users work on the same copy of the document.
• Template: Document automatically opens in fiddle mode. It is editable even to viewers, with the first edit causing a new unsaved copy of the document to be created.
• Tutorial: Document automatically opens with a new copy.

(The last part, I am actually unsure about, it may need a different explanation.)

At the bottom, there would be a "Change type" and "Cancel" buttons.

What do you think?

1. The description could still be "Convert the document", with the three options "Regular", "Template", "Tutorial", like you made. One minor comment here is to use a plain dropdown (as for "Python version used" e.g. here) rather than an autocomplete.
2. Once another option is selected, open a modal confirmation dialog, also similar to the one for "Python version used", but with a longer description (like for "Reload data engine" option)

[lusebille]

To discuss , here are the proposals :
https://www.figma.com/design/wcpetFt6aOKzTszcvPPWLQ/%5B05%2F24%5D-Grist-Design?node-id=559-80548&t=SbtMMmgw2rANLbqL-1

[lusebille]

Ready from a designed point of view @hexaltation
https://www.figma.com/design/wcpetFt6aOKzTszcvPPWLQ/%5B05%2F24%5D-Grist-Design?node-id=559-80548&t=1gSxqwNd6ZsiXdFO-1

[dsagal]

Design looks good -- thank you @lusebille ! I added some comments in Figma, to clarify the text and to point to how we show notification toasts.

[dsagal]

I added a comment in figma, but adding here as well, to update the text in the popup:

• Title: "Change document mode"
• "Regular document" -- this is fine ("Regular document behavior, all users work on the same copy of the document.")
• "Template" -- "Document automatically opens in fiddle mode. Anyone may edit, which will create a new unsaved copy."
• "Tutorial" -- "Document automatically opens as a user-specific copy."

(Cc'ing @jr-grist)

[jr-grist]

@dsagal Chiming in here with two proposed changes –

(1) Instead of titling the dialog "Change document mode", I recommend "Change document type" (or "Change type of document" – indifferent on that one) Reason being: a document "type" better aligns with the language that the end user of a shared document would be familiar with, as opposed to "mode" which has more advanced/technical connotations within Grist. Would be best to align the language being used by creators/editors and viewers/invited or new users.

(2) Under the "Regular document" option, I recommend changing the language as follows: "Regular document -- Normal document behavior**.** All users work on the same copy of the document." Small tweak to sound more natural.

Looks good otherwise.

[georgegevoian]

Reopening, since #1181 was reverted due to an issue with the update API call failing in our testing of Grist SaaS.

@hexaltation, part of the issue was how the call to update the document was being made. We confirmed that the diff below addresses the issue with the failing call:

diff --git a/app/client/ui/DocumentSettings.ts b/app/client/ui/DocumentSettings.ts
index ff583067e..cf99b923e 100644
--- a/app/client/ui/DocumentSettings.ts
+++ b/app/client/ui/DocumentSettings.ts
@@ -29,7 +29,7 @@ import {commonUrls, GristLoadConfig} from 'app/common/gristUrls';
 import {not, propertyCompare} from 'app/common/gutil';
 import {getCurrency, locales} from 'app/common/Locales';
 import {isOwner, isOwnerOrEditor} from 'app/common/roles';
-import {DOCTYPE_NORMAL, DOCTYPE_TEMPLATE, DOCTYPE_TUTORIAL, DocumentType, persistType} from 'app/common/UserAPI';
+import {DOCTYPE_NORMAL, DOCTYPE_TEMPLATE, DOCTYPE_TUTORIAL, DocumentType} from 'app/common/UserAPI';
 import {
   Computed,
   Disposable,
@@ -333,7 +333,7 @@ export class DocSettingsPage extends Disposable {
       const selected = Observable.create<DocTypeOption>(owner, currentDocTypeOption);
 
       const doSetDocumentType = async () => {
-        const docId = docPageModel.currentDocId.get();
+        const docId = docPageModel.currentDocId.get()!;
         let docType: DocumentType;
         if (selected.get() === DocTypeOption.Regular) {
           docType = DOCTYPE_NORMAL;
@@ -342,7 +342,7 @@ export class DocSettingsPage extends Disposable {
         } else {
           docType = DOCTYPE_TUTORIAL;
         }
-        await persistType(docType, docId);
+        await docPageModel.appModel.api.updateDoc(docId, {type: docType});
         const {trunkId} = docPageModel.currentDoc.get()!.idParts;
         window.location.replace(urlState().makeUrl({
           docPage: "settings",
diff --git a/app/common/UserAPI.ts b/app/common/UserAPI.ts
index f58f05b8d..d64e10e56 100644
--- a/app/common/UserAPI.ts
+++ b/app/common/UserAPI.ts
@@ -135,16 +135,6 @@ export const DOCTYPE_TUTORIAL = 'tutorial';
 // null stands for normal document type, the one set by default at document creation.
 export type DocumentType = 'template'|'tutorial'|null;
 
-export function persistType(type: DocumentType, docId: string|undefined) {
-  docId = docId?.split("~")[0];
-  return fetch(`/o/docs/api/docs/${docId}`, {
-    method: 'PATCH',
-    headers: {"Content-Type": "application/json"},
-    credentials: 'include',
-    body: JSON.stringify({type})
-  });
-}
-
 // Non-core options for a document.
 // "Non-core" means bundled into a single options column in the database.
 // TODO: consider smoothing over this distinction in the API.

I haven't followed the development of this feature closely, but one question I had was what the intended behavior of changing the document mode of a fork was? During testing, we weren't able to successfully round-trip a regular document to a tutorial and back (after applying the patch above). I imagine it's because the code above is not updating the trunk document's type when called from a fork document - was that the intended behavior?

[hexaltation]

Hello @georgegevoian

For the fork to normal conversion the user story is :

I'm a builder, I need to update the tutorial/template that is already shared with a lot of people having a link to it. I click on it, I go to document settings, I convert it back to normal to make persistent changes. When done I transform it back to tutorial.

If it still don't make any sense, can you enlight me on what is the attended journey for such task?
Cause your question make me feel that what was developed was maybe not a good idea.

[fflorent]

Waiting for the internal tests that were failing to be open-sourced. Otherwise it's hard for us to work on this feature.

[paulfitz]

Waiting for the internal tests that were failing to be open-sourced. Otherwise it's hard for us to work on this feature.

@georgegevoian @dsagal do you have some details on the failing tests?

[paulfitz]

Got some details. The failing tests were: DocTypeConversion.ts - second test, as soon as it starts trying to convert anything. I'll work on replicating this. I assume the reason for failures were the tests running in a somewhat different environment, with different URLs. I have a second piece of feedback on that part:

Instead of using UserAPI’s updateOrg, it does a fetch() to some manually constructed URL. That URL uses the wrong hostname, which may work in some configurations and not others. That’s where the test fails. But also we have a way that all frontend code uses to call the API; the code should use that, not construct a manual fetch?

So @fflorent the test is already open source. I'll try to figure out a way to run it to tickle the problem. You might already see the problem if you try running the code on your SaaS.

[georgegevoian]

Hello @georgegevoian

For the fork to normal conversion the user story is :

I'm a builder, I need to update the tutorial/template that is already shared with a lot of people having a link to it. I click on it, I go to document settings, I convert it back to normal to make persistent changes. When done I transform it back to tutorial.

If it still don't make any sense, can you enlight me on what is the attended journey for such task? Cause your question make me feel that what was developed was maybe not a good idea.

One problem with this approach is that during the time you're making changes to the tutorial or template, anyone opening it will also open it as a regular document.

A solution to this is to make the changes in a fork, and replace the original document. This should work for both tutorials and templates, as both should have a Replace Original... item in the Share menu.

There's also the /m/default modifier that you can append to a document URL to force it to open as a regular document. You can do this as well to modify a tutorial or template without interrupting other users, though unlike the previous approach, it doesn't bundle all changes into a single "transaction", so it's not ideal for when you need to modify multiple things in the original document.

Are there other stories that require the document type to be modifiable from within a fork? If not, I wonder if it's best to not show it in those cases. A fork document modifying the trunk document should ideally only happen when you perform an operation like replace original (which explicitly warns you). It's not clear from the current UI that this is what will happen if you change the document type of a fork. Maybe it's best to only show the option when you're on the trunk, and require temporarily switching the document mode (e.g. /m/default) to make such changes? I admit /m/default isn't very discoverable - perhaps that can be a separate improvement?

[paulfitz]

I'll try to figure out a way to run it to tickle the problem. You might already see the problem if you try running the code on your SaaS.

I think the reason the failure isn't yet seen in grist-core is that multi-server testing isn't done yet for the github CI (needs port allocation that works with parallel mocha environment) https://github.com/gristlabs/grist-core/blob/main/test/nbrowser/testServer.ts#L128-L143

An awkward way to replicate the problem:

REDIS_URL=redis://localhost NODE_PATH=$PWD/_build:$PWD/_build/stubs HOME_PORT=9801 STATIC_PORT=9802 DOC_PORT=9803 DOC_WORKER_COUNT=5 node ./_build/app/server/devServerMain.js

Visit Grist on port 8080. Make a doc, change it to tutorial, watch front-end browser logs. There's be a patch that 404s, and the document type won't persist.

The port numbers simulate different parts of Grist being on different machines.