ParallelDOM: New high level API for descriptive text

[kathy-phet]

When an accordian box contains a complex node that isn't available in the pdom, we should be able to put a descriptive paragraph there in the pdom - similar to the way alt-text is used for images.

[kathy-phet]

Example of where this would be used, if available, is in MOTHA electron levels:

phetsims/models-of-the-hydrogen-atom#85

[jessegreenberg]

I don't think we should add options to AccordionBox (and other content containers) for this. I think that the "alt-text" content should be implemented on the content Node of the AccordionBox.

With the low level API, that looks something like

energyLevelDiagram.tagName = 'p';
energyLevelDiagram.innerContent = 'This is a description of the energy level diagram...';

More broadly, we should think about the high level API for this for any Node. We could keep using accessibleName

energyLevelDiagram.accessibleName = 'This is a description of the energy level diagram...';

Or maybe we will create something new

energyLevelDiagram.altText = 'This is a description of the energy level diagram...';

I don't see a scenery issue for deciding on the high level API for non-interactive content. I would like to create that in scenery and close this issue. @kathy-phet is that OK with you?

[kathy-phet]

Making an issue in scenery for the high level API that can serve this type of function sounds good, thanks! accessiblename doesn’t match the goal of providing a longer description (could be multiple sentences) of what is visible in the node. I like the api name you suggested. Discussion can continue over in the scenery issue.

…

________________________________ From: Jesse Greenberg ***@***.***> Sent: Thursday, November 14, 2024 1:27:21 PM To: phetsims/sun ***@***.***> Cc: Kathy Perkins ***@***.***>; Assign ***@***.***> Subject: Re: [phetsims/sun] Accordian Box: Add API for passing descriptive paragraph (Issue phetsims/scenery#1673) [External email - use caution] Assigned phetsims/scenery#1673<#1673> to @kathy-phet<https://github.com/kathy-phet>. — Reply to this email directly, view it on GitHub<#1673>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABG4KZEOMGZ4P5K3LTFEIHT2AUBSTAVCNFSM6AAAAABRZQZ7XKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVGMYDIMBTGYZTINY>. You are receiving this because you were assigned.Message ID: ***@***.***>

[kathy-phet]

Maybe you could just transfer this issue and re-title it.

[jessegreenberg]

Sounds great. The issue has been transferred and I updated the title to reflect this idea.

[terracoda]

I think we need to approach how we refer to this kind of description (essentially a Static State description) differently, and understand that the description may need more structure than just a paragraph. See Molecules and Light's Light Spectrum Diagram as a good example. This diagram is implemented in an non-modal dialog, but it could exist in an accordion box, or in a modal dialog.

I think we could be general and just call it a "description" or an "accessibleDescription", and the API needs to offer document structures in addition to a paragraph.

Could "accessibleDescription" be the higher-level API, defaulting to a paragraph, but offering devs an easy way to add document structures for all sorts of descriptions. It would be the responsibility of the description designer to indicate what structures are needed for a complex description like in the case of the Energy Spectrum Diagram.

[terracoda]

I think this Buoyancy commit is related to making additional information and/or RichText accessible. Important sim content that is outside of names and help text.

https://github.com/phetsims/density-buoyancy-common/blob/6ba45b6a5fdbece5cba9590f520571a529697b35/js/buoyancy/view/ReadoutListAccordionBox.ts#L127-L130

[kathy-phet]

Yes, seems useful for @jessegreenberg to look at. Perhaps this provides ideas about an higher level optional API piece for RichText in general?

[terracoda]

This same kind of text (technically dynamic state descriptions) is also in the Values accordion box in Trig Tour

We want it to be straightforward to make screen text accessible. For the Voicing feature, this non-interactive but still dynamic on-screen text would need to be made into reading blocks in order to be voiced.

[jessegreenberg]

I think this Buoyancy commit is related to making additional information and/or RichText accessible. Important sim content that is outside of names and help text.

That is an example of the low level API, using tagName and innerContent. I thought we wanted something that was more high level than that - like a single option we could use for all Nodes for "descriptive text", that is neither "accessible name" or "help text".

[jessegreenberg]

We are going to try this out in the context of trig-tour as we need it for the rows fo content in this AccordionBox:

[kathy-phet]

Yes we want the high level API.

[jessegreenberg]

I have boilerplate ready for a new high level function in ParallelDOM. We should meet to discuss:

• What should it be called?
• What should be the default behavior (elements in the PDOM)?
• How should it behave with other high level functions? Can you set both an accessibleName and this new "descriptive text" on a component at the same time? Or is that a use case for "accessibleName" and "helpText"?

Patch with this:

Subject: [PATCH] Sound drag listeners optionally take a TSoundPlayer instead of creating one internally, see https://github.com/phetsims/scenery-phet/issues/889
---
Index: scenery/js/accessibility/pdom/ParallelDOM.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/scenery/js/accessibility/pdom/ParallelDOM.ts b/scenery/js/accessibility/pdom/ParallelDOM.ts
--- a/scenery/js/accessibility/pdom/ParallelDOM.ts	(revision 26c7e12c241702eee9d00507f86f8a3ab4a73b1e)
+++ b/scenery/js/accessibility/pdom/ParallelDOM.ts	(date 1734048675155)
@@ -208,6 +208,8 @@
   'accessibleNameBehavior',
   'helpText',
   'helpTextBehavior',
+  'accessibleDescription',
+  'accessibleDescriptionBehvaior',
   'pdomHeading',
   'pdomHeadingBehavior',
 
@@ -572,6 +574,12 @@
   // Function that returns the options needed to set the appropriate accessible name for the Node
   private _accessibleNameBehavior: PDOMBehaviorFunction;
 
+  // Sets the 'Accessible Description' for the Node.
+  private _accessibleDescription: PDOMValueType | null = null;
+
+  // Function that returns the options needed to set the appropriate accessible description for the Node
+  private _accessibleDescriptionBehavior: PDOMBehaviorFunction
+
   // Sets the help text of the Node, this most often corresponds to description text.
   private _helpText: PDOMValueType | null = null;
 
@@ -660,6 +668,7 @@
 
     this._accessibleNameBehavior = ParallelDOM.BASIC_ACCESSIBLE_NAME_BEHAVIOR;
     this._helpTextBehavior = ParallelDOM.HELP_TEXT_AFTER_CONTENT;
+    this._accessibleDescriptionBehavior = ParallelDOM.BASIC_ACCESSIBLE_DESCRIPTION_BEHAVIOR;
     this._headingLevel = null;
     this._pdomHeadingBehavior = DEFAULT_PDOM_HEADING_BEHAVIOR;
     this.pdomBoundInputEnabledListener = this.pdomInputEnabledListener.bind( this );
@@ -892,6 +901,64 @@
     }
   }
 
+  public setAccessibleDescription( accessibleDescription: PDOMValueType | null ): void {
+    if ( accessibleDescription !== this._accessibleDescription ) {
+      if ( isTReadOnlyProperty( this._accessibleDescription ) && !this._accessibleDescription.isDisposed ) {
+        this._accessibleDescription.unlink( this._onPDOMContentChangeListener );
+      }
+
+      // If there is no tag name yet, give this Node one so that it appears in the DOM.
+      if ( this._tagName === null ) {
+        this.tagName = 'p';
+      }
+
+      this._accessibleDescription = accessibleDescription;
+
+      if ( isTReadOnlyProperty( accessibleDescription ) ) {
+        accessibleDescription.lazyLink( this._onPDOMContentChangeListener );
+      }
+
+      this.onPDOMContentChange();
+    }
+  }
+
+  public set accessibleDescription( accessibleDescription: PDOMValueType | null ) { this.setAccessibleDescription( accessibleDescription ); }
+
+  public get accessibleDescription(): string | null { return this.getAccessibleDescription(); }
+
+  /**
+   * Get the accessible name that describes this Node.
+   */
+  public getAccessibleDescription(): string | null {
+    if ( isTReadOnlyProperty( this._accessibleDescription ) ) {
+      return this._accessibleDescription.value;
+    }
+    else {
+      return this._accessibleDescription;
+    }
+  }
+
+  public setAccessibleDescriptionBehavior( accessibleDescriptionBehavior: PDOMBehaviorFunction ): void {
+
+    if ( this._accessibleDescriptionBehavior !== accessibleDescriptionBehavior ) {
+
+      this._accessibleDescriptionBehavior = accessibleDescriptionBehavior;
+
+      this.onPDOMContentChange();
+    }
+  }
+
+  public set accessibleDescriptionBehavior( accessibleDescriptionBehavior: PDOMBehaviorFunction ) { this.setAccessibleDescriptionBehavior( accessibleDescriptionBehavior ); }
+
+  public get accessibleDescriptionBehavior(): PDOMBehaviorFunction { return this.getAccessibleDescriptionBehavior(); }
+
+  /**
+   * Get the help text of the interactive element.
+   */
+  public getAccessibleDescriptionBehavior(): PDOMBehaviorFunction {
+    return this._accessibleDescriptionBehavior;
+  }
+
   /**
    * Remove this Node from the PDOM by clearing its pdom content. This can be useful when creating icons from
    * pdom content.
@@ -3278,6 +3345,14 @@
     }
     return options;
   }
+
+  public static BASIC_ACCESSIBLE_DESCRIPTION_BEHAVIOR( node: Node, options: ParallelDOMOptions, accessibleDescription: PDOMValueType ): ParallelDOMOptions {
+    if ( PDOMUtils.tagNameSupportsContent( node.tagName! ) ) {
+      options.innerContent = accessibleDescription;
+    }
+
+    return options;
+  }
 
   /**
    * A behavior function for accessible name so that when accessibleName is set on the provided Node, it will be forwarded
Index: scenery/js/accessibility/pdom/PDOMPeer.js
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/scenery/js/accessibility/pdom/PDOMPeer.js b/scenery/js/accessibility/pdom/PDOMPeer.js
--- a/scenery/js/accessibility/pdom/PDOMPeer.js	(revision 26c7e12c241702eee9d00507f86f8a3ab4a73b1e)
+++ b/scenery/js/accessibility/pdom/PDOMPeer.js	(date 1734048529814)
@@ -205,6 +205,11 @@
       assert && assert( typeof options === 'object', 'should return an object' );
     }
 
+    if ( this.node.accessibleDescription !== null ) {
+      options = this.node.accessibleDescriptionBehavior( this.node, options, this.node.accessibleDescription, callbacksForOtherNodes );
+      assert && assert( typeof options === 'object', 'should return an object' );
+    }
+
     // create the base DOM element representing this accessible instance
     // TODO: why not just options.focusable? https://github.com/phetsims/scenery/issues/1581
     this._primarySibling = createElement( options.tagName, this.node.focusable, {
Index: trig-tour/js/trig-tour/view/readout/CoordinatesRow.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/trig-tour/js/trig-tour/view/readout/CoordinatesRow.ts b/trig-tour/js/trig-tour/view/readout/CoordinatesRow.ts
--- a/trig-tour/js/trig-tour/view/readout/CoordinatesRow.ts	(revision e484b1a294522095e9137fb508a5ef1bafef2c25)
+++ b/trig-tour/js/trig-tour/view/readout/CoordinatesRow.ts	(date 1734048688519)
@@ -53,6 +53,8 @@
 
     this.trigTourModel = trigTourModel;
 
+    this.accessibleDescription = 'This is a row of the readout that displays the coordinates of the point on the unit circle.';
+
     // initialize fonts for this row
     const fontInfo = { font: DISPLAY_FONT, fill: TEXT_COLOR };
     const largeFontInfo = { font: DISPLAY_FONT_LARGE, fill: TEXT_COLOR };

[terracoda]

I'm sketching out this example in psuedo HTML code

BUTTON + H3: {{Cosine}} Values
DIV

what is the name we need for content like this?
P: (x,y) = {{value01}}
P: angle = {{value02}} degrees
P: {{cos}} = {{value03}}

H4: Angle Units
UL
LI: degrees (accessible name for radio)
LI: radians

Additional question: Can the high-level API for accessible name and help text for accordionbox and radiobutton group be used along side this other yet-to-be-named high-level API?

Somewhere along the design/implementation cycle, someone needs to choose the HTML structure for the content (static or dynamic) that is displayed as on-screen text or as an on-screen diagram.

Name idea brainstorming (just ideas):

• Accessible Description for onscreen text
• Accessible Description for onscreen Image, Diagram, or Chart
• Accessible Description for non-interactive visual content
  In all three cases, structure for the accessible descriptions will be needed. We could default to a Paragraph fro short descriptions, or default to a list for meaningful groups of related content.

Adding:

• Accessible Description for visible content

[jessegreenberg]

@terracoda @kathy-phet and I met today to discuss this. We decided on this:

• setAccessibleParagraph( content ): A high level API setter that will make the Node a p element with the provided content.
• setAccessibleList( content[], type ): A high level API setter that will make the Node an 'ul' or an 'ol' and then add list items for each content within it.

In my patch above I had started with accessibleDescription, but that provides too much flexibility. This setter should ALWAYS create a paragraph, and if that is not sufficient then you need to use the lower level functions.

I am going to add these and then we can bring it to a dev meeting for discussion.

We also discussed that #855 is an important part of the high level API.

[jessegreenberg]

Here is a patch for accessibleParagraph I will finish and commit on Monday:

Subject: [PATCH] Sound drag listeners optionally take a TSoundPlayer instead of creating one internally, see https://github.com/phetsims/scenery-phet/issues/889
---
Index: js/accessibility/pdom/ParallelDOM.ts
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/js/accessibility/pdom/ParallelDOM.ts b/js/accessibility/pdom/ParallelDOM.ts
--- a/js/accessibility/pdom/ParallelDOM.ts	(revision 26c7e12c241702eee9d00507f86f8a3ab4a73b1e)
+++ b/js/accessibility/pdom/ParallelDOM.ts	(date 1734130703671)
@@ -208,6 +208,7 @@
   'accessibleNameBehavior',
   'helpText',
   'helpTextBehavior',
+  'accessibleParagraph',
   'pdomHeading',
   'pdomHeadingBehavior',
 
@@ -572,6 +573,9 @@
   // Function that returns the options needed to set the appropriate accessible name for the Node
   private _accessibleNameBehavior: PDOMBehaviorFunction;
 
+  // Sets the 'Accessible Paragraph' for the Node.
+  private _accessibleParagraph: PDOMValueType | null = null;
+
   // Sets the help text of the Node, this most often corresponds to description text.
   private _helpText: PDOMValueType | null = null;
 
@@ -892,6 +896,37 @@
     }
   }
 
+  /**
+   * TODO - document this API
+   */
+  public setAccessibleParagraph( accessibleParagraph: PDOMValueType | null ): void {
+    assert && assert( this.tagName === null || this.tagName === 'p', 'accessibleParagraph can only be set on a Node with a p tag' );
+    if ( accessibleParagraph !== this._accessibleParagraph ) {
+
+      // Forward to the lower level API.
+      this.tagName = 'p';
+      this.innerContent = accessibleParagraph;
+
+      this._accessibleParagraph = accessibleParagraph;
+    }
+  }
+
+  public set accessibleParagraph( accessibleParagraph: PDOMValueType | null ) { this.setAccessibleParagraph( accessibleParagraph ); }
+
+  public get accessibleParagraph(): string | null { return this.getAccessibleParagraph(); }
+
+  /**
+   * Get the accessible name that describes this Node.
+   */
+  public getAccessibleParagraph(): string | null {
+    if ( isTReadOnlyProperty( this._accessibleParagraph ) ) {
+      return this._accessibleParagraph.value;
+    }
+    else {
+      return this._accessibleParagraph;
+    }
+  }
+
   /**
    * Remove this Node from the PDOM by clearing its pdom content. This can be useful when creating icons from
    * pdom content.
@@ -1133,6 +1168,11 @@
   public setTagName( tagName: string | null ): void {
     assert && assert( tagName === null || typeof tagName === 'string' );
 
+    // For the higher level API, if using accessibleParagraph the tag name must be 'p'.
+    if ( this._accessibleParagraph ) {
+      assert && assert( tagName === 'p' || tagName === null, 'accessibleParagraph can only be set on a Node with a p tag' );
+    }
+
     if ( tagName !== this._tagName ) {
       this._tagName = tagName;