Update Gutenberg > Experiments page to encourage feedback

[annezazu]

Pulling @spencerfinnell's feedback here #44214 (comment), I wanted to open an issue about updating the language in the Gutenberg Experiments page found here: https://github.com/WordPress/gutenberg/blob/6db6ffc39571ae44c3507cd4054733d4e22a8882/lib/experiments-page.php Currently, the description doesn't clarify that these features are open for feedback nor where to share that feedback. Here's the description:

The block editor includes experimental features that are useable while they're in development. Select the ones you'd like to enable. These features are likely to change, so avoid using them in production.

Proposed text

Help Test Experimental Features
The Gutenberg plugin includes experimental features that can be used while they are in active development in order to get feedback early and often. These features are very like to change. Do not use in production. Select the ones you'd like to enable and please leave any feedback you have in the Gutenberg GitHub repo. Thanks for testing!

Here's a very rough idea of what I can imagine, including a button that would link to https://github.com/WordPress/gutenberg/issues that we could always remove if somehow this leads to a flood of not helpful feedback:

@juanmaguitar @ryanwelcher @ndiego could one of you help with this?

Of note, I'd love to get into a state of iterating towards this idea in the future with the Experiments page in general, especially as phase 3 gets underway: #30922

[hanneslsm]

Agree. I only suggest using a clearer instruction:

These features are very likely to change and should not be used in production.
=
These features are very like to change. Do not use in production.

Somehow Related: On the other end of the spectrum, there is a discussion on the wording "is not advised": #54715
Maybe we should 😉 create a best practice guideline for UX writing.

[spencerfinnell]

Thank you for opening this @annezazu,

I'd like to echo the concerns outlined in #30922 (comment)

Only point I'd consider is how many users understand Github and how much of a hurdle this is. Perhaps your idea of make would be easier or a blog post to ease someone into a conversation

If someone installs Gutenberg from .org, I don't think it should be assumed they have a GitHub account, or even know what GItHub is.

Regardless, if GitHub is the direction people are pointed, I think there should be specific places to leave feedback, not a general issue. For example, if I want to leave feedback about forms, should it be in...

#44214
#44186
#36114
#36113
#36112
#38107
#53737
#44671
...etc

Using a special category in GitHub Discussions like what was added for the Interactivity API -- https://github.com/WordPress/gutenberg/discussions/categories/interactivity-api -- would allow people unfamiliar with GitHub but may still have an account to at least be able to find a centralized place to leave feedback about a specific experiment, and help keep lists like above from forming.

Another option could be a sticky thread on https://wordpress.org/support/plugin/gutenberg/ that explains more about what the experiments are, and links to individual forum threads to leave feedback, a specific blog post, or back to the GitHub discussion category.

[annezazu]

These features are very like to change. Do not use in production.

Updated the language in the main issue :)

Great insights around GitHub. I agree that folks may not have an account but we're going to run into that regardless with Trac or GitHub. I don't want perfect to get in the way of good essentially and am curious to experiment (get it) with this for now and iterate as we need to.

As for issue specific links, that feels a bit precarious as often folks require new issues are opened rather than piled on in an overview or tracking issue. We have a crew of folks who do triage too so we don't need folks to categorize themselves and can rely on the wider group to handle (myself included in that). I also know these tracking issues evolve and change overtime and I'd hate to send someone to a dead end. How does that sound to you?

[spencerfinnell]

I agree that folks may not have an account but we're going to run into that regardless with Trac or GitHub.

What about a WordPress.org account?

If there isn't an expectation that people using the experiments should have knowledge of development processes/how to use them then perhaps the experiments shouldn't be presented to all users who install the plugin from .org by default?

As for issue specific links, that feels a bit precarious as often folks require new issues are opened rather than piled on in an overview or tracking issue.

Discussions allow this. When viewing a discussion category the "New Discussion" automatically categorizes the new topic: https://github.com/WordPress/gutenberg/discussions/new?category=interactivity-api

[annezazu]

What about a WordPress.org account?

Exactly! Folks will need an account in that case.

If there isn't an expectation that people using the experiments should have knowledge of development processes/how to use them then perhaps the experiments shouldn't be presented to all users who install the plugin from .org by default?

I think that's taking a fairly extreme approach, especially when the Gutenberg plugin description clearly states the following:

"EARLY ACCESS
Are you a tech-savvy early adopter who likes testing bleeding-edge and experimental features, and isn’t afraid to tinker with features that are still in active development? If so, this beta plugin gives you access to the latest Gutenberg features for block and full site editing, as well as a peek into what’s to come."

I'd rather we focus this issue on updating the experiments page to make this clearer for any folks who land there :)

Discussions allow this.

Discussions aren't a majorly used part of the Gutenberg repo at this time. I don't want bugs and enhancements to get lost there out of the flow.

[spencerfinnell]

Exactly! Folks will need an account in that case.

My thought was that it would be more likely for someone to have a .org account when downloading a plugin from .org, than having a GitHub account when Github is only mentioned as "The development hub for the Gutenberg project" at the end of the readme.

Are you a tech-savvy early adopter who likes testing bleeding-edge and experimental features, and isn’t afraid to tinker with features that are still in active development? If so, this beta plugin gives you access to the latest Gutenberg features for block and full site editing, as well as a peek into what’s to come.

Maybe this should be updated as well? Again, there is no reference to being receptive to feedback/changes or a place to leave feedback. It could be nice to add under the "CONTRIBUTORS WANTED" section.

Discussions aren't a majorly used part of the Gutenberg repo at this time. I don't want bugs and enhancements to get lost there out of the flow.

Perhaps have the "Give Feedback" button link to a specific GitHub issue template that tags it? I understand there is triage that happens but as linked in the many related issues above, things definitely still get lost, or are not searched for/found before creating new issue.

[annezazu]

Quick note that I shared some longer thoughts on #30922 (comment) around sharing a link to a PR or overview issue for folks to learn more. These are good to think about for a v2 or v3 of this effort!

My thought was that it would be more likely for someone to have a .org account when downloading a plugin from .org, than having a GitHub account when Github is only mentioned as "The development hub for the Gutenberg project" at the end of the readme.

Unfortunately, feedback isn't gathered in a place where a WP.org account will be useful right now. In many cases, if someone were to post on trac, they'd get redirected here. I can't change that in time for this specific effort but I hear you. Just trying to deal with the reality of where feedback is most likely to be seen today.

Maybe this should be updated as well? Again, there is no reference to being receptive to feedback/changes or a place to leave feedback. It could be nice to add under the "CONTRIBUTORS WANTED" section.

Totally game to get that updated too! Let's do it.

Perhaps have the "Give Feedback" button link to a specific GitHub issue template that tags it? I understand there is triage that happens but as linked in the many related issues above, things definitely still get lost, or are not searched for/found before creating new issue.

I think we'll just have to rely on triagers as we'd have to create individual templates for every single experiment in that case which would greatly clutter up the options we already have:

I feel strongly that I'd rather have multiple reports of the same thing (which happens) than no reports.

[spencerfinnell]

I think we'll just have to rely on triagers as we'd have to create individual templates for every single experiment in that case which would greatly clutter up the options we already have:

In my opinion, there is still nothing in the screenshot above that indicates general feedback. Maybe updating Feature Request to Feature Request / Feedback 🤷🏻

Perhaps one additional template for experiments that has an issue form with a dropdown that allows a specific experiment to be selected?

See: https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms

"Bug Report" and "Bug Report (Mobile)" could also potentially be condensed in to a single template using the issue form technique linked above.

[annezazu]

In my opinion, there is still nothing in the screenshot above that indicates general feedback. Maybe updating Feature Request to Feature Request / Feedback 🤷🏻

Generally speaking, we need feedback in the form of bugs and feature requests. The mobile setup has been added to help distinguish issues there and ensure they get properly labeled (that does get a mobile label added).

With where we stand now, @ryanwelcher or @ndiego is this something you all could run with and iterate on for the experiments page part of this?

[spencerfinnell]

@annezazu to be honest, reception to feedback even in this issue has been pretty disheartening... it seems like the issue was opened with a solution in mind vs. being open to alternative ideas. I fear the same will occur when linking to PRs as you described in #30922 (comment)

I've been subscribed to this repository for the last 5 years, reading through as many of the 100+ new issues/comments a day as I can. There are many examples of contributors voicing their opinion that it is difficult to leave feedback in issues/PRs because of the amount of noise, and the fact they are often developer-focused -- and on a mission -- vs. being a place for open discussion. Seen recently:

#50891 (comment)

I feel discomfort reading the linked PRs from you, because I don't know what they all mean exactly. PRs seem for dev personal only to understand.

---

I understand the need for iteration and taking small steps, but adding a single link to a very overwhelming repository provides very little value in my opinion.

Trying GitHub discussions, or specific labels will at least allow tracking the source of feedback. If the discussions or labels don't get used, then iterate again and try something else. If the discussion/labels get used, then great, iterate faster on #30922. But linking to https://github.com/WordPress/gutenberg/issues provides no way of knowing if the experiments page is even the source of submission or if further iteration is needed.

---

If the consensus is "you should be a developer and familiar with GitHub when using the experiments" then I will reiterate my previous comment that maybe they should not be made immediately available to 50% of the web who has the opportunity to install Gutenberg with a single click. Enabling them via a constant or filter could help ensure people realizing they are "opting in" similar to private APIs

[annezazu]

to be honest, reception to feedback even in this issue has been pretty disheartening... it seems like the issue was opened with a solution in mind vs. being open to alternative ideas.

Thanks for the reflection back! I'm in the midst of juggling 6.4 responsibilities being on the release squad so am being a bit more concise than usual. I fear that's definitely impacted this effort as I'm trying to narrow the scope, iterate, and then repeat. To share a bit more, I took what you said in the prior issue I pulled your feedback from really seriously! This is my attempt to act on that, work with you, and find a middle ground. I think we don't agree on everything and I think that's okay. Please don't mistake that for me not being open to feedback. Like you, I'm trying to share my perspective and knowledge. From my point of view, we will not be able to do everything in one go though as you can see with my comment here where I'm zooming out to look even longer term while trying not to lose focus on what we can improve today: #30922 (comment) I can't guarantee we will "iterate faster" which is why making these moves need to be done carefully. For example, adding more issue templates can end up being a big deterrent and cause overwhelm (I already get that feedback from folks). When it come to the various templated issues, we've tried to keep it as simple as possible as a result: #34007 It's also outside the scope of getting some pretty solid updates in place on the Experiments page itself.

@priethor I know we've worked on the templates in the past for issues but I'm curious for your take here. I'm hesitant to create new ones as we already have quite a few and I think what is there now will still cover the bases for what one might open an issue for with Experiments, including the ever present "Open a blank issue" option. In my mind, I'd rather focus on the text on the experiments page to ensure folks know we want feedback in the form of bug reports and enhancement requests with a link that guide folks there.

But linking to https://github.com/WordPress/gutenberg/issues provides no way of knowing if the experiments page is even the source of submission or if further iteration is needed.

I don't think we need to know that that's the source and that that can be described in the issue itself.

If the consensus is "you should be a developer and familiar with GitHub when using the experiments" then I will reiterate my #55174 (comment) that maybe they should not be made immediately available to 50% of the web who has the opportunity to install Gutenberg with a single click.

I think it's less that you should be a developer and more that you should proceed with caution, developer or not, as these are features that are both likely to change and not meant for production. In general, Gutenberg itself is meant as more of early adopter tool: https://wordpress.org/news/2021/04/become-an-early-adopter-with-the-gutenberg-plugin/

Let's keep working together to refine the language on that page itself 🙌🏼 Thanks for chatting this through and taking the time.

[annezazu]

On the flip side, happy to let this stay as is so you and others can share more feedback for a larger iteration. I was definitely hoping to get a simple iteration in place sooner rather than later after reading your comments and seeing the opportunity to work with you but sounds like that's not the ideal.

If that's the case, I'd recommend we close this out and leave feedback on this issue #30922 for when a larger effort can be undertaken sometime in the future. That's definitely a larger effort and the more appropriate place to chat that through in more detail 💥.

[spencerfinnell]

@annezazu I agree -- #30922 seems more worthwhile.

[annezazu]

Sweet. Let's follow up there and close out here. Especially with phase 3, I am hopeful we can get that page renewed in a larger way. At the least, your feedback here can help influence things there. I'll leave a comment connecting the two.

[priethor]

@priethor I know we've worked on the templates in the past for issues but I'm curious for your take here. I'm hesitant to create new ones as we already have quite a few and I think what is there now will still cover the bases for what one might open an issue for with Experiments, including the ever present "Open a blank issue" option.

I'm happy to follow up in #30922, but I'm answering here specifically to the issue templates topic.

My experience trying to streamline template issues was mixed, as the feedback consensus was it's good to guide users reporting feedback but not to the point it can become too complex. Removing friction, not adding it.

In this particular case, I personally feel having a template per experiment would be overkill, as having too many options to report an issue would add friction, not reduce it. Moreover, I find it confusing mixing very widely used options like Bug report and Feature request with specific, probably much less used feature-specific options.

[richtabor]

Just sharing what Github does for their experiments:

The "Let us know" link goes to https://github.com/orgs/community/discussions/68734.

[spencerfinnell]

In this particular case, I personally feel having a template per experiment would be overkill, as having too many options to report an issue would add friction, not reduce it.

I agree as well. My original thinking was a single "Experimental Feature Feedback" template that provided a dropdown to choose which experiment you are leaving feedback for.

...but I can see how even the name of that issue template could be confusing in the grand scheme of Gutenberg where the whole plugin can be seen as "experimental" vs. the dedicated "Experiments".