Improve contributor experience by labeling complexity, ambiguity, and consensus of an issue #11788
Comments
|
Thanks for writing this all up! I feel like the more complex we make the process, the harder it will be for everyone. I'm especially unsure how we'll keep up with rating consensus. Maybe another way to look at it is that there are stages for an Issue and gates along the way
How do we make it clear what stage we are looking for help in, how difficult that stage is, and what is needed for success? |
weihanglo
changed the title
The original idea of this proposal is “maintainers add labels and contributors evaluate the risk and time if working on it”. The criteria of adding those labels need not be strictly consistent among maintainers. They are just “hints” for potential contributors. I agree the mechanism makes it harder for maintainers to triage issues, but it also eases the pain afterward if someone picks up and works on it. I love the idea of looking them from the angle of “stages”! That makes things clear to know at which stage an issue is currently blocked, as well as how to move forwards. For PR I don't think we need to highlight the bar there, as it is obvious a well-known blocker when submitting PR on an open source project. Maintainers also reserve the right to reject any PR even the purpose and design are great. So, maybe we can focus on easing pains caused by implicit blockers in problem definition and design stages? |
I was speaking from a contributor angle. They would need a single label as an entry point (and discover it), plus 3 effort labels, plus at least 1 category and 1 area label. This means there would be 6+ labels on any given issue which turns it into a soup. You could drop the entry point label but now a user has to discover and decipher 1 of 9 labels as their entry point for possible issues and then walk through the other ones until. they find an issue of interest. |
|
That's fair! I didn't think too much from that angle. It does sound horrible to have 6+ labels. I looked into top 10 most popular project by contributor counts1 on GitHub last year. Each of them has a decent label system but none of them gets such an label for different factors. Although that doesn't mean the proposal is wrong, maybe we should start small and quick. I notice that many of them have
For feature requests:
I don't know if it makes sense but at least we states stage” here. It cannot reflect how big the design work could be, though. I put Footnotes
|
|
Thanks for expanding on this! Some quick thoughts
|
I don't like mcve either; there is a slight difference between
Yes, we could do that. I was thinking about the same thing. However, I want to put
IIRC Eric follows a pattern that issues without labels mean untriaged. I am slightly on the same way. If we aim to let contributors know,
Completely agree on this.
In my mind it means |
|
Something that came up in the cargo meeting is distinguishing between
|
|
I'd like to propose the following which is a bit of a mixture of what is suggested above and my own experience with dealing with issues. My primary goal is to have a clear status for an issue, and how it can move further along. Current landscape of labelsJust to recap what is used today: Status
Experience
Importance
Priority
Categories
Misc
RFC Bot
Proposed changesAdd a clear set of statusesEvery issue should have a clear answer as to "what is the status of this" or "what is it waiting on". Usually there should only be one status label (though that is not a requirement). The following are not in a strictly linear set of steps, but are roughly the direction things flow. People should apply whichever Propose to add the following:
The If someone wants pointers on where they can help, it depends on what they are interested in:
For tracking issues, they should either be Other label changes
|
|
Thank you for summarizing this. I am pretty much sold on the idea, given it is much clearer and actionable than the original one. I am happy to manage all these. Just one question: Do all |
|
I feel like the proposal captures the core of what I was looking for (few, more obvious labels, clear path of progression). |
|
Three members here seem satifsied with Eric's propsal. I am going to do an FCP. If we generally agree on this, someone or I will update both the issue tracker and the contributor guide then. @rfcbot fcp merge I don't want to make it a poll because a poll won't show up in the RFC dashboard. |
|
Team member @weihanglo has proposed to merge this. The next step is review by the rest of the tagged team members: No concerns currently listed. Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
rfcbot
added
proposed-final-comment-period
|
One suggestion: I think the current E-help-wanted and Feature accepted labels should be turned into S-needs-mentor, not S-accepted, unless they are specifically simple or already have a Cargo team member available to help/review. |
rfcbot
added
final-comment-period
|
🔔 This is now entering its final comment period, as per the review above. 🔔 |
No, I think we can use different colors as needed.
Yea, we definitely should review the existing issues before we migrate them. In practice we have been using those as wanting someone to start on them, but we should make sure that is still the case. |
|
Since the team is happy with Eric's proposal, I am going to list the following tasks one by one. Feel free to take over any if it hasn't yet assigned to anyone.
|
Add S-triage auto-label. This automatically adds the https://github.com/rust-lang/cargo/labels/S-triage label to new issues using the template. This is intended to help make it clear that issues have not been triaged, and need some decision on the next state they should go to. cc #11788
|
Okay I've reviewed these labels and issues. I think we can move on to the migration. Hope I didn't miss anything. Click to see more
|
|
This is how new labels look like after the migration.
Dark red indicates something we want from any potential contributor. Grey means it needs efforts from the Cargo team or a wider community. Two questions:
|
I don't have a strong opinion either way. Another option is to make it orange or yellow to be somewhere in-between green and red to correlate with it being somewhere between one of the red ones and being accepted.
No strong opinion here, either. I slightly lean towards the shorter form, but either is fine with me. |
Update contributor guide with new issue labels. This updates the contributor guide with the new labels discussed in #11788, and expands on some of the meanings and process changes related to that.
|
@weihanglo Are there any other things you'd like to see done for this, or can this issue be closed out? |
|
I think that's pretty much of it. Let's see how far the new label system goes. Thank you Eric for your help! |
rfcbot
added
finished-final-comment-period
The problem
The existing labels used to call for participations do not adequately describe the complexity of involved in the progress of staring from an issue to a pull request and merging it. Contributors and maintainers often suffer from this. For instance,
#11786 and #11775 are cases I failed to tell them the consensus and complexity beforehand.
#9599 is a case that the level of design and implementation complexity were embedded in comments.
#11666 is an example that implementation might be relatively easy but need to reach a consensus first.
#11432 is an example that the contributors saw the triage and thought “hmm… does that mean feature get accepted?”
This old #2526 may need an RFC, but is probably more dying for a driver taking care of it for a long while.
Every contributor has different expectations when helping the project. Not everyone can be satisfied by completing
E-easytasks. Also, not everyone loves discussing a design back-and-forth with the risk of proposal rejection. It could save both maintainers and contributors' time if we have had a better labeling system around contributor participation.What we have today
I said the contributor experience of each issue cannot be precisely described by existing labels. However you're not sold. Let's see what we have today.
We have three levels of participation experience. It's not easy to tell if it indicates the discussion is easy or the coding part is hard.
Two labels calling out for design works. We don't know how difficult to disambiguate the requirements into a proper design.
In terms of consensus, we have:
Apart from
I-nominated. All other labels seems more like “there is a consensus” instead of “what is the level of consensus here?” or “how risky is my work to get rejected?”.Factors affecting contributor experience
Jon's original text summarizes these aspects down to three factors: complexity, ambiguity, and consensus. I copied the original words from him verbatim. Click to expand for details.
Complexity: how hard is this issue to fix/feature to implement in terms of just code adjustments.
This focuses solely on the “mechanistic” aspects of the change: how many places need to be changed, how involved are those changes, how many knock-on changes do we expect, etc. A complexity-low issue would only require changing one or two files (ideally just one or two functions) in fairly straightforward ways, or if there are more widespread knock-on changes they are entirely deterministic and could (ideally) be performed by search-replace. A complexity-high issue on the other hand is likely to require involved changes across multiple points in the code-base, and those changes are in turn likely to require non-trivial changes to other parts of the code that interface with the first set.
Ambiguity: how much design work is necessary to land the necessary change.
This is all about the degree to which the bug/feature is un-/under-specified and requires active decision-making — to what extent will a prospective contributor have to sit down and think/consider how to best fix this issue/implement this feature. An ambiguity-low issue should require very few decisions, if any, to be made. A contributor should be able to go make “the changes”, and it should be relatively well-understood (or obvious) what “the changes” are. An ambiguity-high issue on the other hand will require a signifiant exploration of the problem domain and several decisions will have to be made as part of the implementation. Those decisions need to be documented, and a rationale given, in case a subset of them need to be discussed or changed. A a contributor to such an issue is also going to have to think about implications beyond just the change they're making, such as how it interacts with other project features, backwards-compatibility, etc. Very high ambiguity issues are likely to require an RFC.
Consensus: how comfortable are the core project maintainers with the proposed path forward.
This is essentially scoring “how likely is it that this will land without too much discussion after I've done the work”. Alternatively, it can be thought of as how controversial the change is. Unlike the other two, -high here implies “easier” and -low implies “harder”. It is also generally going to be inversely proportional to the ambiguity score, but not always. A consensus-high issue is one where little discussion is likely to have to happen once a PR is made. It'll still have to go through review and such, but it's unlikely that the fundamental approach is going to be questioned or endlessly debated. A consensus-low issue on the other hand is likely to require extensive debate before landing, and has a real danger of not landing even with a perfect implementation because of considerations beyond the quality of the feature or bugfix itself. PRs for consensus-low issues will almost certainly require an FCP.
Labels indicating these factors
The proposal above proposes 9 labels for these factors:
Complexity-highComplexity-mediumComplexity-lowAmbiguity-highAmbiguity-mediumAmbiguity-lowConsensus-highConsensus-mediumConsensus-lowIt is not clear how these labels fit in the existing label system in the wider Rust community. We could probably make
E-easymeansComplexity-easy. We could also makeAmbiguity-*becomeE-ambiguity-*, so that contributors can figure out “Oh! I expected to experience more discussions if working on that.”Consensus-*may indicate that maintainers would have to get a conclusion, so they may fit inI-consensus-*category.But perhaps they should all fit in
E-*so people know they are all about contributor experience.When will it start
I do think this kind of granular labels are invaluable. It is also a two-way door decision — it can always easily be reverted. The only stuff I am not comfortable is naming and how to fit in the current system (and also choosing label colors). So, if any of the team member give me a green light. I may just try one approach and see how far it goes.
Additional information
We could also add a
E-needs-driverto indicate that we need a mid- to long-term driver for a feature/issue, though this is slightly tangent to the issue.The text was updated successfully, but these errors were encountered: