Feedback on the "tidyup" process for making big changes to the tidyverse

[hadley]

@clauswilke, @dgrtwo, @dpseidel, @karawoo, @krlmlr, @LucyMcGowan, @paleolimbot, @smbache, @vspinu, @yutannihilation, @apreshill, @jjallaire, @colearendt, @dcossyleon, @edgararuiz-zz, @evanmiller, @kohske, @markfairbanks, @mgirlich

We’d love to get your thoughts on https://github.com/tidyverse/tidyups/blob/main/001-tidyup-process.md, a proposal for a process for making bigger changes to the tidyverse. Please feel free to contribute however you feel comfortable — you're welcome to make small fixes via PR, comment here, or open bigger discussion topics in an issue. If there are things you’d prefer to discuss in private, please feel free to email me. We’ll plan to close the discussion on August 13, so we can review and make adjustments as needed.

[clauswilke]

I think you need a three letter acronym, such as TIP (tidyverse improvement proposal).

(Not a serious comment. I think it's a great idea, and the proposed framework is as good as any to start.)

[LucyMcGowan]

This is awesome, I don’t think the name is too cutesy (just cutesy enough!). One thought for soliciting feedback is some platform that would allow for up/down voting comments could be nice so you don’t end up with a bunch of people all saying the same thing / it would be easier for someone to wade through to see what has already been said. I don’t have a good suggestion for what that platform might be though.

[karawoo]

I like the proposal! What's the process for deciding whether to move forward with a tidyup? Sounds like there will be some group discussion, but if there's disagreement there or the implementation reveals unexpected complications, how does the final call get made?

[hadley]

@LucyMcGowan interesting idea! One challenge is that I'm not sure popularity will necessarily help much here — I think it's more often some corner case that we haven't considered that makes us go hmmmmmmm and re-think our approach.

@karawoo it feels like in most cases, reaching consensus is relatively straight forward, but if we get stuck for some reason, I'll perform my usual role of making some decision (even if wrong) to keep things moving.

[paleolimbot]

This is awesome!

I'm wondering how (or if) this is related to the idea of a "roadmap". I know roadmaps take a lot of overhead to maintain and probably don't apply to all packages...I just think about it with respect to something like ggplot2 where (1) one can't just review the open issues to get an idea of future directions/what to work on if you have a few hours free and (2) PRs get stuck because there's some overarching direction (e.g., we're planning to rewrite the guides using ggproto) that the PR author didn't know about (and had no way to find out). Or perhaps ggplot2 is a bad example because it's so huge? These are just thoughts!

[yutannihilation]

The proposal looks good to me in general! I like Rust's RFC culture, and I hope the tidyup process will work in a similar way.

Some random comments:

• The process should specify when the PR made on step 2 should be merged (maybe between step 3 and 4?)
• While the process expects the main place for discussion is an issue on tidyverse/tidyups, the discussion might diverge between the issue and the corresponding implementation PR on other repo. While it's unrealistic to prohibit the direct comments on the impl PR, it might be nice to have some general direction or recommendation.
• I think "Open issues" section will contain two types of issues; (1) what should be resolved until the process will be completed, and (2) what won't be resolved by the proposal and might need to be addressed in future. It might be good to have separate subsections or sections for each (c.f. Rust's RFC typically has "Unresolved questions" section for the latter type).
• It might be nice for the proposal to have "Terminology" section. I expect such proposals often introduce new concepts that are unfamiliar to the community. For example, I think the tidyup 1 proposal should define what "the tidyverse team" is.

[hadley]

@paleolimbot I think it's related in the sense that we're trying to do more stuff in public. But roadmaps are hard because there's little benefit to hitting a milestone (vs just announcing when it's done), but there's a large cost to missing a milestone. But I don't think the things that you mention would ever rise to the level of a whole tidyverse roadmap so I think it's inevitable that there will have to be discussion before starting any bigger project.

@yutannihilation I'd forgotten that Rust does this a lot too, so thanks for the reminder. I'll read through their process and reflect on what we can learn from it. Thanks for the other comments — I'll incorporate as needed when I take another pass at it.

[krlmlr]

I like it.

Another example from the recent past: moving generics from tibble to pillar. This worked entirely without breaking changes, next time we'd do a tidyup here? OTOH, the technique should be portable to other scenarios, is this repo still a good place for it?

What are the conditions under which breaking changes are strictly necessary? One example is r-lib/pillar#334 where we add an ellipsis to existing generics in pillar. I suspect that these situations are rare, and that in most cases we can work with soft-deprecation in the individual packages to achieve the desired overall outcome. Should the proposal include a sequence of steps (=package updates) and perhaps a proposed timeframe?

In general I think it helps if the dev versions of all affected packages are not too far away from the CRAN versions before work starts on a feature that spans multiple packages.

[dgrtwo]

Looks like a great idea to me. Only suggestion is

set a date when the review period will end

could use guidelines for what an appropriate review period is.

(I think "two weeks" is a useful anchor: lines up with agile sprint boundaries, guideline for expecting reverse dependencies to be fixed, CRAN deadlines for fixing failing packages... but I could also understand guidelines that scale with the complexity of the tidyup).

[hadley]

@krlmlr I think if there are no user visible breaking changes, we'd be less likely to do it. However, it might still be necessary if a bunch of package developers would have to make changes. I think we want to reserve tidyups for the biggest issues, at least initially.

@dgrtwo yeah, I was thinking two weeks minimum, and probably four weeks (or longer?) for bigger issues.

Also came across https://tools.ietf.org/id/draft-resnick-on-consensus-01.html today, so I'll probably use the term "rough consensus" and link to that doc for more description.

[hadley]

Some insights from reading Rust RFCs + IETF rough consensus:

• RFCs are numbered according the PR number
• They're much less structured than PEPs, and feel like a better fit for the tidyverse
• Nice alignment between unresolved questions and the rough consensus model — we don't have to answer every objection, but we need to at least record
• Governance RFC
• I wish there were more connections between RFCs — i.e. I can't see how to follow the thread of RFCs that affect the RFC process.

[yutannihilation]

IETF rough consensus

Good that you find this! While I agree we can learn from this, I'd like to add that network protocol is a very different beast. There are reasons why it needs to be "rough" consensus, in my understanding.

• There might be conflict of interest between vendors, or between vendors and users. For example, ad tech companies might refuse to accept some idea for protecting users' privacy. In this case, the consensus will probably be a compromise to both sides, so it can be rough.
• There's no single implementation for a protocol (tidyverse has only one implementation). The developers and vendors might want to implement it differently, so they need to agree to disagree (sorry, they don't disagree) on some specific details.
• While there's multiple implementations, it's often hard to avoid using the protocol itself (R users can always choose not to use tidyverse). For example, we can't imagine we use internet without DNS or HTTP.
• This seldom happens, but the developers and vendors can choose to violate the accepted RFC. While it's not recommended, I think it has happened several times around TCP in history. For a recent example, this article says Google doesn't implement TCP Fast Open RFC properly, while it originates from them.

[dpseidel]

I think the motivation behind this is great and the framework as outlined seems solid! My only comment is in regards to the "Community Feedback" step. Given a primary goal for this process is wider community insight and feedback, have you thought about ways beyond Github to publicize this process or individual tidyups? It seems completely reasonable to keep the discussion in a separate tidyverse/tidyup issue as proposed, and that the Champion/Co-Champion of the tidyup be responsible for tagging community members that might be most affected, but I could imagine that for some of these really big changes, we want to be getting feedback from a wider audience than is natural to tag in an issue. Perhaps this doc should include suggested paths for communicating a new tidyup is open to community feedback (e.g. Twitter? the Tidyverse Blog? Rstudio Commuity?). I could see it being useful to have a similar doc communicating best practices when contributing feedback to a tidyup. Such a doc could also encourage contributors to actively engage their networks for wider dissemination.

Admittedly, It seems likely that these tidyup documents will be fairly technical in nature and relevant primarily to other developers so it's entirely possible this is a problem that solves itself as we socialize and use this framework, or not a problem at all because the community we hope to reach is small and engaged enough already. However, as one of the primary goals is to be transparent with the larger community, how best to encourage uptake and engagement, while keeping feedback manageable and productive, seems like something worth giving further thought to.

[mgirlich]

I really like the proposed idea. Not exactly part of the tidyup process but it would be nice to make the tidyup documents easy to discover as they could be a great learning resource for package developers, contributors, or just for the interested user. Depending on the feature it might make sense to link to it in the NEWS file, in the documentation of a function or on the homepage.

[hadley]

@dpseidel yeah, figuring out how to reach the wider community is still really an open issue. Our plan is to start small with these more technical issues, and then as we get to topics of broader interest we'll think more about how we communicate more widely. I think we'll probably publish the more important topics on the blog (like how we did for magrittr 2.0), but I still don't have a good sense of how to collect feedback from a larger group of stakeholders.

@mgirlich yeah absolutely. In the longer run, we'll also make a proper tidyups website, connect it to http://design.tidyverse.org/, etc etc.

[hadley]

@yutannihilation I like rough consensus because consensus feels a bit too constraining. It's often not possible to get everyone to agree on the "best" path, but it's often easier to get people to agree that it's a reasonable path forward even if it's not the one they would have chosen. I also like the policy of documenting objections/lingering concerns which ties in nicely with your suggestion to add an "unresolved questions" section.

[yutannihilation]

I agree with you. I just wanted to give a bit of the background information here :)

[hadley]

Incorporated feedback so far in #12. Will close review period on August 13, so there's still plenty of time for more discussion 😄

[hadley]

Thanks for the feedback everyone! We now have an official process for proposing bigger changes to the tidyverse: https://github.com/tidyverse/tidyups/blob/main/001-tidyup-process.md 🥳