RFC: Workflow maintenance state

[srallen]

Gravity Spy has a rare, though, reoccurring issue with users sometimes getting demoted from the workflow they are assigned to in the project. I'm fairly confident I know the reason. In the process of selecting a workflow, the user project preferences' settings workflow id is preferred then the the their UPP preferences workflow id:

https://github.com/zooniverse/Panoptes-Front-End/blob/19c547544d57fe94034d1a6637175a7e05653440/app/pages/project/index.cjsx#L191-L196

The settings property is set by the promotion algorithm by the project. The preferences property is set by the user when selecting a workflow from the home page.

If the workflow is inactive, either one or both of those could be cleared out:

https://github.com/zooniverse/Panoptes-Front-End/blob/19c547544d57fe94034d1a6637175a7e05653440/app/pages/project/index.cjsx#L241-L253

This was to prevent an inactive workflow being loaded into the classifier. We have to assume that when a project makes a workflow inactive, they do not want volunteers classifying on it. For every other project, clearing out the UPP is inconsequential. The workflow selection will fallback to the default workflow for the project or random selection of the active workflows.

For Gravity Spy, this is causing the occasional demotion some users report. They regularly are updating the workflows to include new glitch classes or remove retired ones. This updating is happening more often than anticipated. During the few minutes they make the update, they have to make the workflow inactive because of the way the interface currently works. I think the API will reject PUT requests to workflow tasks on active workflows too?

To work around this I have two ideas to solve it:

Option 1: A new maintenance state for workflows. This would be an option the project builder could set that would change the lab UI to allow them to edit the workflow. On the volunteer side, the workflow will still be set as active, but we could also check a boolean on the maintenance state and instead of loading the workflow, load a message that the workflow is under temporary maintenance and to ask the volunteer to work on another workflow or project in the meantime. In this case, we would not clear out that workflow selection out of the UPP's. Potential issues are:

• Project owners may not understand what the purpose of this state is and may put workflows that should be made inactive into maintenance mode. My idea of the maintenance mode is that it should be temporary, no more than a day whereas workflows that really are not needed anymore should be made inactive. This could be hopefully prevented by clear communication in the project builder.
• It adds one more kind of state to workflows which may potentially cause complications I may not be thinking of.

Option 2: Leave the workflow state as is and continue to make workflows that need editing inactive as needed. Instead refactor the front end selection code to not clear out the UPP settings property and then automatically fallback to loading the project default (and if this isn't set, then to random active workflow). Potential issues with this:

• It still leaves the impression with the volunteer that they've been demoted by Gravity Spy at least initially because the project default workflow is level 1. They could go to the project home page and see they still have access to the other workflow levels, but I think it will cause confusion and complaints.

[camallen]

I think the API will reject PUT requests to workflow tasks on active workflows too?

It will if the payload has restricted fields on a "live project's active workflow" (phew), see:
https://github.com/zooniverse/Panoptes/blob/549483f8996f355c59283bf8ec36cbbc1a731e23/app/controllers/api/v1/workflows_controller.rb#L171

All other updates should be fine to do on a live active workflow so i'm not sure why they are modifying those fields instead of making new workflows... thoughts?

[camallen]

Ah - i see:

They regularly are updating the workflows to include new glitch classes or remove retired ones. This updating is happening more often than anticipated

This means they are modifying the survey task....i guess they don't want to make a new workflow in this case, right? that would cause other issues?

[srallen]

Yes, they're modifying the survey task. Making a new workflow would require updating the algorithm to process the new workflow as well as requiring a batch update of all of the users on that current workflow level to be set to the new workflow id at the very least. There may be other updates I'm not thinking of, but I see the potential for errors greater as more resources and the algorithm are being touched.

I could suggest to them to write a script to update the existing workflow as needed, but I'm imagining a maintenance state for workflows might be useful for other projects who often edit their workflows' tasks and would like to via the UI?

At least for Gravity Spy, the goal ultimately is to have a sustainable project after year three funding ends (next October) and presumably the project will continue to get new data as long as the LIGO detectors are functional and doing scientific runs. We're working on prioritizing this sustainability in case there won't be any additional funding acquired or no cost extensions. It may be possible the LIGO people that inherit the maintenance of the project will not have programming skills, so we want to try to do as much as we can in a user friendly way.

[camallen]

@srallen I agree we should make the UI as flexible as we can for non-technical people, though I'm not sure this particular case should be driving it. This RFC seems we have to engineer to support LIGO's offline processing model instead of our own use cases for 99% of projects.

I'm not a fan of the current state we have on projects / workflows, etc and adding more states complicates the whole thing. @marten had a neat suggestion a while back that is better than adding a maintenance state, that is just allowing changes to a live workflow to change but we serve a marked public version instead, #1523. This will solve the issue in question as you can happily change a workflow and then publish the new version as the active one.

However I'm not too sure about the implementation for that strategy as it requires some costly version history reconstruction (for workflows that move on) on a heavily used / critical end point. Any ideas welcome to speed this up taking into account some of the workflow objects are huge. Moving away from papertrail to just versioned workflows table? This performance cost here is the sticking point for me, we solve that and we'll solve the UI un-living issues.

[srallen]

@camallen I really like the suggestion in #1523 and it would be a more generalizable solution than the one I'm proposing. Like I mentioned in my last post, we roughly have about a year to solve this particular issue, however it's good to have it on our radar sooner than later. I'll chat more with @trouille about the project priorities and it sounds either way that solving this may need quite a bit of API work.

[camallen]

@srallen more solid timing constraints will help focus this. Also perhaps this is something to discuss with the team when you are in Oxford?

[srallen]

Yes, I'll chat more with @trouille about it since there are a few things that came up to do for year three of the grant. We'll get a sense of when we'll want this in the next year.

[srallen]

After speaking with @trouille, having a solution in place by June 2018 is the loose goal. There are higher priority projects coming up such as the other IMLS projects that may have back end needs.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[srallen]

This still is needed. I'll check in with @trouille about fitting it in.

[trouille]

We may end up following approach of allowing edits to live workflows, rather than creating workflow maintenance state/versioning. If we do this, need to include clear messaging that change makes public change and that version is saved in classification results, replace autosave with save button, etc.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.