Project app: subject picker

[eatyourgreens]

Package
app-project

Invision design

A popup that shows a table of subjects (with metadata) after picking a subject set to work on.

Question: how do we decide whether volunteers have the option to pick a subject? Is this an option that project owners would set when building a workflow?

[eatyourgreens]

We can page through set_member_subjects to list subjects for a subject set, but that won't include any information about subject retirement status for the selected workflow.

[camallen]

it may be slow for all subjects but the API supports the retirement status of a subject via this resource https://panoptes.docs.apiary.io/#reference/subjectworkflowstatus/subjectworkflowstatus/retrieve-a-single-subjectworkflowstatus

https://github.com/zooniverse/panoptes/blob/3fe89b90ae005460c3293456ce33b7b1a90c9878/app/serializers/subject_workflow_status_serializer.rb#L5-L6

[beckyrother]

This should be a checkbox in the Workflow editor, near the bottom and close to other Subject options:

(sorry it's not an InVision link, I edited in DevTools because I don't have a Sketch file for this page.)

@mrniaboc or @snblickhan might have thoughts on the copy.

[snblickhan]

1. What's the difference between implementing 'Subject Selection' and implementing an 'Indexing Tool'? Will a full use of the indexing tool require Admins to tick multiple boxes? Or is this an intermediary step towards that final goal? Maybe merits more discussion -- I'll move that elsewhere as not to distract from this topic, but wanted to flag.

2. In terms of copy, I'd suggestion "Enable Individual Subject Selection" as not to confuse with the section heading (see below).

3. I suggest we add this to the new "Subject Selection Settings" menu, along with grouped and sequential subject settings, shown here above the Classifier 2.0 (rewrite) settings.

Thanks, @beckyrother!

[eatyourgreens]

What's the difference between implementing 'Subject Selection' and implementing an 'Indexing Tool'? Will a full use of the indexing tool require Admins to tick multiple boxes? Or is this an intermediary step towards that final goal? Maybe merits more discussion -- I'll move that elsewhere as not to distract from this topic, but wanted to flag.

I was thinking that there are going to be projects, like Molly's galaxy project, which use grouped, prioritised selection but where the owner will want people to start at the first subject and classify them all in order. So, we can't just turn this on for all prioritised workflows. There might need to be a workflow config setting which activates the subject selection tools.

[srallen]

Regarding metadata sort, filter, and search, I'll propose an interim implementation to sort, filter, search in javascript against the current 20 subjects that is returned from the API. Grommet's DataTable has built in functionality for this, so should be straight forward and quick to implement. If it's possible to paginate against the response, we could also have a basic pagination UI to get the next page of subjects in queue, which then could be sort, filter, searched against for those 20 in UI.

[eatyourgreens]

Following the conversation in #1876, and a Slack conversation about how we might queue sequential subjects for anonymous users, I think we might request 50 subjects from the /queued endpoint, which gives us a bigger range to choose from and would allow us to only show each volunteer subjects that require their attention.

Alternatively, we could show every volunteer the same paged list of all subjects for this set (like the project builder does for the project team) but then we'd run into the question of how does a volunteer know which of those subjects needs work, and which have been finished for the current workflow (#1884 (comment).)

[snblickhan]

Ok, just checking how things are currently laid out in the InVision design draft. Looks like we're planning to show the completeness measure of the entire subject set, which would imply that we have information on the number of completed subjects in a given set. Perhaps that's a field we could add to the data table structure within a given set, as we've done for ALICE with the Seen/Unseen/Completed metric? Or at the very least a simplified version of that which indicates when things are Complete (we've discussed having those be greyed out so that images can be seen for context, but they're not able to be classified on).

This would be a great topic to discuss with @nciemniak and @zwolf to see what was implemented for ALICE and whether that's possible in Panoptes without a custom database.

In terms of use case, the ideal would be to show everything in a given subject set, but as Cam points out above it might be slow. But Zach & Nicole will be able to walk us through what was done for ALICE so we can make an informed decision.

[eatyourgreens]

I've got zooniverse/panoptes#3450 open to add completeness to subject sets.

[eatyourgreens]

Just realised that's harder than I thought, because a subject set can be used by more than one workflow, so has more than one measure of completeness.

[camallen]

Looks like we're planning to show the completeness measure of the entire subject set, which would imply that we have information on the number of completed subjects in a given set. Perhaps that's a field we could add to the data table structure within a given set, as we've done for ALICE with the Seen/Unseen/Completed metric? Or at the very least a simplified version of that which indicates when things are Complete (we've discussed having those be greyed out so that images can be seen for context, but they're not able to be classified on).

This has been requested before and it's something we can add

One naive approach could be to do this when doing the project completion counts on the hourly schedule? https://github.com/zooniverse/panoptes/blob/e4dcb068ad39bddd960423ef6e794f9917e940ca/app/workers/calculate_completeness_worker.rb

I'm not sure what kind of load the above would add to the system, it might be more prudent to modify the set counts on each retirement event? https://github.com/zooniverse/panoptes/blob/e4dcb068ad39bddd960423ef6e794f9917e940ca/app/workers/retirement_worker.rb#L12

Once those counts are on the SubjectSet resource we can serialize them via the API on the subject set resource.

[eatyourgreens]

it may be slow for all subjects but the API supports the retirement status of a subject via this resource https://panoptes.docs.apiary.io/#reference/subjectworkflowstatus/subjectworkflowstatus/retrieve-a-single-subjectworkflowstatus

Would there be any value in requesting subject-workflow-status for a subject set ID? Otherwise, it sounds like the status has to be queried individually for each subject that we want to show in the list.

In terms of use case, the ideal would be to show everything in a given subject set,

Once subjects have started to retire, won't this mean that we're asking volunteers to page/scroll through a list of retired pages before they reach one that needs work (particularly when a set is close to completion.) Is that a good user experience? I'd expect that we'd want to show subjects that need classification up front, at the top of the list. @beckyrother have you got any thoughts on this?

EDIT: I forgot about subjects that haven't retired, but which you have already classified. If we're listing all subjects in the set, and you are logged in, then maybe the list should start with the last page that you worked on in this set? Or start at the beginning but grey out all the pages you've done (could be annoying if you're near the end of the set.)

[camallen]

Good point about retirement status on each subject, requesting the SubjectWorkflowStatus resource for each subject isn't an optimal approach as data volumes grow. We don't have a decent way to expose / serialize this set of retired subject data via the API(s).

Would there be any value in requesting subject-workflow-status for a subject set ID? Otherwise, it sounds like the status has to be queried individually for each subject that we want to show in the list.

For small data sets i can see this working, once it moves beyond a few pages this will be slow loading times and higher load on the API. I'd rather we calculate and store these metrics on the subject set resource to optimize this end point.

[eatyourgreens]

For small data sets i can see this working, once it moves beyond a few pages this will be slow loading times and higher load on the API. I'd rather we calculate and store these metrics on the subject set resource to optimize this end point.

Sure. I was thinking about the table in the subject picker (above), where we'd want to flag retired subjects, or subjects that you have already classified, individually.

Shall we keep the subject set and subject conversations separate? Subject set conversations can go on zooniverse/panoptes#3450 and this issue can be used for discussing how we list status for individual subjects.