Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement batch aggregation on cli #260

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ the API behind [the Zooniverse](https://www.zooniverse.org/).
The Panoptes CLI is written in Python, so in order to install it you will need
to install Python 3 along with `pip`. Please note: while still compatible with
Python 2.7, we have ended support for use of the CLI with this deprecated version.
macOS and Linux already come with Python installed, so run this to see if you
macOS and Linux already come with Python installed, so run this to see if you
already have everything you need:

```
Expand All @@ -18,7 +18,7 @@ $ python --version && pip --version
If you see an error like `python: command not found` or `pip: command not found`
then you will need to install this:

- [Python installation](https://wiki.python.org/moin/BeginnersGuide/Download)
- [Python installation](https://wiki.python.org/moin/BeginnersGuide/Download)
(or [Miniconda installation](https://docs.conda.io/en/latest/miniconda.html))
- [Pip installation](https://pip.pypa.io/en/stable/installing/)

Expand Down Expand Up @@ -280,6 +280,34 @@ $ panoptes workflow retire-subjects 101 2001 2002
$ panoptes workflow unretire-subjects 101 2001 2002
```

### Run aggregations

```
# for running batch aggregation on workflow with id 101, user id 2001 and conditional delete flag -d
$ panoptes workflow run-aggregation 101 2001 -d
```

### Get batch aggregations

```
# for fetching existing batch aggregation on workflow with id 101
$ panoptes workflow get-batch-aggregations 101
```

### Check batch aggregation run status

```
# for checking existing batch aggregation status on workflow with id 101
$ panoptes workflow check-batch-aggregation-run-status 101
```

### Get batch aggregation links

```
# for fetching links to the run aggregation on workflow with id 101
$ panoptes workflow get-batch-aggregation-links 101
```

#### By subject sets, i.e. for all the linked subjects in a subject set

```
Expand Down Expand Up @@ -325,7 +353,7 @@ Optional: include an updated_since timestamp (string) to include only observatio
$ panoptes inaturalist import-observations --taxon-id 46017 --subject-set-id 999999 --updated-since 2022-12-03
```

The `--updated-since` argument is a standard ISO timestamp, such as '2022-12-03' or `2022-12-03T18:56:06+00:00'. It is passed directly to the iNat Observations v2 API as the 'updated_since' query parameter.
The `--updated-since` argument is a standard ISO timestamp, such as '2022-12-03' or `2022-12-03T18:56:06+00:00'. It is passed directly to the iNat Observations v2 API as the 'updated_since' query parameter.



Expand Down
45 changes: 44 additions & 1 deletion panoptes_cli/commands/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

from panoptes_cli.scripts.panoptes import cli
from panoptes_client import Workflow

from panoptes_client.panoptes import PanoptesAPIException

@cli.group()
def workflow():
Expand Down Expand Up @@ -230,6 +230,49 @@ def delete(force, workflow_ids):
workflow.delete()


@workflow.command()
@click.argument('workflow-id', required=True, type=int)
@click.argument('user-id', required=True, type=int)
@click.option(
'--delete-if-exists',
'-d',
is_flag=True,
help='Delete if it exists.')
def run_aggregation(workflow_id, user_id, delete_if_exists):
"""Kicks off a new aggregation job."""

agg = Workflow(workflow_id).run_aggregation(user_id, delete_if_exists)
try:
click.echo(agg.raw)
except PanoptesAPIException as err:
click.echo(err)


@workflow.command()
@click.argument('workflow-id', required=True, type=int)
def get_batch_aggregations(workflow_id):
"""Gets existing batch aggregations."""

agg = Workflow(workflow_id).get_batch_aggregations()
click.echo(agg.object_list)


@workflow.command()
@click.argument('workflow-id', required=True, type=int)
def check_batch_aggregation_run_status(workflow_id):
"""Fetches the run status of existing aggregation."""

click.echo(Workflow(workflow_id).check_batch_aggregation_run_status())


@workflow.command()
@click.argument('workflow-id', required=True, type=int)
def get_batch_aggregation_links(workflow_id):
"""Fetches batch aggregation download links."""

click.echo(Workflow(workflow_id).get_batch_aggregation_links())


def echo_workflow(workflow):
click.echo(
u'{} {}'.format(
Expand Down
Loading