Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move GTO docs here from mlem.ai #4822

Merged
merged 4 commits into from
Sep 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions content/docs/gto/command-reference/assign.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# assign

Assign stage to specific artifact version.

## Synopsis

```usage
usage: gto assign [-r <text>] [--version <text>]
[--stage <text>] [-m <text>]
[--simple <text>] [--force] [--push] [--sr]
[-h]
name [ref]

arguments:
name Artifact name
[ref] Git reference to use
```

## Description

To assign an actionable stage for a specific artifact version use the same
`gto assign` command. Stages can mark the artifact readiness for a specific
consumer. You can plug in a real downsteam system via CI/CD or web hooks, e.g.
to redeploy an ML model.

```cli
$ gto assign awesome-model --version v0.0.1 --stage prod
Created git tag 'awesome-model#prod#1' that assigns stage to 'v0.0.1'
```

GTO creates a special Git tag in
[the standard format](/doc/gto/user-guide#git-tags-format).

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `--version <text>` - If you provide REF, this will be used to name new version
- `--stage <text>` - Stage to assign
- `-m <text>`, `--message <text>` - Message to annotate the Git tag with
- `--simple <text>` - Use simple notation, e.g. `rf#prod` instead of `rf#prod-5`
[supported values: auto, true, false] [default: auto]
- `--force` - Create the Git tag even if it already exists and is in effect
- `--push` - Push created tag automatically (experimental)
- `--sr`, `--skip-registration` - Don't register a version at specified commit
- `-h`, `--help` - Show this message and exit.

## Examples

Assign artifact "nn" to "prod" at specific Git ref instead of supplying artifact
version (note that this will also register a version if it doesn't exist):

```cli
$ gto assign nn abcd123 --stage prod
```

Assign stage at specific Git ref and name the version explicitly (this assumes
that version was not registered yet):

```cli
$ gto assign nn abcd123 --version v1.0.0 --stage prod
```
36 changes: 36 additions & 0 deletions content/docs/gto/command-reference/check-ref.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# check-ref

Find out the artifact version registered/assigned with ref.

## Synopsis

```usage
usage: gto check-ref [-r <text>] [--json] [--name] [--version]
[--event] [--stage] [-h]
ref

arguments:
ref Git reference to analyze
```

## Description

You can use `gto check-ref` to interpret a Git tag:

```cli
$ gto check-ref -r build/example-gto churn#prod#3
βœ… Stage "prod" was assigned to version "v3.0.0" of artifact "churn"
```

For machine-consumable format, use `--json` flag or output specific pieces of
information with `--name`, `--version`, `--stage` or `--event`.

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `--json` - Print output in json format
- `--name` - Show artifact name
- `--version` - Output artifact version
- `--event` - Show event
- `--stage` - Show artifact stage
- `-h`, `--help` - Show this message and exit.
141 changes: 141 additions & 0 deletions content/docs/gto/command-reference/deprecate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# deprecate

Deprecate artifact, deregister a version, or unassign a stage.

## Synopsis

```usage
usage: gto deprecate [-r <text>] [-m <text>] [--simple <text>]
[--force] [-d] [--push] [-h]
name [version] [stage]

arguments:
name Artifact name
[version] Artifact version
[stage] Stage to unassign
```

## Description

The command supports three use cases:

```cli
# deprecate an artifact:
$ gto deprecate nn

# deprecate a version:
$ gto deprecate nn v0.0.1

# unassign a stage:
$ gto deprecate nn v0.0.1 prod
```

### Unassigning a stage

Sometimes you need to mark an artifact version no longer ready for a specific
consumer, and maybe signal a downstream system about this. You can use
`gto deprecate` for that:

```cli
$ gto deprecate awesome-model v0.0.1 prod
Created git tag 'awesome-model#prod!#2' that unassigns a stage from 'v0.0.1'
```

<details>

### Unassigning a stage: some details and options

GTO creates a special Git tag in
[the standard format](/doc/gto/user-guide#git-tags-format).

Note, that later you can create this stage again, if you need to, by calling
`$ gto assign` again.

You also may want to delete the git tag instead of creating a new one. This is
useful if you don't want to keep extra tags in you Git repo, don't need history
and don't want to trigger a CI/CD or another downstream system. For that, you
can use:

```cli
$ gto deprecate awesome-model v0.0.1 prod --delete
Deleted git tag 'awesome-model#prod#1' that assigned a stage to 'v0.0.1'
To push the changes upstream, run:
git push origin awesome-model#prod#1 --delete
```

</details>

### Deregister a version

Sometimes you need mark a specific artifact version as a no longer ready for
usage. You could just delete a git tag, but if you want to preserve a history of
the actions, you may again use `gto deprecate`.

```cli
$ gto deprecate awesome-model v0.0.1
Created git tag '[email protected]!' that deregistered a version.
```

<details>

### Deregister a version: some details and options

If you want to deregister the version by deleting the Git tags itself, you could
use

```cli
$ gto deprecate awesome-model v0.0.1 --delete
Deleted git tag '[email protected]' that registered a version.
Deleted git tag 'awesome-model#prod#1' that assigned a stage to 'v0.0.1'.
Deleted git tag 'awesome-model#prod!#2' that unassigned a stage to 'v0.0.1'.
To push the changes upstream, run:
git push origin [email protected] awesome-model#prod#1 awesome-model#prod!#2 --delete
```

This includes all Git tags related to the version: a tag that registered it and
all tags that assigned stages to it.

</details>

### Deprecating an artifact

Sometimes you need to need to mark the artifact as "deprecated", usually meaning
it's outdated and will no longer be developed. To do this, you could run:

```cli
$ gto deprecate awesome-model
Created Git tag 'awesome-model@deprecated' that deprecates an artifact.
```

<details>

### Deprecating an artifact: some details and options

With `awesome-model@deprecated` Git tag the artifact will be considered
deprecated until you register a new version or assign a new stage to it after
the deprecation.

If you want to deprecate an artifact by deleting git tags, you'll need to delete
all of them for the artifact. You could do that with

```cli
$ gto deprecate awesome-model --delete
Deleted git tag '[email protected]' that registered a version.
Deleted git tag 'awesome-model#prod#1' that assigned a stage to 'v0.0.1'.
Deleted git tag 'awesome-model#prod!#2' that unassigned a stage to 'v0.0.1'.
To push the changes upstream, run:
git push origin [email protected] awesome-model#prod#1 awesome-model#prod!#2 --delete
```

</details>

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `-m <text>`, `--message <text>` - Message to annotate the Git tag with
- `--simple <text>` - Use simple notation, e.g. `rf#prod` instead of `rf#prod-5`
[supported values: auto, true, false] [default: auto]
- `--force` - Create the Git tag even if it already exists and is in effect
- `-d`, `--delete` - Delete the git tag(s) instead of creating the new one
- `--push` - Push created tag automatically (experimental)
- `-h`, `--help` - Show this message and exit.
52 changes: 52 additions & 0 deletions content/docs/gto/command-reference/describe.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# describe

Display enrichments for an artifact.

## Synopsis

```usage
usage: gto describe [-r <text>] [--rev <text>] [--type] [--path]
[--description] [-h]
name

arguments:
name Artifact name
```

## Description

To get details about an artifact (from `artifacts.yaml`) use `gto describe`:

```cli
$ gto describe churn -r https://github.com/iterative/example-gto
{
"type": "model",
"path": "models/churn.pkl",
"virtual": false
}
```

The output is in JSON format for ease of parsing programmatically.

Note, that for local repos the `artifacts.yaml` is read from the workspace
without Git, so if you have uncommitted changes, they will be reflected in the
output. If you want to read from specific commit, you need to specify `--rev`
option.

You can also get annotation for specific versions (these are the same shortcuts
as in `gto show`):

```cli
$ gto describe churn@latest # highest version by SemVer
$ gto describe churn#dev # version in stage `dev`
$ gto describe [email protected] # version `v3.0.0`
```

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `--rev <text>` - Repo revision to use
- `--type` - Show type
- `--path` - Show path
- `--description` - Show description
- `-h`, `--help` - Show this message and exit.
36 changes: 36 additions & 0 deletions content/docs/gto/command-reference/doctor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# describe

Display GTO version and check the registry for problems.

## Synopsis

```usage
usage: gto doctor [-r <text>] [-A] [-h]
```

## Description

This will check the registry and print all the issues if found:

```cli
$ gto doctor
πŸͺ΄ GTO Version: 0.2.5
---------------------------------
INDEX='artifacts.yaml'
TYPES=None
STAGES=None
LOG_LEVEL='INFO'
DEBUG=False
ENRICHMENTS=[]
AUTOLOAD_ENRICHMENTS=True
CONFIG_FILE_NAME='.gto'
EMOJIS=True
---------------------------------
βœ… No issues found
```

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `-A`, `--all-commits` - Read all commits
- `-h`, `--help` - Show this message and exit.
50 changes: 50 additions & 0 deletions content/docs/gto/command-reference/history.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# history

Show a journal of registry operations.

## Synopsis

```usage
usage: gto history [-r <text>] [-a] [-A] [--json] [--plain]
[--asc] [-h]
[name]

arguments:
[name] Artifact name to show. If empty, show all.
```

## Description

This command prints a journal of the events that happened to an artifact. This
allows you to audit the changes.

```cli
$ gto history churn -r https://github.com/iterative/example-gto
╒═════════════════════╀════════════╀══════════════╀═══════════╀═════════╀══════════╀═════════════════╕
β”‚ timestamp β”‚ artifact β”‚ event β”‚ version β”‚ stage β”‚ commit β”‚ ref β”‚
β•žβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•ͺ════════════β•ͺ══════════════β•ͺ═══════════β•ͺ═════════β•ͺ══════════β•ͺ═════════════════║
β”‚ 2022-11-09 13:40:33 β”‚ churn β”‚ assignment β”‚ v3.1.1 β”‚ dev β”‚ 2f2a8de β”‚ churn#dev#5 β”‚
β”‚ 2022-11-09 13:40:33 β”‚ churn β”‚ registration β”‚ v3.1.1 β”‚ - β”‚ 2f2a8de β”‚ [email protected] β”‚
β”‚ 2022-11-08 09:53:53 β”‚ churn β”‚ commit β”‚ v3.1.1 β”‚ - β”‚ 2f2a8de β”‚ 2f2a8de β”‚
β”‚ 2022-11-07 06:07:13 β”‚ churn β”‚ assignment β”‚ v3.1.0 β”‚ dev β”‚ 064f173 β”‚ churn#dev#4 β”‚
β”‚ 2022-11-06 02:20:33 β”‚ churn β”‚ assignment β”‚ v3.0.0 β”‚ prod β”‚ ddae695 β”‚ churn#prod#3 β”‚
β”‚ 2022-11-04 22:33:53 β”‚ churn β”‚ assignment β”‚ v3.1.0 β”‚ staging β”‚ 064f173 β”‚ churn#staging#2 β”‚
β”‚ 2022-11-03 18:47:13 β”‚ churn β”‚ assignment β”‚ v3.0.0 β”‚ dev β”‚ ddae695 β”‚ churn#dev#1 β”‚
β”‚ 2022-11-02 15:00:33 β”‚ churn β”‚ registration β”‚ v3.1.0 β”‚ - β”‚ 064f173 β”‚ [email protected] β”‚
β”‚ 2022-11-01 11:13:53 β”‚ churn β”‚ commit β”‚ v3.1.0 β”‚ - β”‚ 064f173 β”‚ 064f173 β”‚
β”‚ 2022-10-28 23:53:53 β”‚ churn β”‚ registration β”‚ v3.0.0 β”‚ - β”‚ ddae695 β”‚ [email protected] β”‚
β”‚ 2022-10-27 20:07:13 β”‚ churn β”‚ commit β”‚ v3.0.0 β”‚ - β”‚ ddae695 β”‚ ddae695 β”‚
β•˜β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•β•§β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•›
```

Use `--all-branches` and `--all-commits` to read more than just HEAD.

## Options

- `-r <text>`, `--repo <text>` - Local or remote repository [default: .]
- `-a`, `--all-branches` - Read heads from all branches
- `-A`, `--all-commits` - Read all commits
- `--json` - Print output in json format
- `--plain` - Print table in grep-able format
- `--ascending`, `--asc` - Show new first
- `-h`, `--help` - Show this message and exit.
11 changes: 11 additions & 0 deletions content/docs/gto/command-reference/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# GTO Command Reference

GTO is a command line tool. It works on top of Git. For a list of all commands,
type `gto -h`.

## Typical GTO workflow

- Register artifact versions with `gto register`
- Assign stages to them with `gto assign`
- Read the registry with `gto show` and `gto history`
- Read and interpret Git tag with `gto check-ref`
Loading
Loading