Skip to content

Commit

Permalink
Update flytefile.md (#5428)
Browse files Browse the repository at this point in the history
Signed-off-by: Yee Hing Tong <[email protected]>
  • Loading branch information
wild-endeavor authored May 29, 2024
1 parent 75b33f8 commit 22d81c6
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 9 deletions.
4 changes: 2 additions & 2 deletions docs/user_guide/data_types_and_io/flytefile.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,14 @@ When this task finishes, the Flytekit engine returns the `FlyteFile` instance, u

Lastly, define a workflow. The `normalize_csv_files` workflow has an `output_location` argument which is passed to the `location` input of the task. If it's not an empty string, the task attempts to upload its file to that location.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/data_types_and_io/data_types_and_io/file.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/data_types_and_io/data_types_and_io/file.py
:caption: data_types_and_io/file.py
:pyobject: normalize_csv_file
```

You can run the workflow locally as follows:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/data_types_and_io/data_types_and_io/file.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/data_types_and_io/data_types_and_io/file.py
:caption: data_types_and_io/file.py
:lines: 75-95
```
Expand Down
14 changes: 7 additions & 7 deletions docs/user_guide/extending/custom_types.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ To clone and run the example code on this page, see the [Flytesnacks repo][flyte

First, we import the dependencies:

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/extending/extending/custom_types.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/extending/custom_types.py
:caption: extending/custom_types.py
:lines: 1-7
```
Expand All @@ -38,7 +38,7 @@ First, we import the dependencies:

Defined type here represents a list of files on the disk. We will refer to it as `MyDataset`.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/extending/extending/custom_types.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/extending/custom_types.py
:caption: extending/custom_types.py
:pyobject: MyDataset
```
Expand All @@ -53,31 +53,31 @@ The `TypeTransformer` is a Generic abstract base class. The `Generic` type argum
that we want to work with. In this case, it is the `MyDataset` object.
:::

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/extending/extending/custom_types.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/extending/custom_types.py
:caption: extending/custom_types.py
:pyobject: MyDatasetTransformer
```

Before we can use MyDataset in our tasks, we need to let Flytekit know that `MyDataset` should be considered as a valid type.
This is done using {py:class}`~flytekit:flytekit.extend.TypeEngine`'s `register` method.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/extending/extending/custom_types.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/extending/custom_types.py
:caption: extending/custom_types.py
:lines: 87
```

The new type should be ready to use! Let us write an example generator and consumer for this new datatype.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/extending/extending/custom_types.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/extending/custom_types.py
:caption: extending/custom_types.py
:lines: 91-114
```

This workflow can be executed and tested locally. Flytekit will exercise the entire path even if you run it locally.

```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/master/examples/extending/extending/custom_types.py
```{rli} https://raw.githubusercontent.com/flyteorg/flytesnacks/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/extending/custom_types.py
:caption: extending/custom_types.py
:lines: 119-120
```

[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/master/examples/extending/
[flytesnacks]: https://github.com/flyteorg/flytesnacks/tree/0ec8388759d34566a0ffc0c3c2d7443fd4a3a46f/examples/extending/

0 comments on commit 22d81c6

Please sign in to comment.