Skip to content

Commit

Permalink
Merge pull request #115 from ynput/documentation/update_to_sitesync_f…
Browse files Browse the repository at this point in the history
…or_ayon

Updated documentation for Site Sync for Ayon
  • Loading branch information
kalisp authored Dec 15, 2023
2 parents fea4947 + c619e20 commit e9438d5
Show file tree
Hide file tree
Showing 13 changed files with 27 additions and 78 deletions.
57 changes: 27 additions & 30 deletions website/docs/addon_site_sync_admin.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,18 @@ import versions from '@site/docs/assets/json/Ayon_addons_version.json'
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

:::warning
**This feature is** currently **in a beta stage** and it is not recommended to rely on it fully for production.
:::

Site Sync allows users and studios to synchronize published assets between multiple 'sites'. Site denotes a storage location,
which could be a physical disk, server, cloud storage. To be able to use site sync, it first needs to be configured.

The general idea is that each user acts as an individual site and can download and upload any published project files when they are needed. that way, artist can have access to the whole project, but only every store files that are relevant to them on their home workstation.

:::note
At the moment site sync is only able to deal with publishes files. No workfiles will be synchronized unless they are published. We are working on making workfile synchronization possible as well.
At the moment site sync is only able to deal with published files (eg. in `publish` folder). **No workfiles (in `work` folder) will be synchronized.** We are working on making workfile synchronization possible as well.
:::

## System Settings
## Studio Settings

To use synchronization, *Site Sync* needs to be enabled globally in **AYON Settings/System/Modules/Site Sync**.
First of all, you need to include Site Sync Addon to your bundle, and to use synchronization, *Site Sync* needs to be enabled globally in **ayon+settings://sitesync**.

![Configure module](assets/site_sync_system.png)

Expand All @@ -47,7 +43,7 @@ Each AYON Tray app works with two sites at one time. (Sites can be the same, and

Sites could be configured differently per project basis.

Each new site needs to be created first in `System Settings`. Most important feature of site is its Provider, select one from already prepared Providers.
Each new site needs to be created first in `Studio Settings`. Most important feature of site is its Provider, select one from already prepared Providers.

#### Alternative sites

Expand All @@ -60,25 +56,29 @@ same location >> file is accessible on 'sftp' site right away, no need to sync i

##### Example
![Configure module](assets/site_sync_system_sites.png)
Admin created new `sftp` site which is handled by `SFTP` provider. Somewhere in the studio SFTP server is deployed on a machine that has access to `studio` drive.

Admin created new `SFTP` site which is handled by `SFTP` provider. Somewhere in the studio SFTP server is deployed on a machine that has access to `studio` drive.

Alternative sites work both way:
- everything published to `studio` is accessible on a `sftp` site too
- everything published to `sftp` (most probably via artist's local disk - artists publishes locally, representation is marked to be synced to `sftp`. Immediately after it is synced, it is marked to be available on `studio` too for artists in the studio to use.)
- everything published to `studio` is accessible on a `SFTP` site too
- everything published to `SFTP` (most probably via artist's local disk - artists publishes locally, representation is marked to be synced to `SFTP`. Immediately after it is synced, it is marked to be available on `studio` too for artists in the studio to use.)

## Project Settings

Sites need to be made available for each project. Of course this is possible to do on the default project as well, in which case all other projects will inherit these settings until overridden explicitly.

You'll find the setting in **Settings/Project/Global/Site Sync**
You'll find the setting in **ayon+settings://sitesync**

The attributes that can be configured will vary between sites and their providers.

## Local settings
## Site settings

Each user should configure root folder for their 'local' site via **Local Settings** in AYON Tray. This folder will be used for all files that the user publishes or downloads while working on a project. Artist has the option to set the folder as "default"in which case it is used for all the projects, or it can be set on a project level individually.
Each user should configure root folder for their 'local' site via **Root** tab in AYON Server. This folder will be used for all files that the user publishes or downloads while working on a project.

Artists can also override which site they use as active and remote if need be.
![Local overrides](assets/site_sync_local_root.png)

Artists can also override which site they use as active and remote to activet Site Sync functionality if sites are not set in Project Settings.
Here artist could also change all user based configuration based on provider (for example password for sftp user etc.)

![Local overrides](assets/site_sync_local_setting.png)

Expand Down Expand Up @@ -144,28 +144,25 @@ Beware that ssh key expects OpenSSH format (`.pem`) not a Putty format (`.ppk`)!

#### How to set SFTP site

- Enable Site Sync module in Settings
- Add side with SFTP provider
- Add side with SFTP provider in Studio Settings

![Enable syncing and create site](assets/site_sync_sftp_system.png)

- In Projects setting enable Site Sync (on default project - all project will be synched, or on specific project)
- In Projects setting enable Site Sync
- Configure SFTP connection and destination folder on a SFTP server (in screenshot `/upload`)

![SFTP connection](assets/site_sync_project_sftp_settings.png)

- if you want to force syncing between local and sftp site for all users, use combination `active site: local`, `remote site: NAME_OF_SFTP_SITE`
- if you want to allow only specific users to use SFTP syncing (external users, not located in the office), use `active site: studio`, `remote site: studio`.

![Select active and remote site on a project](assets/site_sync_sftp_project_setting_not_forced.png)

- Each artist can decide and configure syncing from his/her local to SFTP via `Local Settings`
- Each artist can decide and configure syncing from his/her local to SFTP via `Site Settings`

![Select active and remote site on a project](assets/site_sync_sftp_settings_local.png)

### Custom providers

If a studio needs to use other services for cloud storage, or want to implement totally different storage providers, they can do so by writing their own provider plugin. We're working on a developer documentation, however, for now we recommend looking at `abstract_provider.py`and `gdrive.py` inside `ayon/modules/sync_server/providers` and using it as a template.
If a studio needs to use other services for cloud storage, or want to implement totally different storage providers, they can do so by writing their own provider plugin. We're working on a developer documentation, however, for now we recommend looking at [`abstract_provider.py`](https://github.com/ynput/ayon-sitesync/blob/develop/client/ayon_sitesync/providers/abstract_provider.py) and [`gdrive.py`](https://github.com/ynput/ayon-sitesync/blob/develop/client/ayon_sitesync/providers/gdrive.py) inside `ayon-sitesync/client/ayon-sitesync/providers` and using it as a template.

### Running Site Sync in background

Expand All @@ -181,30 +178,30 @@ To configure all sites where all published files should be synced eventually you
This is an example of:
- Site Sync is enabled for a project
- default active and remote sites are set to `studio` - eg. standard process: everyone is working in a studio, publishing to shared location etc.
- (but this also allows any of the artists to work remotely, they would change their active site in their own Local Settings to `local` and configure local root.
- (but this also allows any of the artists to work remotely, they would change their active site in their own Site Settings to `local` and configure local root.
This would result in everything artist publishes is saved first onto his local folder AND synchronized to `studio` site eventually.)
- everything exported must also be eventually uploaded to `sftp` site
- everything exported must also be eventually uploaded to `SFTP` site

This eventual synchronization between `studio` and `sftp` sites must be physically handled by background process.
This eventual synchronization between `studio` and `SFTP` sites must be physically handled by background process.

As current implementation relies heavily on Settings and Local Settings, background process for a specific site ('studio' for example) must be configured via Tray first to `syncserver` command to work.
As current implementation relies heavily on Settings and Site Settings, background process for a specific site ('studio' for example) must be configured via Tray first to `syncserver` command to work.

To do this:

- run OP `Tray` with environment variable AYON_LOCAL_ID set to name of active (source) site. In most use cases it would be studio (for cases of backups of everything published to studio site to different cloud site etc.)
- start `Tray`
- check `Local ID` in information dialog after clicking on version number in the Tray
- open `Local Settings` in the `Tray`
- open `Site Settings` in the `Ayon Server`
- configure for each project necessary active site and remote site
- close `Tray`
- run OP from a command line with `syncserver` and `--active_site` arguments
- run Ayon launcher from a command line with `syncserver` and `--active_site` arguments


This is an example how to trigger background syncing process where active (source) site is `studio`.
(It is expected that OP is installed on a machine, `openpype_console` is on PATH. If not, add full path to executable.
(It is expected that OP is installed on a machine, `ayon_console` is on PATH. If not, add full path to executable.
)
```shell
openpype_console syncserver --active_site studio
ayon_console syncserver --active_site studio
```


Expand Down
1 change: 0 additions & 1 deletion website/docs/artist_tools_loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,5 @@ You can use multiselection for representations to mark them for downloading in o

If the AYON Tray is running, these files will be transferred in the background for you and show up on the local active site.

For more details of progress, state, or possible error details, open the [Sync Queue](#Sync-Queue) item in the AYON Tray app.

![tools_loader_sync](assets/tools/tools_loader_sync.png)
46 changes: 0 additions & 46 deletions website/docs/artist_tools_sync_queu.md

This file was deleted.

Binary file modified website/docs/assets/site_sync_always_on.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/site_sync_local_root.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/assets/site_sync_local_setting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/assets/site_sync_project_settings.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/assets/site_sync_sftp_settings_local.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/assets/site_sync_sftp_system.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/assets/site_sync_system.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/docs/assets/site_sync_system_sites.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ module.exports = {
"artist_tools_inventory",
"artist_tools_workfiles",
"artist_tools_look_assigner",
"artist_tools_sync_queue",
],
},
],
Expand Down

0 comments on commit e9438d5

Please sign in to comment.