Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server: New server page with docs specific to developers on the server #197

Merged
merged 4 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/server/enhance_gql.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/server/enhancing_endpoint.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/server/overwriting_types.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/server/prod_and_staging.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/server/query_hook_types.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/docs/assets/server/writing_gql_query.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
121 changes: 121 additions & 0 deletions website/docs/server_api_architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
---
id: server_api_architecture
title: Server API Architecture
sidebar_label: Server API Architecture
---

The backend exposes two APIs and redux is setup to handle both under a single api slice.

:::note
There are still a lot of legacy queries that inject instead of enhance. Over time we hope to convert these legacy queries to the new methodology.
:::

REST - https://ayon.dev/doc/api
GraphQL - https://ayon.dev/explorer

## Code Generation
We use code generation tools to automatically generate types and queries based off the restAPI and graphql schema.

Here is an overview of what is generated (green) and how it is consumed in the app.

![RTK Query oveview](./assets/server/api_querying_overview_diagram.png)


**RTK Query docs**

[RestAPI openAPI codegen](https://redux-toolkit.js.org/rtk-query/usage/code-generation#openapi)

[Graphql codegen](https://the-guild.dev/graphql/codegen/plugins/typescript/typescript-rtk-query)

## Codegen setup
The generated types are included in the repo but they need to be regenerated every time the API changes or if you write new/update graphql queries.

1. `yarn generate-rest` - Only needs to be done periodically if the RestAPI changes.
2. Generate graphql types and queries
a. Set `NAME` and `PASSWORD` values in your `.env.local` file. This can be for any user but I recommend using the admin credentials.
b. `yarn generate-token` - Fetches an auth token and writes to `.env` . Used by graphql-codegen to get the schema.
c. `yarn generate-graphql` - Generates types and rtk queries based on the schema and your `.graphql` files.
3. Both code gens create RTK queries. These queries can then be "enhanced" with more config and file separation.
4. Both files are created inside `/src/api/` `rest.ts` and `graphql.ts` . They are exported automatically.

:::note
These files will have already been generated and so you probably won't have to do it again.
:::

## Writing RTK Queries

Note: the images reference API.rest and API.graphql but you can now just use api.enhanceEndpoints for both.

### Rest

This is the easiest query to write because it should have already been generated for you by the codegen.

Look at http://localhost:3000/doc/api to see all the endpoints available. Every endpoint will have been created in /src/api/rest.ts as an RTK Query.

![Get Project Stats](./assets/server/rest_api_endpoint_example.png)

Once you have found the endpoint you want to use we can use it's title as the query name. In this case **Get Project Stats** ➝ `getProjectStats` .

You can always search for the endpoint in rest.ts to check if the query exists.

![Searching for Queries](./assets/server/searching_for_queries.png)

:::note
The query might already be in use, do a global search for getProjectStats to see if it's already setup and if you can reuse it.
:::

1. Now either create a new folder in `/src/services/` like projects with a new file `getProjects.ts`. Or you can add it to an already created file if it fits. In this case we will add it to `getProjects.ts`.

:::note
Although the queries have already been "injected" into the api inside rest.ts we always "enhance" these queries and then export the enhanced queries react hooks.
:::



2. Start typing your query name and it should autocomplete if it exists.
![Enhance an endpoint](./assets/server/enhancing_endpoint.png)

3. Do any enhancements required.
![Provide tags to endpoints](./assets/server/enhance_with_provide_tags.png)

4. Export the enhanced query as a hook to be consumed in a react component.

5. Use the react hook inside a component.
![Using RTK Query hook](./assets/server/using_RTK_query_hook.png)


## GraphQL

Writing graphql queries are similar but involve one extra step at the start to define the queries first.

1. Create a folder called inside /src/services/{queryFolderName}/gql . This is where you will write your graphql queries. Try to match the folder name to the query name and use PascalCase.
*We use PascalCase so that it's easier to distinguish between rest and graphql queries.*

![Writing gql query](./assets/server/writing_gql_query.png)

2. Create a new file called {GetQueryName}.graphql and write your query. Intellisense should provide suggestions and eslint should highlight when a field has an error.
![Writing gql query](./assets/server/writing_gql_query_2.png)


3. Once finished `run yarn generate-gql`. This will take your new .graphql file and generate a RTK Query from it whilst also generating the matching types.

*You can check this at the bottom of /src/api/graphql.ts *


4. Enhance the query similar to how the rest queries are enhanced; except in this case we use API.graphql.enhanceEndpoints .
![Enhancing gql query](./assets/server/enhance_gql.png)

5. Follow the same steps from step 3 of creating rest queries.

## Typescript overrides

Often you will want to transform the data inside the query to better structure it using transformResponse . Typescript only knows the full queries response type and so we need to tell it that the query will actually respond with our transformed type.

We do this by overriding the result type for the query to match the actual response type. As you can see the the query responds with a boolean and we need to tell typescript this.

![Overriding types](./assets/server/overwriting_types.png)


Now when we use the query hook we should see the new type we defined as the data. If we hadn't done this then the type would have been the generated query from step 3.

![Using types](./assets/server/query_hook_types.png)
58 changes: 58 additions & 0 deletions website/docs/server_introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
id: server_introduction
title: Server Introduction
sidebar_label: Server Introduction
---

## Getting Started

AYON Server is made up of two main repos, [ayon-backend](https://github.com/ynput/ayon-backend) and [ayon-frontend](https://github.com/ynput/ayon-frontend).

The fastest and easiest way to start testing both together is through docker using [ayon-docker](https://github.com/ynput/ayon-docker).

Once the server is running it can be accessed through the frontend UI using `http//localhost:5000`.

:::tip
Running the docker container is also a great way to start developing on the frontend. Set env variable `SERVER_URL=http//localhost:5000`.
:::

## AYON Overview

This is a general overview of the different parts of AYON. The server development is written in different parts and then deployed as one docker container that interacts with the AYON Launcher (client).

![Ayon server overview diagram](./assets/server/ayon_server_diagram.png)

## AYON Backend

- Python 3.12
- FastAPI, Strawberry (GraphQL)
- PostgreSQL (asyncpg), Redis (aioredis)
- gunicorn/uvicorn server (option to use granian)

## AYON Frontend

- Yarn, Nodejs 20+, Vite
- ReactJs
- Typescript (migration in progress)
- Redux toolkit with RTK Query
- AYON React Component Library (ARC)
- Styled Components

### **Env variables**

Create a `env.local` file in the root directory.

`SERVER_URL=http://localhost:5000` - URL of the backend server

### **Recommended VSCode Extensions**
- CSS Variable Autocomplete
- GitHub Pull Requests
- vscode-styled-components
- GraphQL: Language Feature Support
- GraphQL: Syntax Highlighting

## AYON React Component Library (ARC)

The frontend uses a custom component library for colors, fonts and reusable React components. The components are separated so that AYON addons can also utilize them for their UIs. Any component that could be useful for an addon and is not too specific should be created in ARC.

Read more about the style theme of AYON here: [Server Theme](server_theme.md)
114 changes: 114 additions & 0 deletions website/docs/server_theme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
id: server_theme
title: Server Theme
sidebar_label: Server Theme
---

The ayon theme is based on the material design philosophy: https://m2.material.io/.

Every color used has a variable from the theme and will always start with --md-sys-color-{name} .

## Colors

### Surface Containers

Surfaces go highest to lowest, elements with a higher z level should use higher surface levels.

- Highest (buttons) `--md-sys-color-surface-container-highest`
- High (cards) `--md-sys-color-surface-container-high`
- Neutral (cards) `--md-sys-color-surface-container`
- Low (BGs) `--md-sys-color-surface-container-low`
- Lowest (BGs) `--md-sys-color-surface-container-lowest`

### Primary Color

The primary color is a bright blue and is used in areas to draw attention, like save buttons. When using a color, ensure any content on top uses the inverse to ensure sufficient contrast ratios.

There are other colors like secondary and tertiary, but they are rarely used.

**Primary - important stuff **

```CSS
background-color: var(--md-sys-color-primary);
color: var(--md-sys-color-on-primary);
```

**Primary container - less important selections**

```CSS
background-color: var(--md-sys-color-primary-container);
color: var(--md-sys-color-on-primary-container);
```

### Text and border colors

Mostly uses `--md-sys-color-on-surface` but can use the inverse of a color like `--md-sys-color-on-primary` .

For a more fade text use `--md-sys-color-outline` or even more faded `--md-sys-color-outline-variant`.

These are also used for borders (try to avoid outlines).

### Developer Mode

Developer mode actions and UI use a specific purple color to make them standout and to signal they are for developers only. Rarely should you see this color when not in developer mode.

This color is a custom color outside of the theme color.

![Purple UI elements for developer mode actions](./assets/server/develoepr_mode_purple.png)

```CSS
var(--color-hl-developer);
var(--color-hl-developer-container);
var(--color-hl-developer-hover);
var(--color-hl-developer-surface);
var(--color-hl-developer-surface-hover)
```

### Production and Staging

Two specific colors are used to specify the difference between "production" and "staging". These color are outside of the theme color.

![Production and Staging UI elements](./assets/server/prod_and_staging.png)

```CSS
var(--color-hl-production)
var(--color-hl-production-hover)
var(--color-hl-production-active)

var(--color-hl-staging)
var(--color-hl-staging-hover)
var(--color-hl-staging-active)
```

:::note
Text on top of these colors should always be dark.
:::

## Useful Variables

- **Padding**: `--padding-s` (4px) `--padding-m` (8px) `--padding-l` (16px)
- **Border radius**: `--border-radius-m` (4px) `--border-radius-l` (8px) (there are more but they are wrong, this needs fixing)

## Typography

If you want to change the font size we have a theme of sizes.

Instead of using font-size: 14px use a className module from ARC typography.module.css.

(this could change)

```jsx
import Typography from "@theme/typography.module.css";

<span className={Typography.titleSmall}>Hello</span>;
```

## Loading state

- Avoid spinners
- Avoid layout shifts when the data loads in.
- Use skeleton shimmer placeholders.
- Use `${getShimmerStyles}` in styled components.

Example loading shimmer
![Loading shimmer](https://github.com/ynput/ayon-frontend/assets/49156310/f589ca02-37a3-41e4-a64a-3e2062083407)
5 changes: 5 additions & 0 deletions website/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,11 @@ const config = {
label: "Dev Docs",
position: "left",
},
{
to: "docs/server_introduction",
label: "Server Docs",
position: "left",
},
{
to: "/api",
label: "REST API Docs",
Expand Down
15 changes: 14 additions & 1 deletion website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,11 @@ module.exports = {
type: "category",
collapsed: false,
label: "Production Tracking",
items: ["artist_my_tasks_page", "artist_activity_feed", "artist_inbox"],
items: [
"artist_my_tasks_page",
"artist_activity_feed",
"artist_inbox",
],
},
],
Admin: [
Expand Down Expand Up @@ -185,4 +189,13 @@ module.exports = {
items: ["addon_aquarium_developer"].sort(sorted),
},
],
Server: [
"server_introduction",
{
type: "category",
label: "AYON Frontend",
collapsed: false,
items: ["server_api_architecture", "server_theme"],
},
],
};
Loading