Merge branch 'yorkie-team:main' into main

.github/workflows/create-yorkie-app-publish.yml

@@ -0,0 +1,31 @@
+name: create-yorkie-app-publish
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+    paths:
+      - tools/create-yorkie-app/**
+      - examples/**
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
+    steps:
+      - name: Checkout 🛎️
+        uses: actions/checkout@v2
+      - name: Setup Node 🔧
+        uses: actions/setup-node@v2
+        with:
+          node-version: '18'
+          cache: 'npm'
+          cache-dependency-path: package-lock.json
+          registry-url: 'https://registry.npmjs.org'
+      - run: npm install
+      - run: cd tools/create-yorkie-app && npm run build
+      - run: cd tools/create-yorkie-app && npm publish --provenance
+        continue-on-error: true
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/npm-publish.yml

@@ -5,6 +5,9 @@ on:
 jobs:
   build:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      id-token: write
     steps:
       - name: Checkout 🛎️
         uses: actions/checkout@v2

CONTRIBUTING.md

@@ -49,9 +49,9 @@ $ npm install
 $ npm run build
 ```
 
-For generating proto messages and the service client stub classes with protoc and the protoc-gen-grpc-web.
+To generate proto messages, we use protoc-gen-connect-es, which is a code generator plugin for Protocol Buffer compilers, like buf and protoc. It generates both clients and server definitions from Protocol Buffer schema.
 
-How to install protoc-gen-grpc-web: https://github.com/grpc/grpc-web#code-generator-plugin
+For more details, see [@connectrpc/protoc-gen-connect-es](https://github.com/connectrpc/connect-es/tree/main/packages/protoc-gen-connect-es).
 
 ```bash
 # generate proto messages and the service client stub classes
@@ -61,9 +61,9 @@ $ npm run build:proto
 > Primary "source of truth" location of protobuf message is
 > in [yorkie](https://github.com/yorkie-team/yorkie/tree/main/api). We manage the messages in the repository.
 
-### Testing yorkie-js-sdk with Envoy, Yorkie and MongoDB.
+### Testing yorkie-js-sdk with Yorkie and MongoDB.
 
-Start MongoDB, Yorkie and Envoy proxy in a terminal session.
+Start MongoDB, Yorkie in a terminal session.
 
 ```bash
 $ docker-compose -f docker/docker-compose.yml up --build -d
@@ -82,9 +82,38 @@ $ docker pull yorkieteam/yorkie:latest
 $ docker-compose -f docker/docker-compose.yml up --build -d
 ```
 
+To print specific console logs, delete the line `return false` in the `onConsoleLog()` function within [`vitest.config.ts`](https://github.com/yorkie-team/yorkie-js-sdk/blob/main/vitest.config.ts#L16).
+
+```ts
+export default defineConfig({
+  test: {
+    include: ['**/*_{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}'],
+    exclude: ['**/bench/*'],
+    coverage: {
+      provider: 'istanbul',
+      reporter: ['lcov', 'text-summary'],
+    },
+    onConsoleLog() {
+      return false; // <<------ delete here.
+    },
+    environment: 'custom-jsdom',
+    globals: true,
+    testTimeout: isCI ? 5000 : Infinity,
+  },
+  plugins: [tsconfigPaths()],
+});
+```
+
+To run only specific suites or tests, use `.only` and execute the following command with the path to the desired test file.
+Refer to [Test Filtering](https://vitest.dev/guide/filtering#selecting-suites-and-tests-to-run) in `vitest` for more details:
+
+```bash
+$ npm run test {test file path} # e.g. npm run test integration/tree_test.ts
+```
+
 ### Starting co-editing example with CodeMirror
 
-Start MongoDB, Yorkie and Envoy proxy in a terminal session.
+Start MongoDB and Yorkie in a terminal session.
 
 ```bash
 $ docker-compose -f docker/docker-compose.yml up --build -d

README.md

@@ -8,20 +8,6 @@ The Yorkie JavaScript SDK implements the client-side libraries.
 
 To get started using Yorkie JavaScript SDK, see: https://yorkie.dev/docs/js-sdk
 
-## How yorkie-js-sdk works
-
-yorkie-js-sdk uses gRPC-web for communicating with Yorkie Server built on gRPC.
-
-```
- +--Browser--+           +--Envoy---------+         +--Yorkie-----+
- |           |           |                |         |             |
- | gRPC-web  <- HTTP1.1 -> gRPC-web proxy <- HTTP2 -> gRPC server |
- |           |           |                |         |             |
- +-----------+           +----------------+         +-------------+
-```
-
-For more details: https://grpc.io/blog/state-of-grpc-web/
-
 ## Contributing
 
 See [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches and the contribution workflow.

buf.gen.yaml

@@ -0,0 +1,12 @@
+version: v1
+plugins:
+  - plugin: es
+    out: .
+    opt: 
+    - target=js+dts
+    - js_import_style=legacy_commonjs
+  - plugin: connect-es
+    out: .
+    opt: 
+    - target=js+dts
+    - js_import_style=legacy_commonjs

config/webpack.config.js

@@ -47,6 +47,7 @@ module.exports = {
   mode: 'production',
   optimization: {
     minimize: false,
+    nodeEnv: false,
   },
   module: {
     rules: [

design/README.md

@@ -0,0 +1,25 @@
+# Design Document
+
+## Contents
+
+- [Devtools Extension](devtools.md): Message flow between devtools extension and yorkie-js-sdk
+
+## Maintaining the Document
+
+For significant scope and complex new features, it is recommended to write a
+Design Document before starting any implementation work. On the other hand, we
+don't need to design documentation for small, simple features and bug fixes.
+
+Writing a design document for big features has many advantages:
+
+- It helps new visitors or contributors understand the inner workings or the
+  architecture of the project.
+- We can agree with the community before code is written that could waste effort
+  in the wrong direction.
+
+While working on your design, writing code to prototype your functionality may
+be useful to refine your approach.
+
+Authoring Design document is also proceeded in the same
+[contribution flow](../CONTRIBUTING.md) as normal Pull Request such as function
+implementation or bug fixing.

design/devtools.md

@@ -0,0 +1,72 @@
+---
+title: devtools
+target-version: 0.4.13
+---
+
+# Devtools
+
+## Summary
+
+[Yorkie Devtools](https://github.com/yorkie-team/yorkie-js-sdk/tree/main/tools/devtools) is a Chrome extension designed to assist in debugging Yorkie. The devtools extension consists of a `panel` that displays Yorkie data and a `content script` for communication. This document examines the configuration of the extension and explains how it communicates with the yorkie-js-sdk.
+
+### Goals
+
+This document aims to help new SDK contributors understand overall message flow of the extension.
+
+### Non-Goals
+
+While this document explains the communication process and data flow within devtools, it does not cover the specifics of the data exchanged or how received data is presented.
+
+## Proposal Details
+
+### Devtools Extension Configuration and Message Flow
+
+A Chrome extension consists of various files, including popups, content scripts, background scripts, etc. Among them, Yorkie Devtools utilizes a `devtools panel` to display Yorkie data and a `content script` to communicate with the SDK.
+
+- [Devtools Panels](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/user_interface/devtools_panels)
+  - When an extension provides tools useful for developers, it can add a UI for them within the browser's developer tools as a new panel.
+  - `Yorkie 🐾` panel displays Yorkie data and is built using React. ([Code](https://github.com/yorkie-team/yorkie-js-sdk/blob/c0da57b3134d37cd8b113fe2c3aba612e3c89ecf/tools/devtools/src/devtools/panel/index.tsx))
+- [Content Scripts](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Content_scripts)
+  - Content scripts are injected into web pages within the browser and share the context with the page, allowing access to the page's content using DOM APIs. ([Code](https://github.com/yorkie-team/yorkie-js-sdk/blob/c0da57b3134d37cd8b113fe2c3aba612e3c89ecf/tools/devtools/src/content.ts))
+
+<img width="500" alt="Devtools Extension" src="media/devtools-extension.png">
+
+Content scripts play a role in passing messages between the panel and the SDK. To enable this communication between the panel and the SDK within webpages, appropriate APIs are used for each context. Communication between content script and the panel relies on Chrome's `port.postMessage` and `port.onMessage.addListener`, while communication between content script and the SDK relies on DOM's `window.postMessage` and `window.addEventListener` APIs.
+
+The diagram below illustrates the flow when the devtools panel "requests data" from the SDK. The panel initiates a connection to the inspected window and requests messages. The content script then relays the received message from the panel to the SDK, and the SDK subsequently forwards the message back to the panel.
+
+![Devtools Message Flow](media/devtools-message-flow.png)
+
+### Devtools Panel Lifecycle
+
+Let's examine the lifecycle of interaction between the devtools panel and the SDK in two scenarios:
+
+| ![Devtools life cycle 1](media/devtools-lifecycle1.png) |  ![Devtools life cycle 2](media/devtools-lifecycle2.png)  |
+| :-----------------------------------------------------: | :-------------------------------------------------------: |
+|    When the devtools panel is opened after page load    | When the devtools panel is open, and a new page is loaded |
+
+#### 1. When the Devtools Panel Is Opened After Page Load
+
+1. Upon opening the panel, it establishes a connection with the currently inspected window.
+   [tabs.connect()](https://developer.chrome.com/docs/extensions/develop/concepts/messaging#connect) creates a reusable channel (port) for long-term message passing between the panel and a content script.
+2. The content script confirms the port connection using `chrome.runtime.onConnect`.
+3. The panel, after establishing the port connection, sends a `devtools::connect` message to the SDK.
+   The content script plays a role in relaying messages between the panel and the SDK.
+4. The SDK, upon receiving the `devtools::connect` message, sends a `doc::available` message to the panel, including the currently connected document key.
+5. The panel, upon receiving the document key, sends a `devtools::subscribe` message to indicate its intention to subscribe to the corresponding document.
+6. The SDK, upon receiving the `devtools::subscribe` message, initiates synchronization. Initially, it sends a `doc::sync::full` message with all document information, and subsequently, it sends `doc::sync::partial` data whenever there are changes to the document.
+7. When the panel is closed, it is detected by the content script using `port.onDisconnect`, which then sends a `devtools::disconnect` message to the SDK.
+8. The SDK, upon detecting the panel disconnection, stops synchronization.
+
+#### 2. When the Devtools Panel Is Open, and a New Page Is Loaded
+
+1. Upon reloading the page, the existing port connection of the panel is closed. Upon completion of the new page load (`chrome.tabs.onUpdated`), a new port connection is established.
+2. The content script confirms the new port connection using `chrome.runtime.onConnect`.
+3. The panel, after establishing the port connection, sends a `devtools::connect` message to the SDK.
+   If yorkie-js-sdk is not ready at this point, no action is taken.
+4. Subsequently, when yorkie-js-sdk creates a new document and executes the [setup devtools](https://github.com/yorkie-team/yorkie-js-sdk/blob/c0da57b3134d37cd8b113fe2c3aba612e3c89ecf/src/devtools/index.ts#L98), it sends a `refresh-devtools` message.
+5. The panel, upon receiving the `refresh-devtools` message, sends `devtools::connect` message. The subsequent steps are identical to those in the first scenario (steps 1-4 to 1-8).
+
+### Risks and Mitigation
+
+Currently, Yorkie devtools can only inspect the entire data for changes to documents and presence. In the future, we plan to enhance devtools to show operations and changes, as well as introduce a time travel feature to observe how documents change when operations are applied.

design/template.md

@@ -0,0 +1,27 @@
+---
+title: feature-name
+---
+
+# Feature Name
+
+## Summary
+
+Write a brief description of the feature here.
+
+### Goals
+
+List the specific goals of the proposal. How will we know that this has succeeded?
+
+### Non-Goals
+
+What is out of scope for this proposal? Listing non-goals helps to focus discussion
+and make progress.
+
+## Proposal Details
+
+This is where we detail how to use the feature with snippet or API and describe
+the internal implementation.
+
+### Risks and Mitigation
+
+What are the risks of this proposal and how do we mitigate.

docker/docker-compose-ci.yml

@@ -1,31 +1,14 @@
 version: '3.3'
 
 services:
-  envoy:
-    build:
-      context: ./
-      dockerfile: ./envoy.Dockerfile
-    image: 'grpcweb:envoy'
-    container_name: 'envoy'
-    restart: always
-    ports:
-      - '8080:8080'
-      - '9901:9901'
-    command: ['/etc/envoy/envoy-ci.yaml']
-    depends_on:
-      - yorkie
   yorkie:
     image: 'yorkieteam/yorkie:latest'
     container_name: 'yorkie'
-    command: [
-      'server',
-      '--mongo-connection-uri',
-      'mongodb://mongo:27017',
-    ]
+    command: ['server', '--mongo-connection-uri', 'mongodb://mongo:27017']
     restart: always
     ports:
-      - '11101:11101'
-      - '11102:11102'
+      - '8080:8080'
+      - '8081:8081'
     depends_on:
       - mongo
   mongo:

docker/docker-compose.yml

@@ -1,31 +1,11 @@
 version: '3.3'
 
 services:
-  envoy:
-    build:
-      context: ./
-      dockerfile: ./envoy.Dockerfile
-    image: 'grpcweb:envoy'
-    container_name: 'envoy'
-    restart: always
-    ports:
-      - '8080:8080'
-      - '9901:9901'
-    command: ['/etc/envoy/envoy.yaml']
-    depends_on:
-      - yorkie
-    # If you're using Mac or Windows, this special domain name("host.docker.internal" which makes containers able to connect to the host)
-    # is supported by default.
-    # But if you're using Linux and want an envoy container to communicate with the host,
-    # it may help to define "host.docker.internal" in extra_hosts.
-    # (Actually, other hostnames are available, but in that case you should update clusters[].host configurations of envoy.yaml)
-    extra_hosts:
-      - 'host.docker.internal:host-gateway'
   yorkie:
     image: 'yorkieteam/yorkie:latest'
     container_name: 'yorkie'
     command: ['server', '--enable-pprof']
     restart: always
     ports:
-      - '11101:11101'
-      - '11102:11102'
+      - '8080:8080'
+      - '8081:8081'