feat: Implement offline support through CozyPouchLink #1209

Ldoppea · 2024-05-28T18:23:33Z

We want to make the application work when offline.

To make this happens we want to configure a local PouchDB in order to replicated remote CouchDB data and then we want to serve them to the cozy-client instances in both the react-native side and the cozy-apps side (in WebViews)

This PR implement offline support first for cozy-home and mespapiers, other cozy-apps will come later when the new mechanism is proved to work

Related PR:

### ✨ Features

* Add support for Offline mode (for now only on cozy-home and mespapiers)

### 🐛 Bug Fixes

*

### 🔧 Tech

*

TODO:

Upgrade cozy-client after Meta PR: Implement offline support through CozyPouchLink cozy-client#1507 being merged
Upgrade cozy-intent after feat: Add offline related methods to cozy-intent and to cozy-device-helper cozy-libs#2562 being merged

Since cozy/cozy-flagship-app#1209 the Flagship app can inject the `offline_available` attribute into WebViews metadata in order to tell the cozy-app when offline mode is supported The new `` method can now be used to check this attribute

zatteo · 2024-09-02T08:32:33Z

src/pouchdb/platformReactNative.appState.ts

+  Boolean(appState.match(/active/) && nextAppState === 'background')
+
+const isGoingToWakeUp = (nextAppState: AppStateStatus): boolean =>
+  Boolean(appState.match(/background/) && nextAppState === 'active')


Is there a reason to use a regex here ?

No special reason (as far as i know), this is just a copy from other appState listener in the app (ex)

zatteo · 2024-09-02T08:41:01Z

src/libs/client.js

+    }
+
+    throw err
+  }


So this cache has an unlimited lifetime ?

For now, yes it is. I imagine that opening the app after a very long time AND while being offline is a rare event so this should be acceptable for now. But for sure we may want to add an invalidation mechanism soon.

src/pouchdb/sendDbByEmail.ts

zatteo · 2024-09-02T09:03:51Z

src/pouchdb/sendDbByEmail.ts

+  const deviceInfo = `Device info: ${deviceBrand} ${deviceModel} ${os} ${version}`
+
+  return `${appInfo}\n${bundleInfo}\n${deviceInfo}`
+}


Same method as src/app/domain/logger/fileLogger.ts ? Need to duplicate ?

We can yes, for now it is not really important, those are debug methods so it won't be problematic if they differ someday

src/components/webviews/jsInteractions/jsCozyInjection.ts

zatteo · 2024-09-02T09:07:18Z

Maybe to do in another PR : replace the atob and btoa polyfill we have here

cozy-flagship-app/src/App.js

Line 70 in be4e458

// Polyfill needed for cozy-client connection

by react-native-quick-base64

zatteo · 2024-09-02T09:08:43Z

src/pouchdb/platformReactNative.storage.ts

+  },
+  removeItem: async (key: string): Promise<void> => {
+    return AsyncStorage.removeItem(key)
+  }


A prefix is added to the key by PouchDB ?

No, the keys will be:

cozy-client-pouch-link-lastreplicateddocid

cozy-client-pouch-link-lastreplicationsequence

cozy-client-pouch-link-synced

zatteo · 2024-09-02T09:10:04Z

No real changes requested, just a lot of question.

As always, very easy to read 👍

Since cozy/cozy-flagship-app#1209 the Flagship app can inject the `offline_available` attribute into WebViews metadata in order to tell the cozy-app when offline mode is supported The new `` method can now be used to check this attribute

`react-native-quick-sqlite` is enforced to version `8.0.6` in order to be compatible with RN0.72 Related article: https://dev.to/craftzdog/a-performant-way-to-use-pouchdb7-on-react-native-in-2022-24ej

Related article: https://dev.to/craftzdog/a-performant-way-to-use-pouchdb7-on-react-native-in-2022-24ej

We want the Flagship app to work when offline To make this possible we configure cozy-client with CozyPouchLink which role will be to synchronize necessary doctypes into a local PouchDB and serve them from it instead of from the cozy-stack when the device is offline For now the list of synchronized doctypes is hardcoded but in the future we expect to implement a dynamic list based on cozy-apps' manifests Related PR: cozy/cozy-client#1507

In previous commit we configured cozy-client to use CozyPouchLink for its queries This commit also adds StackLink as the first Link so by default it will do its queries through the remote cozy-stack CozyClient has been modified to handle offline mode and redirect to the next link when it is detected Related PR: cozy/cozy-client#1507

This plugin is required by cozy-client to process `find` queries Related PR: cozy/cozy-client#1507

PouchDB can handle only `client.query()` requests For all other requests, like `client.fetchJSON()` we want to be able to store the request result in the device's AsyncStorage so we can re-use this result when the device is offline Data is stored using the client's url and the request content as key (it can be a string or a full request object)

`/apps/:slug/open` request is based on `client.fetchJSON()` and is on the critical execution path for the application to boot So we want to use the new caching mechanism from previous commit in order to retrieve this request's cached result if the application is booted while the device is offline

`cozy-intent` has been upgraded to `2.23.0` to retrieve new interfaces for FlagshipLink and files downloading Related PR: cozy/cozy-libs#2562

In previous commits we configured the application's cozy-client to use CozyPouchLink This link is configured for all react-native side queries, but not for cozy-app queries as they are served inside of WebViews and use a different cozy-client instance We want cozy-apps to benefits from the CozyPouchLink by redirecting their queries to the react-native side using the new FlagshipLink interface With this interface, all cozy-apps queries can be redirected to the react-native side using cozy-intent/post-me In order to intercept them, then we should declare `FlagshipLinkRequest()` method `localMethods` Related PR: cozy/cozy-client#1505

When a cozy-app's WebView is served offline, then the cozy-app bundle is served from local folder using the CozyProxyWebview, and since previous commits, the cozy-client's queries are served through `FlagshipLink` With those mechanisms, nearly all data are available offline except two things: - cozy-stack static assets (i.e. some css files, avatar, partners logos etc) - `fetchJSON()` requests This commit tries to handle the static assets part Those assets are often served with HTTP cache activated, and so the WebView should be able to load them correctly when the device is offline. However on Android, the WebView's cache seems not to persist after the application is closed Activating the `cacheMode=LOAD_CACHE_ELSE_NETWORK` seems to fix this behavior. But it prioritizes too much the cache over fresh data, so we want to activate it only when strictly necessary, which is when the application is offline This also allows to load partners logos that are not cached by the cozy-stack (as of today). But we expect to modify the cozy-stack to enable cache on them

With previous changes, we now expect the CozyPouchLink to be the last link of the chain, so forwarding the query would result to an exception thrown This mean we cannot forward the query when warmup queries are not finished yet So we want to allow CozyPouchLink to ignore the verification step and process the query independently of the warmup queries status To allow this we introduce the `ignoreWarmup` parameter. When set to `true` the CozyPouchLink will process the query even if warmup is not finished yet Related PR: cozy/cozy-client#1506

Some cozy-apps declare a data `schema` into cozy-client instance This schema is then used for processing queries As we now process cozy-app queries on the react-native side through FlagshipLinkRequest, we want to declared this schema in the Flagship app's cozy-client For now we only need to implement the one from cozy-home The ideal implementation would be to extract it from the cozy-app's bundle, but we did not implement anything to support this (we should first investigate how to do this). So for now it is hardcoded

By adding offline support through PouchDB, we expect database related bugs to happens in the future In order to ease debugging them, we want to allow exploring the local PouchDB files The easier way is to add the ability to extract them from the device and send them through email to cozy's support team

For now we don't want to replicate local changes into remote PouchDB as we first want to ensure everything works as expected locally before activating two-way replication in the future

In #1209 we added CozyPouchLink to CozyClient's instance By default CozyPouchLink starts replicating all its database directly after being initialized This is problematic as this happens during the App's startup and so it will unnecessary slow the startup process This is unnecessary because on startup, either the app is offline and so replication cannot be done, either the app is online and so it will use the cozy-stack. So replication can be delayed We want to defer the replication to a short delay after the startup (for now 30s)

Ldoppea requested review from acezard and zatteo as code owners May 28, 2024 18:23

Ldoppea marked this pull request as draft May 28, 2024 18:23

Ldoppea mentioned this pull request May 28, 2024

feat: Make CozyPouchLink compatible with ReactNative cozy/cozy-client#1483

Merged

Ldoppea force-pushed the feat/open_offline branch from 3e2ed92 to a3828ba Compare May 31, 2024 14:47

This was referenced May 31, 2024

feat: Handle Virtual Documents persistance in PouchLink cozy/cozy-client#1486

Merged

feat: Forward operation in StackLink.request() when offline cozy/cozy-client#1487

Merged

Ldoppea mentioned this pull request Jul 26, 2024

Meta PR: Implement offline support through CozyPouchLink cozy/cozy-client#1507

Merged

Ldoppea force-pushed the feat/open_offline branch 2 times, most recently from 21dbcff to 00fcd7d Compare July 30, 2024 18:16

Ldoppea changed the title ~~feat: Add offline support~~ feat: Implement offline support through CozyPouchLink Jul 30, 2024

Ldoppea force-pushed the feat/open_offline branch 2 times, most recently from 2cbaf5d to bcf0320 Compare August 26, 2024 15:42

Ldoppea changed the base branch from master to feat/meta_offline August 26, 2024 16:01

Ldoppea marked this pull request as ready for review August 26, 2024 16:01

Ldoppea mentioned this pull request Aug 26, 2024

Meta PR: Implement offline support #1239

Open

2 tasks

zatteo reviewed Sep 2, 2024

View reviewed changes

src/pouchdb/sendDbByEmail.ts Show resolved Hide resolved

zatteo reviewed Sep 2, 2024

View reviewed changes

src/components/webviews/jsInteractions/jsCozyInjection.ts Show resolved Hide resolved

zatteo reviewed Sep 2, 2024

View reviewed changes

Ldoppea force-pushed the feat/open_offline branch 2 times, most recently from 289a7b9 to 27ed108 Compare September 24, 2024 11:09

Ldoppea mentioned this pull request Sep 24, 2024

feat: Prevent local PouchDB modification to be replicated to remote DB #1243

Closed

feat: Add react-native modules for PouchDB

9edfd8b

`react-native-quick-sqlite` is enforced to version `8.0.6` in order to be compatible with RN0.72 Related article: https://dev.to/craftzdog/a-performant-way-to-use-pouchdb7-on-react-native-in-2022-24ej

Ldoppea added 16 commits September 25, 2024 15:49

feat: Configure PouchDB to run on the project

6f9d3ec

Related article: https://dev.to/craftzdog/a-performant-way-to-use-pouchdb7-on-react-native-in-2022-24ej

feat: Add cozy-pouch-link module

6d95b9d

feat: Add PouchDBFind plugin to the PouchDB configuration

4badb35

This plugin is required by cozy-client to process `find` queries Related PR: cozy/cozy-client#1507

feat: Upgrade cozy-intent to 2.23.0

b5e9ba0

`cozy-intent` has been upgraded to `2.23.0` to retrieve new interfaces for FlagshipLink and files downloading Related PR: cozy/cozy-libs#2562

feat: Add react-native-mail plugin

ffd0b10

feat: Declare offline support in cozy-apps injected metadata

fa2d727

feat: Prevent local PouchDB modification to be replicated to remote DB

749ef40

For now we don't want to replicate local changes into remote PouchDB as we first want to ensure everything works as expected locally before activating two-way replication in the future

Ldoppea force-pushed the feat/meta_offline branch from a2a1b6a to d0ae666 Compare September 25, 2024 14:02

Ldoppea force-pushed the feat/open_offline branch from 27ed108 to 749ef40 Compare September 25, 2024 14:02

Ldoppea merged commit b3b1a4b into feat/meta_offline Dec 11, 2024
1 check passed

Ldoppea deleted the feat/open_offline branch December 11, 2024 14:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Implement offline support through CozyPouchLink #1209

feat: Implement offline support through CozyPouchLink #1209

Ldoppea commented May 28, 2024 •

edited

Loading

zatteo Sep 2, 2024

Ldoppea Sep 24, 2024

zatteo Sep 2, 2024

Ldoppea Sep 24, 2024

zatteo Sep 2, 2024 •

edited

Loading

Ldoppea Sep 24, 2024

zatteo commented Sep 2, 2024 •

edited

Loading

zatteo Sep 2, 2024

Ldoppea Sep 24, 2024

zatteo commented Sep 2, 2024

feat: Implement offline support through CozyPouchLink #1209

feat: Implement offline support through CozyPouchLink #1209

Conversation

Ldoppea commented May 28, 2024 • edited Loading

zatteo Sep 2, 2024

Choose a reason for hiding this comment

Ldoppea Sep 24, 2024

Choose a reason for hiding this comment

zatteo Sep 2, 2024

Choose a reason for hiding this comment

Ldoppea Sep 24, 2024

Choose a reason for hiding this comment

zatteo Sep 2, 2024 • edited Loading

Choose a reason for hiding this comment

Ldoppea Sep 24, 2024

Choose a reason for hiding this comment

zatteo commented Sep 2, 2024 • edited Loading

zatteo Sep 2, 2024

Choose a reason for hiding this comment

Ldoppea Sep 24, 2024

Choose a reason for hiding this comment

zatteo commented Sep 2, 2024

Ldoppea commented May 28, 2024 •

edited

Loading

zatteo Sep 2, 2024 •

edited

Loading

zatteo commented Sep 2, 2024 •

edited

Loading