Implement File Provider file synchronisation engine for macOS #5527

claucambra · 2023-03-15T12:17:54Z

This PR properly implements virtual files on macOS by implementing a sync engine using Apple's FileProvider framework.

Closes #1337

github-actions

⚠️ clang-tidy found issue(s) with the introduced code (1/2)

src/gui/macOS/fileprovidersocketserver_mac.mm

shell_integration/MacOSX/NextcloudIntegration/NCDesktopClientSocketKit/LineProcessor.h

src/gui/macOS/fileproviderdomainmanager_mac.mm

shell_integration/MacOSX/NextcloudIntegration/FinderSyncExt/FinderSyncSocketLineProcessor.h

shell_integration/MacOSX/NextcloudIntegration/FinderSyncExt/FinderSync.h

shell_integration/MacOSX/NextcloudIntegration/NCDesktopClientSocketKit/LocalSocketClient.m

src/gui/macOS/fileprovider_mac.mm

..._integration/MacOSX/NextcloudIntegration/NCDesktopClientSocketKit/NCDesktopClientSocketKit.h

shell_integration/MacOSX/NextcloudIntegration/NCDesktopClientSocketKit/LocalSocketClient.h

github-actions

⚠️ clang-tidy found issue(s) with the introduced code (2/2)

src/gui/macOS/fileprovider.h

shell_integration/MacOSX/NextcloudIntegration/FinderSyncExt/FinderSyncSocketLineProcessor.m

codecov · 2023-03-15T13:05:43Z

Codecov Report

Merging #5527 (23855a1) into master (9b77da6) will decrease coverage by 0.16%.
The diff coverage is 0.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5527      +/-   ##
==========================================
- Coverage   59.35%   59.20%   -0.16%     
==========================================
  Files         143      143              
  Lines       18461    18469       +8     
==========================================
- Hits        10958    10934      -24     
- Misses       7503     7535      +32

Impacted Files	Coverage Δ
src/libsync/configfile.cpp	`25.68% <0.00%> (-0.34%)`	⬇️

... and 5 files with indirect coverage changes

src/gui/application.cpp

src/gui/macOS/fileprovider_mac.mm

src/gui/macOS/fileprovider.h

src/gui/macOS/fileprovidersocketserver.h

src/gui/macOS/fileproviderdomainmanager.h

mgallien · 2023-03-15T13:30:45Z

src/gui/macOS/fileproviderdomainmanager_mac.mm

+    return domainIdentifierForAccount(account.get());
+}
+
+QString domainDisplayNameForAccount(const OCC::Account * const account)


why not OCC::AccountPtr ?

We do have one with OCC::AccountPtr but for convenience, we can just overload rather than having to do the conversion each time depending on what we receive

so the question becomes why do you handle raw Account pointers instead of using OCC::AccountPtr ?

Because this is what we receive from the Account/PushNotification signals:

Pushnotifications:

void filesChanged(OCC::Account *account);

Account:

void pushNotificationsReady(OCC::Account *account);

the I would say push an extra commit that is changing

emit pushNotificationsReady(this);

to

emit pushNotificationsReady(sharedFromThis());

the current API is confusing because we mix AccountPtr and Account*
someone is going to be confused when reading this code (and yes I was)

I would say this is out of scope as it affects some other parts of the client -- I'll open a new PR with these changes

I opened #5543 for this

Note to self: adjust to changes once merged

src/gui/macOS/fileproviderdomainmanager_mac.mm

CMakeLists.txt

src/gui/application.cpp

src/gui/macOS/fileprovider.h

src/gui/macOS/fileproviderdomainmanager.h

mgallien · 2023-03-21T17:24:06Z

src/gui/macOS/fileproviderdomainmanager_mac.mm

+    return domainIdentifierForAccount(account.get());
+}
+
+QString domainDisplayNameForAccount(const OCC::Account * const account)


the I would say push an extra commit that is changing

emit pushNotificationsReady(this);

to

emit pushNotificationsReady(sharedFromThis());

the current API is confusing because we mix AccountPtr and Account*
someone is going to be confused when reading this code (and yes I was)

sonarcloud · 2023-05-12T09:04:51Z

SonarCloud Quality Gate failed.

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

0.0% Coverage
0.0% Duplication

berezovskyi · 2023-05-12T15:38:13Z

Thank you for this feature, been waiting for it eagerly! 🎉 🕺 💃

Where do we find the relevant macOS builds and what requirements does it put on the server version, please? Or is switching to the beta channel on the Desktop app enough?

claucambra · 2023-05-15T03:28:19Z

Thank you for this feature, been waiting for it eagerly! 🎉 🕺 💃

Glad to hear it! :)

Where do we find the relevant macOS builds and what requirements does it put on the server version, please?

Check the linked issue thread, there is a link to download a beta build in the announcement post

Or is switching to the beta channel on the Desktop app enough?

No, this module will not be provided in the beta builds under major version 3.8, but it will now be part of the daily builds

Still, we recommend the use of the beta build linked above for now. I'm working on releasing a second beta with additional improvements and bugfixes soon :)

…-try-2" This reverts commit c4d1211, reversing changes made to 9b77da6.

…-try-2" This reverts commit c4d1211, reversing changes made to 9b77da6. Signed-off-by: Claudio Cambra <[email protected]>

core-code · 2023-06-13T13:18:14Z

please tell me there is a way to turn this off! we just switched away from Dropbox because the integration with the native macOS cloud API makes it completely unusable.

Freshou · 2023-06-13T14:59:41Z

Here : https://github.com/nextcloud/desktop/releases

it say it have been implemented in version 3.9 ...

But I see nothing to activate Sync with File Provider ...

Did I missed a step to activate it ???

And yes I added macFileProviderModuleEnabled=True to nextcloud.cfg ....

berezovskyi · 2023-06-13T15:27:24Z

@Freshou it seems to have been reverted without explanation: #5772

misku · 2023-06-13T15:33:49Z

@claucambra could we ask why the feature was removed from 3.9?
The log is a bit confusing at the moment:

Implement File Provider file synchronisation engine for macOS by @claucambra in Implement File Provider file synchronisation engine for macOS #5527
...
Revert inclusion of File Provider module in 3.9 branch by @claucambra in Revert inclusion of File Provider module in 3.9 branch #5772

core-code · 2023-06-13T17:02:38Z

And yes I added macFileProviderModuleEnabled=True to nextcloud.cfg ....

does that mean i can macFileProviderModuleEnabled=False to prevent this from ever becoming enabled?

BJKle · 2023-06-16T10:37:05Z

@claucambra If it has been disabled or even removed from 3.9, is there a target release?
Thank you

claucambra · 2023-06-16T10:41:12Z

We have unfortunately had to remove the File Provider module from 3.9 due to a major regression which broke the Finder Sync extension.

We will instead be releasing a further macOS VFS beta with additional improvements and bugfixes as we continue to work on this.

Our target stable release for a stable File Provider module will be 3.10; in the mean time, expect more VFS betas :)

BJKle · 2023-06-16T11:44:07Z

Thanks for the update

…er-try-2" This reverts commit 21c5dec.

BJKle · 2023-08-07T12:49:55Z

Hi, ist there any update on a future release or beta? This feature is so nice on a windows PC and I really miss it on MacOS.

usethevibe · 2023-08-24T10:37:55Z

Hi evevryone! are there news??

ghost · 2023-09-01T11:47:08Z

Any updates?

m-emelchenkov · 2023-09-01T11:57:46Z

Don't get me wrong, I don't want to be negative. But if it will be ready anytime soon, don't expect much. Tens and tens of thousands of files (i.e. for real-time sync any Node projects and Git repositories) can't handle any existing software on the market with FileProvide API support: Synology Drive, Resilio Connect, StrongSync, SeaDrive, OneDrive for Business. I tried a lot of software and I can indirectly guess two problems: a) issues with FileProvider API itself — simply works not very good, b) issues with dev. teams — they simply can't program big workloads well, lack of expertise. So, I doubt Nextcloud can change current landscape, but wish them all luck.

core-code · 2023-09-01T15:23:22Z

So, I doubt Nextcloud can change current landscape, but wish them all luck.

thats why the only thing i am interested here is whether there will be a way to turn if off when this lands.

BJKle · 2023-09-13T11:29:24Z

We have unfortunately had to remove the File Provider module from 3.9 due to a major regression which broke the Finder Sync extension.

We will instead be releasing a further macOS VFS beta with additional improvements and bugfixes as we continue to work on this.

Our target stable release for a stable File Provider module will be 3.10; in the mean time, expect more VFS betas :)

@claucambra
Is that still true. It looks like 3.10 will be released soon, which is great news.
Where can I find the necessary documentation?

claucambra · 2023-11-13T07:31:51Z

So, I doubt Nextcloud can change current landscape, but wish them all luck.

thats why the only thing i am interested here is whether there will be a way to turn if off when this lands.

There will be a separate release explicitly with File Provider sync. We have no plans to replace the conventional sync client

gnordli · 2023-11-13T15:18:27Z

Hi. Are you targeting this for 3.11 now?
thanks,
Geoff

Sidirius · 2024-02-14T13:06:38Z

Any updates on this?
Is there a planned target?

jackbeeby · 2024-05-27T06:05:58Z

Any updates or release? Possibly a beta?

claucambra · 2024-05-27T11:09:38Z

This has already shipped, see #6656 :)

claucambra requested review from mgallien, camilasan and allexzander March 15, 2023 12:17

claucambra self-assigned this Mar 15, 2023

github-actions bot requested changes Mar 15, 2023

View reviewed changes

src/gui/macOS/fileprovider.h Outdated Show resolved Hide resolved

shell_integration/MacOSX/NextcloudIntegration/FinderSyncExt/FinderSyncSocketLineProcessor.m Show resolved Hide resolved

mgallien requested changes Mar 15, 2023

View reviewed changes

claucambra force-pushed the feature/file-provider-try-2 branch from 857df97 to 80831a4 Compare March 15, 2023 15:10

claucambra requested a review from mgallien March 15, 2023 15:34

claucambra force-pushed the feature/file-provider-try-2 branch 14 times, most recently from 063754f to b4d8290 Compare March 20, 2023 15:09

claucambra added os: 🍎 macOS feature: 💽 virtual filesystem beta labels Mar 20, 2023

claucambra force-pushed the feature/file-provider-try-2 branch from 3ac84ea to bd1ae98 Compare March 21, 2023 15:32

mgallien requested changes Mar 22, 2023

View reviewed changes

claucambra requested a review from mgallien March 22, 2023 14:34

claucambra merged commit c4d1211 into master May 12, 2023

claucambra deleted the feature/file-provider-try-2 branch May 12, 2023 09:05

claucambra added a commit that referenced this pull request Jun 6, 2023

Revert "Merge pull request #5527 from nextcloud/feature/file-provider…

10847fb

…-try-2" This reverts commit c4d1211, reversing changes made to 9b77da6.

claucambra added a commit that referenced this pull request Jun 6, 2023

Revert "Merge pull request #5527 from nextcloud/feature/file-provider…

99b6c45

…-try-2" This reverts commit c4d1211, reversing changes made to 9b77da6. Signed-off-by: Claudio Cambra <[email protected]>

claucambra added a commit that referenced this pull request Jun 7, 2023

Revert "Merge pull request #5527 from nextcloud/feature/file-provider…

be88d7a

…-try-2" This reverts commit c4d1211, reversing changes made to 9b77da6. Signed-off-by: Claudio Cambra <[email protected]>

claucambra added a commit that referenced this pull request Jun 7, 2023

Revert "Merge pull request #5527 from nextcloud/feature/file-provider…

21c5dec

…-try-2" This reverts commit c4d1211, reversing changes made to 9b77da6. Signed-off-by: Claudio Cambra <[email protected]>

claucambra added a commit that referenced this pull request Jul 3, 2023

Unrevert "Merge pull request #5527 from nextcloud/feature/file-provid…

619f458

…er-try-2" This reverts commit 21c5dec.

mgallien added this to the 3.9.0 milestone Jul 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement File Provider file synchronisation engine for macOS #5527

Implement File Provider file synchronisation engine for macOS #5527

claucambra commented Mar 15, 2023 •

edited

Loading

github-actions bot left a comment

github-actions bot left a comment

codecov bot commented Mar 15, 2023 •

edited

Loading

mgallien Mar 15, 2023

claucambra Mar 15, 2023

mgallien Mar 15, 2023

claucambra Mar 15, 2023

mgallien Mar 21, 2023

claucambra Mar 22, 2023

claucambra Mar 22, 2023

mgallien Mar 21, 2023

sonarcloud bot commented May 12, 2023

berezovskyi commented May 12, 2023

claucambra commented May 15, 2023 •

edited

Loading

core-code commented Jun 13, 2023

Freshou commented Jun 13, 2023

berezovskyi commented Jun 13, 2023

misku commented Jun 13, 2023

core-code commented Jun 13, 2023

BJKle commented Jun 16, 2023

claucambra commented Jun 16, 2023 •

edited

Loading

BJKle commented Jun 16, 2023

BJKle commented Aug 7, 2023

usethevibe commented Aug 24, 2023

ghost commented Sep 1, 2023

m-emelchenkov commented Sep 1, 2023

core-code commented Sep 1, 2023

BJKle commented Sep 13, 2023

claucambra commented Nov 13, 2023 •

edited

Loading

gnordli commented Nov 13, 2023

Sidirius commented Feb 14, 2024

jackbeeby commented May 27, 2024

claucambra commented May 27, 2024

Implement File Provider file synchronisation engine for macOS #5527

Implement File Provider file synchronisation engine for macOS #5527

Conversation

claucambra commented Mar 15, 2023 • edited Loading

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

codecov bot commented Mar 15, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarcloud bot commented May 12, 2023

berezovskyi commented May 12, 2023

claucambra commented May 15, 2023 • edited Loading

core-code commented Jun 13, 2023

Freshou commented Jun 13, 2023

berezovskyi commented Jun 13, 2023

misku commented Jun 13, 2023

core-code commented Jun 13, 2023

BJKle commented Jun 16, 2023

claucambra commented Jun 16, 2023 • edited Loading

BJKle commented Jun 16, 2023

BJKle commented Aug 7, 2023

usethevibe commented Aug 24, 2023

ghost commented Sep 1, 2023

m-emelchenkov commented Sep 1, 2023

core-code commented Sep 1, 2023

BJKle commented Sep 13, 2023

claucambra commented Nov 13, 2023 • edited Loading

gnordli commented Nov 13, 2023

Sidirius commented Feb 14, 2024

jackbeeby commented May 27, 2024

claucambra commented May 27, 2024

claucambra commented Mar 15, 2023 •

edited

Loading

codecov bot commented Mar 15, 2023 •

edited

Loading

claucambra commented May 15, 2023 •

edited

Loading

claucambra commented Jun 16, 2023 •

edited

Loading

claucambra commented Nov 13, 2023 •

edited

Loading