[Bug]: Linux Desktop Client is constantly syncing and writing to disc even no changes happened

[dishorned]

⚠️ Before submitting, please verify the following: ⚠️

• This is a bug, not a question or a configuration issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server and Desktop Client are up to date. See Server Maintenance and Release Schedule and Desktop Releases for supported versions.
• I agree to follow Nextcloud's Code of Conduct

Bug description

Since one of the last two updates (3.14.1 or 3.14.0) the nextcloud desktop client is syncing every second. This means that the icon changes from green with white checkmark to the blue syncing sign for about half a second, changes back to green sign and again after half of a second back to the syncing sign.

In the system monitor, the nextcloud desktop app writes constantly 1.7 mb/s to disk, which is about 50 GB on a full working day. This happens all the time, even no program is open or writing to the nextcloud folder. It does not occure, when no network connection is present. So maybe it could also be a problem on the nextcloud server.

Steps to reproduce

1. Check that a network connection to the server is possible
2. Start the nextcloud desktop app

Expected behavior

The nextcloud desktop app should only sync, when a new file is created or a file has changed.

Which files are affected by this bug

Don't know which files are affected

Operating system

Linux

Which version of the operating system you are running.

ZorinOS 17.2

Package

Community FlatPak

Nextcloud Server version

29.0.8

Nextcloud Desktop Client version

3.14.1

Is this bug present after an update or on a fresh install?

Updated from a minor version (ex. 3.4.2 to 3.4.4)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

Are you using an external user-backend?

• Default internal user-backend
• LDAP/ Active Directory
• SSO - SAML
• Other

Nextcloud Server logs

No response

Additional info

While this bug is present, I have updated the nextcloud server version from 29.0.7 (i guess) to 29.0.8 but the bug still exists. I am not a 100% sure if I have updated the desktop client as well, but I think the problem started with version 3.14.0 and I have updated it to 3.14.1 without success. I think the problem started after a desktop client update and not after a server update.

I have also tried to reinstall the nextcloud desktop client and deleted the local copies of all files and resynced them without success as well.

I am not syncing local data of the nextcloud server, the data are connected by smb to the nextcloud server from a Qnap NAS. I connected it via the GUI of Nextcloud and not via fstab or something on the linux server where nextcloud is installed.

[limes007]

Same or similar issue here. For me its syncing every ~7 seconds. I'm using v 3.14.1
Regarding the huge writes to disk, please look at #5302. I managed to disable the logging, but it still sync every ~7 seconds.

#=#=#=# Syncrun started 2024-10-16T12:22:25Z
#=#=#=#=# Propagation starts 2024-10-16T12:22:28Z (last step: 2802 msec, total: 2802 msec)
#=#=#=# Syncrun finished 2024-10-16T12:22:29Z (last step: 1393 msec, total: 4196 msec)
#=#=#=# Syncrun started 2024-10-16T12:22:32Z
#=#=#=#=# Propagation starts 2024-10-16T12:22:35Z (last step: 2786 msec, total: 2786 msec)
#=#=#=# Syncrun finished 2024-10-16T12:22:37Z (last step: 1371 msec, total: 4158 msec)
#=#=#=# Syncrun started 2024-10-16T12:22:40Z
#=#=#=#=# Propagation starts 2024-10-16T12:22:43Z (last step: 2896 msec, total: 2896 msec)
#=#=#=# Syncrun finished 2024-10-16T12:22:44Z (last step: 1521 msec, total: 4418 msec)
#=#=#=# Syncrun started 2024-10-16T12:22:48Z
#=#=#=#=# Propagation starts 2024-10-16T12:22:50Z (last step: 2732 msec, total: 2732 msec)
#=#=#=# Syncrun finished 2024-10-16T12:22:52Z (last step: 1332 msec, total: 4064 msec)
#=#=#=# Syncrun started 2024-10-16T12:22:55Z
#=#=#=#=# Propagation starts 2024-10-16T12:22:58Z (last step: 2794 msec, total: 2794 msec)
#=#=#=# Syncrun finished 2024-10-16T12:22:59Z (last step: 1412 msec, total: 4206 msec)
#=#=#=# Syncrun started 2024-10-16T12:23:02Z
#=#=#=#=# Propagation starts 2024-10-16T12:23:05Z (last step: 2788 msec, total: 2788 msec)
#=#=#=# Syncrun finished 2024-10-16T12:23:07Z (last step: 1414 msec, total: 4203 msec)

Update: I downgraded temporarily to version 3.13.4 but its the same here.

[dishorned]

hmmm, I am wondering if my ssd-lifespan got eaten by the log files since the beginning of using nextcloud. I am also wondering, why they ignore this since years, because this can heavily impact the lifespan of all user's ssd.

I checked it on my machine and it seems to write a lot of log-files as well. It also compresses the logfile 25 times a minute. So every minute I get 25 new .gz files. Due to other users posts and bug-reports, I think this problem could be existing since I started using nextcloud a couple of years ago.

On the other hand, the syncing problem exists only since one of the last two updates. I have never seen changing the symbol so frequently. Normally the sync process only starts when new files saved to the nextcloud folder or existing ones got updated.

Currently the nextcloud app is on "pause sync", so no log files really will be written. When I write/edit any file I manually start syncing once. This really is a pain in the ass. But the real problem is not the syncing, it's more the huge amount of files written. I thought this is just one problem, but know I think when the sync problem is solved, the writing problem could still exist.

How do you disabled the logging? Like how the user thvitt did it on the bugreport you mentioned, or another way?

[limes007]

See my comment here #5302 (comment)

[limes007]

I had some errors in the server side-protocol, notably missing permissions on directories which blocked full scanning of files.
I fixed them all and now... my client no longer syncs every x seconds. So it looks like the server is triggering all these syncs.

[thorsten-schwartz]

I had some errors in the server side-protocol, notably missing permissions on directories which blocked full scanning of files. I fixed them all and now... my client no longer syncs every x seconds. So it looks like the server is triggering all these syncs.

Hi, seems to be the same use case than mine. What permissions have been missing in your Samba server? I am running Samba in docker environment.

Greets Thorsten

[dishorned]

I tried to only sync a local nextcloud server folder, then the sync and write problem were gone. A nex logfile was written once every several minutes or so and not 25 times a minute. To be honest, there was only 1 file in it, while there are hundreds and thousands in the smb-folder I normally sync.

On the Qnap nas (smb-server) the share has read/write/execute for owner/group/others (777 in linux). The user who accesses the share has read/write (execute cannot be configured) access. There aren't really more things you can change on the GUI. So I am also really interessted in what you have changed.

[limes007]

I do not use Samba, I use local external storage. This storage contained some sub-folders that were not accessible to the Nextcloud user. As a result, the file scan couldn't complete. I changed the file permissions and now the file scan is complete and the constant syncing on the client side is gone.

If you already have 777 on all files and (sub-)folders, there may be another problem on your end. You should look at the server protocol and try to resolve any errors listed there.

[dishorned]

Yesterday I realized, that after I changed the folder to synchronize from the smb-folder to the local nextcloud folder and then changing back, it does not sync the whole smb-folder again. There were some files missing, even the client doesn't show anything. Maybe I have to reproduce that and create a new bug report.

Back to this issue, I changed the log-level on the server to debug and let the client active for about 30 seconds as it produces a lot of entries and viewing the server log became really slow and laggy since they changed it a couple of versions ago. I know I can download the log and have a view in another program, but for this time it was ok. The log only showed some dirty tables read. Don't know if they have anything to to with the client.

dirty table reads: SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `filecache`.`name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `filecache`.`permissions`, `checksum`, `unencrypted_size`, `metadata_etag`, `creation_time`, `upload_time`, `meta`.`json` AS `meta_json`, `meta`.`sync_token` AS `meta_sync_token` FROM `*PREFIX*filecache` `filecache` LEFT JOIN `*PREFIX*filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` LEFT JOIN `*PREFIX*files_metadata` `meta` ON `filecache`.`fileid` = `meta`.`file_id` WHERE `filecache`.`parent` = :dcValue1 ORDER BY `name` ASC

dirty table reads: SELECT `name`, `propertypath`, `propertyname`, `propertyvalue`, `valuetype` FROM `*PREFIX*filecache` `f` LEFT JOIN `*PREFIX*properties` `p` ON (`propertypath` = CONCAT(:dcValue1, `name`)) AND (`userid` = :dcValue2) WHERE `parent` = :dcValue3

dirty table reads: SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `*PREFIX*share` `s` LEFT JOIN `*PREFIX*filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `*PREFIX*storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`s`.`file_source` = :dcValue1) AND (`s`.`share_type` = :dcValue2) AND (`s`.`share_with` IN (:dcValue3)) AND ((`s`.`item_type` = :dcValue4) OR (`s`.`item_type` = :dcValue5)) ORDER BY `s`.`id` ASC

Any ideas?

[Christopher-Hayes]

I'm also seeing this issue with "external storage" folders. In my case, my external storage is an S3 bucket. The fact that it also impacts S3 bucket external storage, makes me think it's a broader issue.

Desktop app: 3.14.1 (Flatpak)
Server: 29.0.4 (Docker)

My server does use SQLite for the database since it's a single-user instance. This is the only file-sync issue I've had.

[thorsten-schwartz]

Hi,
I use a Samba Docker container for Nextcloud as an external drive. If I now enable this external drive to be synchronization with my Liux desktop client for Netxcoud, the 2s autosync problem arises.

The Samba Docker log then shows then:
idmap range not specified for domain '*'

Does anyone have an idea?

[s0ww0s]

Hi,

I only have external storage linked to Nextcloud via SMB.

For now, I've temporarily downgraded to version 3.12.3 (Flatpak). It's not a permanent solution either... I can't say why this version works, but maybe it’ll help someone.

Nextloud works fine on Mac and Windows, but on Linux it's not working with the newer versions.

[nursoda]

I also see the "re-sync every other second" issue for one of my instances, but I realized another cause: Do you by any chance use shares with read-only permission? If so, please check if the issue is resolved by these steps:

1. make sure all local files are also on the server
2. remove all read-only folders from the desktop settings
3. stop the desktop client
4. move the corresponding local folders aside (outside any synced folder)
5. restart the desktop client

If you now no longer see the "sync every other second" issue, the local write permissions (and their handling by the client) are the culprit. Probably check our #7318. This goes back on #6296 which landed in 3.13 in March 2024. I just filed #7586 "Issue 2" describes in detail what I was referring to here, but this seems to be 3.15+ specific.

[dishorned]

For me, the answer is "no". I am the only one using nextcloud and I configured read/write permissions in the smb-connector on the nextcloud GUI. The underlying QNAP NAS also has read/write permission for the user, who connects from nextcloud to Qnap. So I should have read/write permissions on any folder/file.

In my last post I have written, that after changing to a local nextcloud folder only, the sync worked, but after changing back to the smb folder, not all files were syncing correctly. So something under the hood seems to make problems, but I had no time yet to really check that again and create a bug report. But both problems could be related.

Back to the problem about the excessive syncing, I get several log entries for ALL files on the nextcloud. So more files you have, more log entries exist. Here are some important log-entry-examples. The first couple of lines show the start of the sync process, while line 7 has a list of all files and folders, which are several hundred. The three lines in the middle exist for all files in the nextcloud I think and the last line shows the successful sync status.

The logfile has a size of about 10 MB with thousands of lines an will be created about 25 times a minute. Hard to analyse that to find out what the problem could be. May someone else have an idea, otherwise the desktop client is nearly useless as currently I only sync once a day.

[ info nextcloud.gui.folder.manager /run/build/nextcloud-client/src/gui/folderman.cpp:702 ]:	Schedule folder  "1"  to sync!
[ info nextcloud.gui.application /run/build/nextcloud-client/src/gui/owncloudgui.cpp:251 ]:	Sync state changed for folder  "$url$" :  "Not yet started"
[ info nextcloud.gui.folder.manager /run/build/nextcloud-client/src/gui/folderman.cpp:883 ]:	Starting the next scheduled sync in 0 seconds
[ info nextcloud.gui.application /run/build/nextcloud-client/src/gui/owncloudgui.cpp:251 ]:	Sync state changed for folder  "$url$" :  "Preparing to sync"
[ info nextcloud.gui.folder /run/build/nextcloud-client/src/gui/folder.cpp:1046 ]:	*** Start syncing  "$url$"  - Nextcloud client version 3.14.3daily
[ info nextcloud.gui.folder /run/build/nextcloud-client/src/gui/folder.cpp:1080 ]:	Allowing local discovery to read from the database
[ info nextcloud.sync.engine /run/build/nextcloud-client/src/libsync/syncengine.cpp:1220 ]:	paths to discover locally $all_files_and_folders$

[ info nextcloud.sync.discovery /run/build/nextcloud-client/src/libsync/discovery.cpp:86 ]:	STARTING "$folder$" OCC::ProcessDirectoryJob::ParentNotChanged "$folder$" OCC::ProcessDirectoryJob::NormalQuery
[ info nextcloud.sync.discovery /run/build/nextcloud-client/src/libsync/discovery.cpp:550 ]:	"Processing \"$file$" | (db/local/remote) | valid: true/true/db | mtime: 1674661555/1674661555/0 | size: 87804/87804/0 | etag: \"645b41adc0401\"//\"\" | checksum: \"SHA1:ee1e52221674da278b60fb7a58c42a2965816b55\"//\"\" | perm: \"WDNVm\"//\"\" | fileid: \"00074387oc0micu9xkf5\"//\"\" | type: CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeFile/CSyncEnums::ItemTypeFile | e2ee: false/false | e2eeMangledName: \"\"/\"\" | file lock: not locked// | file lock type: \"\"//\"\" | metadata missing: /false/"
[ info nextcloud.gui.folder /run/build/nextcloud-client/src/gui/folder.cpp:641 ]:	Ignoring spurious notification for file "$filename$"

[ info nextcloud.gui.application /run/build/nextcloud-client/src/gui/owncloudgui.cpp:251 ]:	Sync state changed for folder  "$url$" :  "Success, some files were ignored."```