Fixes process overload from file events

[acedrew]

Description

Changes file watcher classes to use "on_closed" rather than "on_any_change" event type, reducing frequency of events.
Fixes #3172

Type of change

Please delete options that are not relevant.

• [*] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Deployed to platforms that were experiencing extreme resource utilization and verified utilization dropped significantly
• Deployed to platforms with ongoing system profiling and verified the hotpaths in the platforms changed.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code

[craig8]

@acedrew This actually breaks the reloading of the web-users.json file.

[acedrew]

Why does it break that particular file and not any of the other ones?

…

On Mon, May 6, 2024, 1:18 PM Craig ***@***.***> wrote: @acedrew <https://github.com/acedrew> This actually breaks the reloading of the web-users.json file. However I found a fix, remove gevent_watchdog and just use watchdog. I am testing now to make sure it doesn't have any other concequences. — Reply to this email directly, view it on GitHub <#3175 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHOUB3KCGRWLL2VN3ZUIZLZA63M5AVCNFSM6AAAAABGTN2WAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJWGUZTIMBRGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[craig8]

I don't know...my guess is that we are reading auth.json in more than we do the web. So there are places that read it regardless of it being actually run through the event itself. Web users is only read/written in a couple of places.

[craig8]

I suppose the other thing could be some gevent stuff happening that unless you use all_events then it's not getting propagated correctly? Not 100% sure

[craig8]

@acedrew I think I found the issue. So the webusers uses a PersistentDict to write the data to a file. It writes data to a temp file and the copies the file into the proper space. This apparently is not what the filewatch mechanisms is expecting so it sends an infinite list of opened events! Sigh!