Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Listen to filesystem events to support multiple Stores for a shared directory #2

Open
sanity opened this issue Mar 18, 2017 · 2 comments

Comments

@sanity
Copy link
Member

sanity commented Mar 18, 2017

Currently only one Store instance can listen to a given directory at a time (although it isn't currently enforced through locking, although it should be).

However, through the Java WatchService it should be able to listen for file modifications caused by other Store instances, possibly running in a different JVM or even on a different server (using a shared filesystem like Amazon's EFS).

Unfortunately this is complicated by the fact that WatchService doesn't work reliably across platforms. There are workarounds like this Mac specific library, so they will need to be used on platforms where WatchService isn't reliable.

Note: Should verify that Amazon's EFS supports realtime file modification notification before we recommend this use-case.

@sanity
Copy link
Member Author

sanity commented Mar 18, 2017

FYI - I asked a question about this on r/java - https://www.reddit.com/r/java/comments/604rfj/is_there_a_reliable_crossplatform_java_library/

Based on what I've learned so-far, I'm concerned that there may not be a sufficiently reliable and efficient way to be notified of file creations, deletions, or updates, that is cross-platform.

To allow Shoebox to scale horizontally, it may therefore be necessary to build on top of something like Hazelcast.

@sanity
Copy link
Member Author

sanity commented Jun 10, 2017

Tape might be an option for logging changes to a file and allowing multiple Shoebox instances to listen for changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant