Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

From toil worker image, abstract out shared worker filesystem #3

Open
theferrit32 opened this issue Jan 22, 2018 · 2 comments
Open

Comments

@theferrit32
Copy link
Member

theferrit32 commented Jan 22, 2018

As a prerequisite for using toil image, require a shared filesystem be set up on all worker nodes, that the toil-workspace used to launch toil can also access.

When toil worker image is run, either mount the filesystem from the worker node as a volume, or accept some sort of argument that tells the container itself what shared filesystem to mount.

Then when toil is run, use the path to that filesystem mount as the --jobStore value, instead of the /renci/irods mount.

@theferrit32
Copy link
Member Author

theferrit32 commented Aug 7, 2018

Launcher and worker containers now work with a hardcoded NFS host mountpoint as an automatically mounted volume, using btrfs as the backing filesystem on the server node. Will look into making this configurable as a parameter. May investigate performance tweaks or other protocols.

@dcvan24
Copy link
Collaborator

dcvan24 commented Aug 7, 2018

Btrfs seems only working on a single node with multiple block devices, but cannot be shared across multiple nodes AFAIK. I think a distributed storage system such as Ceph, Lustre and ZFS may be a better fit for this use case. It needs investigations into these systems to make a wise selection for our environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants