You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As a prerequisite for using toil image, require a shared filesystem be set up on all worker nodes, that the toil-workspace used to launch toil can also access.
When toil worker image is run, either mount the filesystem from the worker node as a volume, or accept some sort of argument that tells the container itself what shared filesystem to mount.
Then when toil is run, use the path to that filesystem mount as the --jobStore value, instead of the /renci/irods mount.
The text was updated successfully, but these errors were encountered:
Launcher and worker containers now work with a hardcoded NFS host mountpoint as an automatically mounted volume, using btrfs as the backing filesystem on the server node. Will look into making this configurable as a parameter. May investigate performance tweaks or other protocols.
Btrfs seems only working on a single node with multiple block devices, but cannot be shared across multiple nodes AFAIK. I think a distributed storage system such as Ceph, Lustre and ZFS may be a better fit for this use case. It needs investigations into these systems to make a wise selection for our environment.
As a prerequisite for using toil image, require a shared filesystem be set up on all worker nodes, that the toil-workspace used to launch toil can also access.
When toil worker image is run, either mount the filesystem from the worker node as a volume, or accept some sort of argument that tells the container itself what shared filesystem to mount.
Then when toil is run, use the path to that filesystem mount as the --jobStore value, instead of the /renci/irods mount.
The text was updated successfully, but these errors were encountered: