Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helper function to detect if backend is single-machine #76

Closed
PeteHaitch opened this issue May 25, 2018 · 1 comment
Closed

Helper function to detect if backend is single-machine #76

PeteHaitch opened this issue May 25, 2018 · 1 comment

Comments

@PeteHaitch
Copy link
Contributor

PeteHaitch commented May 25, 2018

It can be helpful to know if the backend is single-machine (e.g., Bioconductor/DelayedArray#20 (comment)). From reading Section 4.1 'Single machine' of http://bioconductor.org/packages/devel/bioc/vignettes/BiocParallel/inst/doc/Introduction_To_BiocParallel.pdf, I derived the following:

.isSingleMachineBackend <- function(BPPARAM) {
    if (is(BPPARAM, "SerialParam") || is(BPPARAM, "MulticoreParam")) {
        return(TRUE)
    } else if (is(BPPARAM, "SnowParam")) {
        if (is.numeric(bpworkers(BPPARAM)) &&
            BPPARAM$.clusterargs$type == "SOCK") {
            return(TRUE)
        } else {
            return(FALSE)
        }
    } else if (is(BPPARAM, "DoparParam")) {
        # TODO: Can't figure this one out
    } else {
        return(FALSE)
    }
}

I can't quite figure out the logic for DoparParam. Could something like this be added to BiocParallel?

@mtmorgan
Copy link
Collaborator

mtmorgan commented May 26, 2018

I would instead take the approach of trying to determine the computer's host name, e.g., via Sys.info()["nodename"] (but I'm forgetting something about when this fails...), perhaps as part of a unique id, e.g., ipcid(Sys.info()["nodename"]). This would be useful if, for instance, there were both local and and non-local processes in a socket cluster.

I also sketched out a distributed lock using httpuv, where the 'server' would run in one process (e.g., on the host machine) and the 'client' in separate processes. It can be tested by sourcing the server in one R instance, and sourcing the client in another. It would not be hard to further wrap this so that the runServer() command was started in a forked (via mcparallel()) or spawned (by starting a SNOW cluster of size 1 and evaluating there) or maybe using the later paradigm from httpuv. The challenge here is that this would be difficult to make robust -- e.g., the server has to be able to listen on particular ports, and the clients need to talk over those ports, but maybe that robustness is required in at least some of the back-ends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants