Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setup spark standalone #1

Closed
wants to merge 2 commits into from
Closed

Conversation

Tianhao-Gu
Copy link
Collaborator

(dev) tgu@cdm-spark-standalone (master)$docker exec -it spark-user \
    sh -c '
            /opt/bitnami/spark/bin/spark-submit \
                    --master $SPARK_MASTER_URL \
                    /opt/bitnami/spark/examples/src/main/python/pi.py 10 \
                    2>/dev/null
    '
Pi is roughly 3.136920

What's next:
    Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug spark-user
    Learn more at https://docs.docker.com/go/debug-cli/
(dev) tgu@cdm-spark-standalone (master)$

@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@Tianhao-Gu
Copy link
Collaborator Author

code changes are all copied from cdm-jupyterhub repo.

@Tianhao-Gu
Copy link
Collaborator Author

closing PR, we will use old image from ghcr.io/kbase/cdm-jupyterhub for spark standalone mode.

@Tianhao-Gu Tianhao-Gu closed this Aug 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant