Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove allow list for script-based datasets #3004

Closed
severo opened this issue Jul 30, 2024 · 3 comments · Fixed by #3046
Closed

Remove allow list for script-based datasets #3004

severo opened this issue Jul 30, 2024 · 3 comments · Fixed by #3046
Assignees
@severo
Labels
P1 Not as needed as P0, but still important/wanted security

Comments

@severo
Copy link
Collaborator

severo commented Jul 30, 2024 •
edited
Loading

dataset-viewer/chart/values.yaml

Lines 83 to 86 in 5c5be12

# Comma-separated list of the datasets for which we support dataset scripts.
# Unix shell-style wildcards also work in the dataset name for namespaced datasets, for example `some_namespace/*` to refer to all the datasets in the `some_namespace` namespace.
datasetScriptsAllowList: "hf-internal-testing/dataset_with_script,togethercomputer/RedPajama-Data-1T,togethercomputer/RedPajama-Data-V2,gaia-benchmark/GAIA,poloclub/diffusiondb,mozilla-foundation/common_voice_*,google/fleurs,speechcolab/gigaspeech,espnet/yodas,ylecun/mnist,zalando-datasets/fashion_mnist"
# URL of the HuggingFace Hub

Then remove the code that allow script-based datasets in this repo + exception DatasetModuleNotInstalledError. Related to #2478.
@severo severo added security P1 Not as needed as P0, but still important/wanted labels Jul 30, 2024
@severo
Copy link
Collaborator Author

severo commented Aug 23, 2024 •
edited
Loading

It means moving these datasets from script to data-only.

Already data-only:

@severo severo mentioned this issue Aug 23, 2024
Merged
@severo severo self-assigned this Aug 23, 2024
@severo
Copy link
Collaborator Author

severo commented Aug 23, 2024 •
edited
Loading

we will simply remove the support for script-based datasets. The dataset viewer will still be working for these datasets ^, until some update occurs.

internal discussion: https://huggingface.slack.com/archives/C067U56EQBB/p1724426617131639

@severo severo mentioned this issue Aug 23, 2024
Merged
@severo
Copy link
Collaborator Author

severo commented Aug 23, 2024

PR is here: #3046

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@severo severo

Labels
P1 Not as needed as P0, but still important/wanted security
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

remove support for script-based datasets huggingface/dataset-viewer
1 participant
@severo