Update datasets to 2.16.1 (#2231)

* update to 2.16.1 * pass trust_remote_code * don't use parquet export in `datasets` * update hfh to 0.20.1 * HF_DATASETS_TRUST_REMOTE_CODE=0 * style * minor * update tests * missing line * add comment --------- Co-authored-by: Sylvain Lesage <[email protected]>
huggingface · Jan 5, 2024 · 9338523 · 9338523
1 parent 85d5167
commit 9338523
Show file tree

Hide file tree

Showing 26 changed files with 457 additions and 1,993 deletions.
diff --git a/chart/templates/_env/_envDatasetsBased.tpl b/chart/templates/_env/_envDatasetsBased.tpl
@@ -6,5 +6,7 @@
 # note that the /tmp directory is not shared among the pods
 - name: HF_MODULES_CACHE
   value: "/tmp/modules-cache"
+- name: HF_DATASETS_TRUST_REMOTE_CODE
+  value: "0"
 {{- end -}}