Skip to content

Commit

Permalink
Return an error if you try to redownload an already downloaded dataset
Browse files Browse the repository at this point in the history
  • Loading branch information
dadmobile committed Feb 25, 2024
1 parent ac01231 commit 34eb112
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions transformerlab/routers/data.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,12 @@ async def dataset_preview(dataset_id: str):

@router.get("/download", summary="Download a dataset from the HuggingFace Hub to the LLMLab server.")
async def dataset_download(dataset_id: str):
# Check to make sure we don't have a dataset with this name
# Possibly we want to allow redownloading in the future but for we can't add duplicate dataset_id to the DB
row = await db.get_dataset(dataset_id)
if row is not None:
return {"status":"error", "message": f"A dataset with the name {dataset_id} already exists"}

ds_builder = load_dataset_builder(dataset_id)
try:
dataset = load_dataset(dataset_id)
Expand Down

0 comments on commit 34eb112

Please sign in to comment.