Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log incoherent state error #3118

Merged
merged 1 commit into from
Dec 10, 2024
Merged

Log incoherent state error #3118

merged 1 commit into from
Dec 10, 2024

Conversation

lhoestq
Copy link
Member

@lhoestq lhoestq commented Dec 10, 2024

I got this which triggered a recompute of the dataset but I don't know why it happened, adding some logs in case it happens again

WARNING: 2024-12-10 10:35:59,554 - root - Dataset HuggingFaceFW/fineweb-2 has incoherent entries in the cache. Let's first delete the dataset, then backfill again.

@lhoestq lhoestq merged commit 7540b32 into main Dec 10, 2024
25 checks passed
@lhoestq lhoestq deleted the log-incoherent-state-error branch December 10, 2024 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant