Skip to content

Commit

Permalink
add debugging statement
Browse files Browse the repository at this point in the history
  • Loading branch information
mattyding committed Dec 23, 2024
1 parent bb757c7 commit 6c3e0a7
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions llmfoundry/data/finetuning/dataloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ def build_finetuning_dataloader(
padding/waste rates for different `cfg.dataset.packing_ratio` choices,
given a starting workload YAML.
"""
print(f"🚨🚨🚨 build_finetuning_dataloader was called with the following arguments: tokenizer={tokenizer}, device_batch_size={device_batch_size}, dataset={dataset}, num_workers={num_workers}, drop_last={drop_last}, pin_memory={pin_memory}, prefetch_factor={prefetch_factor}, persistent_workers={persistent_workers}, timeout={timeout}")

dataset_cfg = dataset
is_streaming = (
dataset_cfg.get('remote') is not None or
Expand Down

0 comments on commit 6c3e0a7

Please sign in to comment.