Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename Plugin to AttentionAndDistributedPacking #64

Merged
merged 1 commit into from
Aug 14, 2024

Conversation

achew010
Copy link
Contributor

@achew010 achew010 commented Aug 7, 2024

Description

This PR renames the InstructLab plugin to AttentionAndDistributedPacking. Functionally, it is the same behaviour as the current plugin but all references of instructlab will be replaced.

export DATASET=benchmark_outputs/data/cache_mistralai_Mistral_7B_v0.1.json
python -m tuning.sft_trainer --model_name_or_path mistralai/Mistral-7B-v0.1 --padding_free huggingface --packing False --max_seq_len 4096 --learning_rate 2e-5 --torch_dtype float16 --use_flash_attn True --include_tokens_per_second True --num_train_epochs 1 --gradient_accumulation_steps 1 --gradient_checkpointing True --evaluation_strategy no --save_strategy no --weight_decay 0.01 --warmup_steps 10 --adam_epsilon 1e-4 --lr_scheduler_type linear --logging_strategy steps --logging_steps 10 --max_steps 100 --training_data_path $DATASET --per_device_train_batch_size 4 --output_dir benchmark_outputs/aadp --skip_memory_metrics False

@achew010 achew010 force-pushed the rename-ilab-plugin branch 3 times, most recently from 9330c60 to 142cd60 Compare August 8, 2024 03:13
@achew010 achew010 marked this pull request as ready for review August 8, 2024 03:18
@achew010 achew010 requested a review from fabianlim as a code owner August 8, 2024 03:18
@achew010 achew010 force-pushed the rename-ilab-plugin branch from 142cd60 to 09afb42 Compare August 8, 2024 04:27
@fabianlim
Copy link
Contributor

fabianlim commented Aug 14, 2024

we have renamed the ilab to aadp, we will be deleting the old pypi project https://pypi.org/manage/project/fms-acceleration-ilab/settings/. We will not delete the PYPI project right away, but will be done soon.

@fabianlim fabianlim merged commit 09a3104 into foundation-model-stack:main Aug 14, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants