Reorganize tests to make them easier to find (#768) #16
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add eval loader to eval script * small input tests * updates * fix typing and formatting * fixes, add tests * remove circular dependency * tests pass * nits + small fixes * add metrics at the end, refactor to put icl/gauntlet as helpers * NOT * metrics instead of models, add unit tests * Move tests into directories * add copyright to inits * fix relative paths * fixes * revert gauntlet test change * Support inputs_embeds (#687) * support inputs_embeds * update tests to test inputs_embeds * make iids optional inputs to fwd * remove check for both iids and inputs_embeds in MPTForCausalLM. It is checked in the base model, and it is actually a common practice to pass both during autoregressive generation. Embeds are used first, then once the kvcache is nonempty, iids are used instead * reorder kwargs * add more tests * fix device merge artifact in test_model.oy * fix generate test * yapf * Better error message when test does not complete (#769) * run script tests first * comment out * ascripts -> scripts * bad dirs * try this * hacks * add a note about a_scripts --------- Co-authored-by: Sam Havens <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add eval loader to eval script
small input tests
updates
fix typing and formatting
fixes, add tests
remove circular dependency
tests pass
nits + small fixes
add metrics at the end, refactor to put icl/gauntlet as helpers
NOT
metrics instead of models, add unit tests
Move tests into directories
add copyright to inits
fix relative paths
fixes
revert gauntlet test change
Support inputs_embeds (Support inputs_embeds mosaicml/llm-foundry#687)
support inputs_embeds
update tests to test inputs_embeds
make iids optional inputs to fwd
remove check for both iids and inputs_embeds
in MPTForCausalLM. It is checked in the base model, and it is actually a common practice to pass both during autoregressive generation. Embeds are used first, then once the kvcache is nonempty, iids are used instead
reorder kwargs
add more tests
fix device merge artifact in test_model.oy
fix generate test
yapf
Better error message when test does not complete (Better error message when test does not complete mosaicml/llm-foundry#769)
run script tests first
comment out
ascripts -> scripts
bad dirs
try this
hacks
add a note about a_scripts