Skip to content

Commit

Permalink
clean up
Browse files Browse the repository at this point in the history
  • Loading branch information
micmelesse committed Jun 7, 2024
1 parent 048d445 commit 74654c7
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 8 deletions.
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ var/
# Dev
venv

scripts
# Other
.eggs

.vscode
core
scripts
log*
6 changes: 0 additions & 6 deletions flash_attn/flash_attn_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -788,12 +788,6 @@ def flash_attn_func(
deterministic=False,
return_attn_probs=False,
):
print()
print("flash_attn_func")
print("q:", q.shape)
print("k:", k.shape)
print("v:", v.shape)
print("dropout_p:", dropout_p)
"""dropout_p should be set to 0.0 during evaluation
Supports multi-query and grouped-query attention (MQA/GQA) by passing in KV with fewer heads
than Q. Note that the number of heads in Q must be divisible by the number of heads in KV.
Expand Down

0 comments on commit 74654c7

Please sign in to comment.