Code

Ipanfilo/fa support#96

Code

Open

ipanfilo wants to merge 11 commits intodevROCm/TransformerEngine:devfrom ipanfilo/fa_supportROCm/TransformerEngine:ipanfilo/fa_supportCopy head branch name to clipboard

+85-74

Commits

Commits on Nov 28, 2024

• Fix an argument issue when flash_attn>=2.5.7 (#1068)

  Show description for ec272c9
  

  2 peopleauthored and

  ipanfilo

  committed
  ec272c9

• [PyTorch] Fix issues with cross attention (#1069)

  Show description for 6725d26
  

  2 peopleauthored and

  ipanfilo

  committed
  6725d26

Commits on Nov 29, 2024

• Hide non-necessary symbols from shared object (#1136)

  Show description for 407110d
  

  ksivaman

  authored and

  ipanfilo

  committed
  407110d

Commits on Dec 3, 2024

• ROCm FlashAttn-2 supporting changes

  ipanfilo

  committed
  d59ae1f

• Update tests to support FlashAttn and better support different config

  ipanfilo

  committed
  3855ac0

Commits on Dec 10, 2024

• Enable FA 2.6.0+ and THD related code

  ipanfilo

  committed
  0698559

Commits on Dec 19, 2024

• Support FA2.6+ softmax_lse format for BSHD/SBHD

  ipanfilo

  committed
  f332bb0

Commits on Dec 23, 2024

• Merge dev changes; Disable parallel pytest, disable unsupported FA tests

  ipanfilo

  committed
  5ed885e

Commits on Jan 4, 2025

• Merge dev changes

  ipanfilo

  committed
  264fb34

• Merge dev changes

  ipanfilo

  committed
  87574d2

Commits on Jan 7, 2025

• Fix skip conditions and update readme

  ipanfilo

  committed
  2d505b3