Added an example of Unsplit attack in Split learning LLM

[dixiyao]

Description

Provide three classes on top of the split learning LLM, which are a curious server, a trainer used by the curious server and a model used by the curious server contains a guessed client model for conducting attacks. The attack method is implemented according to Unsplit. But there are two differences made specific to LLM. First, the optimizers used to update the model is changed from SGD to AdamW. Second, the server first reconstructs the embeddings then uses he reconstructed embeddings to reconstruct the private words (token ids). The reason is that the embedding layer cannot backpropagate gradients directly.

The core process is in the function attack in the CuriousTrainer.

How has this been tested?

Run the command

python3 examples/split_learning/llm_split_learning/split_learning_main_attack.py -c examples/split_learning/llm_split_learning/split_learning_wikitext2_gpt2_attack.yml

Types of changes

• Bug fix (non-breaking change which fixes an issue) Fixes #
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code has been formatted using Black and checked using PyLint.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

[netlify]

✅ Deploy Preview for platodocs canceled.

Name	Link
🔨 Latest commit	5f7efd0
🔍 Latest deploy log	https://app.netlify.com/sites/platodocs/deploys/6557f04af364ef0007960ff9