Skip to content

Commit

Permalink
Fix d * lr step log
Browse files Browse the repository at this point in the history
  • Loading branch information
rockerBOO committed Nov 29, 2024
1 parent 87f5224 commit 6593cfb
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion train_network.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ def generate_step_logs(
args.optimizer_type.lower().endswith("ProdigyPlusScheduleFree".lower()) and optimizer is not None
):
logs[f"lr/d*lr/group{i}"] = (
optimizer.param_groups[0]["d"] * optimizer.param_groups[0]["lr"]
optimizer.param_groups[i]["d"] * optimizer.param_groups[i]["lr"]
)

return logs
Expand Down

0 comments on commit 6593cfb

Please sign in to comment.