Skip to content

Commit

Permalink
Fix d * lr step log
Browse files Browse the repository at this point in the history
  • Loading branch information
rockerBOO committed Nov 21, 2024
1 parent ef9a9c1 commit 75b656d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion train_network.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def generate_step_logs(
args.optimizer_type.lower().endswith("ProdigyPlusScheduleFree".lower())
):
logs[f"lr/d*lr/group{i}"] = (
optimizer.param_groups[0]["d"] * optimizer.param_groups[0]["lr"]
optimizer.param_groups[i]["d"] * optimizer.param_groups[i]["lr"]
)

return logs
Expand Down

0 comments on commit 75b656d

Please sign in to comment.