From a7fb38379cc6d45f93a1ccceaafb2137298d27d4 Mon Sep 17 00:00:00 2001
From: Oscar Dowson <o.dowson@gmail.com>
Date: Wed, 13 Nov 2024 13:51:28 +1300
Subject: [PATCH 1/3] [docs] add section and example on common subexpression
 elimination

---
 docs/src/manual/nonlinear.md | 46 ++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/docs/src/manual/nonlinear.md b/docs/src/manual/nonlinear.md
index afd11e58876..72a389e6908 100644
--- a/docs/src/manual/nonlinear.md
+++ b/docs/src/manual/nonlinear.md
@@ -153,6 +153,52 @@ julia> sin(sin(1.0))
 0.7456241416655579
 ```
 
+## Common subexpressions
+
+JuMP does not perform common subexpression elimination. Instead, if you re-use
+an expression in multiple places, JuMP will insert a copy of the expression.
+
+JuMP's lack of common subexpression elimination is a common cause of performance
+problems, particularly in nonlinear models with a pattern like
+`sum(t / common_term for t in terms)`. One example is the logistic loss:
+
+```jldoctest
+julia> model = Model();
+
+julia> @variable(model, x[1:2]);
+
+julia> @expression(model, expr, sum(exp.(x)))
+0.0 + exp(x[2]) + exp(x[1])
+
+julia> @objective(model, Min, sum(exp(x[i]) / expr for i in 1:2))
+(exp(x[1]) / (0.0 + exp(x[2]) + exp(x[1]))) + (exp(x[2]) / (0.0 + exp(x[2]) + exp(x[1])))
+```
+In this model, JuMP will compute the value (and derivatives) of the denominator
+twice, without realizing that the same expression appears twice.
+
+As a work-around, create a new [`@variable`](@ref) and use an `==`
+[`@constraint`](@ref) to constrain the value of the variable to the
+subexpression.
+
+```jldoctest
+julia> model = Model();
+
+julia> @variable(model, x[1:2]);
+
+julia> @variable(model, expr);
+
+julia> @constraint(model, expr == sum(exp.(x)))
+expr - (0.0 + exp(x[2]) + exp(x[1])) = 0
+
+julia> @objective(model, Min, sum(exp(x[i]) / expr for i in 1:2))
+(exp(x[1]) / expr) + (exp(x[2]) / expr)
+```
+
+The reason JuMP does not perform common subexpression elimination automatically
+is for simplicity, and because there is a trade-off: for simple expressions, the
+extra complexity of detecting and merging common subexpressions may outweight
+the cost of computing them independently.
+
 ## Automatic differentiation
 
 JuMP computes first- and second-order derivatives using sparse reverse-mode

From 7668e5ffc4d78c2b94e57f9eebcd391a9969ab05 Mon Sep 17 00:00:00 2001
From: Oscar Dowson <odow@users.noreply.github.com>
Date: Wed, 13 Nov 2024 14:06:28 +1300
Subject: [PATCH 2/3] Apply suggestions from code review

---
 docs/src/manual/nonlinear.md | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/src/manual/nonlinear.md b/docs/src/manual/nonlinear.md
index 72a389e6908..7881d310ab3 100644
--- a/docs/src/manual/nonlinear.md
+++ b/docs/src/manual/nonlinear.md
@@ -174,7 +174,7 @@ julia> @objective(model, Min, sum(exp(x[i]) / expr for i in 1:2))
 (exp(x[1]) / (0.0 + exp(x[2]) + exp(x[1]))) + (exp(x[2]) / (0.0 + exp(x[2]) + exp(x[1])))
 ```
 In this model, JuMP will compute the value (and derivatives) of the denominator
-twice, without realizing that the same expression appears twice.
+twice, without realizing that it is the same expression.
 
 As a work-around, create a new [`@variable`](@ref) and use an `==`
 [`@constraint`](@ref) to constrain the value of the variable to the
@@ -196,8 +196,9 @@ julia> @objective(model, Min, sum(exp(x[i]) / expr for i in 1:2))
 
 The reason JuMP does not perform common subexpression elimination automatically
 is for simplicity, and because there is a trade-off: for simple expressions, the
-extra complexity of detecting and merging common subexpressions may outweight
-the cost of computing them independently.
+extra complexity of detecting and merging common subexpressions may outweigh
+the cost of computing them independently. Instead, we leave it to the user to
+decide which expressions to extract as common subexpressions.
 
 ## Automatic differentiation
 

From 67ce0983aa9d25ecccaf274c083bf75bc3808525 Mon Sep 17 00:00:00 2001
From: Oscar Dowson <odow@users.noreply.github.com>
Date: Wed, 13 Nov 2024 16:59:43 +1300
Subject: [PATCH 3/3] Update docs/src/manual/nonlinear.md

---
 docs/src/manual/nonlinear.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/docs/src/manual/nonlinear.md b/docs/src/manual/nonlinear.md
index 7881d310ab3..f71b5f03c07 100644
--- a/docs/src/manual/nonlinear.md
+++ b/docs/src/manual/nonlinear.md
@@ -155,7 +155,8 @@ julia> sin(sin(1.0))
 
 ## Common subexpressions
 
-JuMP does not perform common subexpression elimination. Instead, if you re-use
+JuMP does not perform [common subexpression elimination](https://en.wikipedia.org/wiki/Common_subexpression_elimination).
+Instead, if you re-use
 an expression in multiple places, JuMP will insert a copy of the expression.
 
 JuMP's lack of common subexpression elimination is a common cause of performance