Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark against PATH #21

Merged
merged 12 commits into from
Dec 25, 2024
Merged

benchmark against PATH #21

merged 12 commits into from
Dec 25, 2024

Conversation

dfridovi
Copy link
Member

@dfridovi dfridovi commented Dec 24, 2024

Adds support for benchmarking against the PATH solver.

  • Get solve times for random QPs
  • Compute summary statistics
  • Tune IP solver to improve metrics
  • Make QPs sparse (sparsity is only known at runtime though, not compile time)

Adding additional solver kwargs:

  • tightening_rate (default 1.0): if an inner loop succeeds, scales ε by 1 - exp(-tightening_rate * inner_iters)
  • loosening_rate (default 1.0): if an inner loop fails, scales ε by 1 + exp(-loosening_rate * inner_iters)
  • verbose (default false): only print warnings if true

Also, adding a small amount of regularization to the Newton step computation for numerical stability.

@dfridovi
Copy link
Member Author

For documentation purposes, at 5a1b9ad the stats are

julia> stats = SolverBenchmarks.summary_statistics(data)
(ip = (success_rate = 0.995, μ = 0.05398038283718592, σ = 0.00722662057721707), path = (success_rate = 0.995, μ = 0.007355797381909546, σ = 0.0016724800239494542))

So... we are about 10x slower than PATH.

@dfridovi
Copy link
Member Author

dfridovi commented Dec 25, 2024

And now only 5x slower

julia> data = SolverBenchmarks.benchmark(; data.ip_mcp, data.path_mcp, ip_kwargs = (; tightening_rate = 0.05, loosening_rate = 0.5, max_inner_iters = 5, tol = 1e-6))
julia> stats = SolverBenchmarks.summary_statistics(data)
(ip = (success_rate = 0.995, μ = 0.03280799303115578, σ = 0.005423971573040027), path = (success_rate = 0.995, μ = 0.007583776933668342, σ = 0.0022406690554635476))

@dfridovi dfridovi marked this pull request as ready for review December 25, 2024 00:23
@dfridovi dfridovi merged commit 4244684 into main Dec 25, 2024
3 checks passed
@dfridovi dfridovi deleted the benchmark/path branch December 25, 2024 00:23
@dfridovi dfridovi mentioned this pull request Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant