Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark on trajectory games #33

Merged
merged 6 commits into from
Jan 6, 2025
Merged

benchmark on trajectory games #33

merged 6 commits into from
Jan 6, 2025

Conversation

dfridovi
Copy link
Member

@dfridovi dfridovi commented Jan 3, 2025

  • factor out benchmarking code to work for different types of problems (i.e., adding BenchmarkType abstraction)
  • introduce TrajectoryGameBenchmark modeled on LaneChange in examples/

@dfridovi dfridovi marked this pull request as draft January 3, 2025 17:10
@dfridovi
Copy link
Member Author

dfridovi commented Jan 3, 2025

Based on these results, I think we need to roll back the Krylov stuff, or at least make it not the default option.

julia> data = SolverBenchmarks.benchmark(SolverBenchmarks.TrajectoryGameBenchmark(); num_samples=10); SolverBenchmarks.summary_statistics(data)
[ Info: Generating random problems...
[ Info: Generating IP MCP...
[ Info: Generating PATH MCP...
[ Info: Warming up IP solver...
[ Info: Warming up PATH solver...
Solving IP MCPs... 100%|█████████████████████████████████| Time: 0:00:53
Solving PATH MCPs... 100%|███████████████████████████████| Time: 0:00:02
[ Info: IP runtime is 2918.485214038064 % that of PATH.
(ip = (success_rate = 0.9, μ = 5.099074351777778, σ = 0.3451189449072598), path = (success_rate = 0.7, μ = 0.17471647028571427, σ = 0.08759241497713036))

@dfridovi dfridovi requested a review from lassepe January 3, 2025 23:05
@dfridovi dfridovi marked this pull request as ready for review January 3, 2025 23:05
@dfridovi dfridovi mentioned this pull request Jan 4, 2025
@dfridovi dfridovi merged commit 9157512 into main Jan 6, 2025
3 checks passed
@dfridovi dfridovi deleted the benchmark/trajectory-game branch January 6, 2025 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant