More tests & better checking of extraction results. #26
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Dec 27, 2023
oflatt
approved these changes
Dec 27, 2023
oflatt
reviewed
Dec 27, 2023
| @@ -78,11 +78,9 @@ fn main() { | |||
|
|
|||
| let start_time = std::time::Instant::now(); | |||
| let result = extractor.extract(&egraph, &egraph.root_eclasses); | |||
| result.check(&egraph); | |||
There was a problem hiding this comment.
should this check happen after we measure the time? (line 83)
|
Does this PR contain the cases where |
Yes, the global-greedy-dag extractor fails, in that it chooses nodes from the wrong class, on some of the egraphs in this PR. |
|
Oh, we should change the CI so that it actually runs all the extractors then. After we fix the bugs! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In creating the nicer ILP extractor (#16) I've had lots of annoying rare errors that have been hard to pin down. So I've made a fuzzer that randomly generates egraphs, then I run different extractors on those random egraphs.
Because the ILP extractors (ilp-cbc & faster-ilp-cbc) are optimal for DAG-extraction, the DAG-extractions from the other extractiors should never be cheaper on a given egraph. The effect of all this is that the extractors get tested on tens of millions of egraphs. It's found lots of problems with the extractors.
That's the back story.
Anyway, this PR contains some egraph that the fuzzer has generated, some that I've handcrafted to check extreme cases and better testing of extraction results.
These extra tests reveal problems with the faster-greedy-dag & global-greedy-dag extractors. Both of which fail on some instances.
The faster-greedy-dag returns extractions with cycles and the global-greedy-dag sometimes returns extractions with the wrong nodes selected (i.e. a node will be selected for the wrong class). I'll create cases for these.