-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Validate 3D metrics #261
base: master
Are you sure you want to change the base?
Validate 3D metrics #261
Conversation
@@ -45,7 +45,7 @@ click = ">=7.1.2,<9.0" # NOTE: COLAB has 7.1.2 and has problems updating | |||
rich = "^10.15.2" | |||
shellingham = "^1.4.0" | |||
scikit-learn = ">=0.24.0" | |||
Shapely = ">=1.7.1" | |||
Shapely = "^1.8.1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gatli didn't we run into problems with Shapely at some point?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, we can't use it. It doesn't have wide enough binary support.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So what should we do about this? @Anirudh-Scale how much do we rely in the shapely dependency? Any chance we can get around this? If not, @gatli how can we still make this work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pfmark I can implement a function to calculate the area intersection without Shapely, it's just going to be significantly slower. If that's a trade-off we're ok with, I can do that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's already a function for polygon area intersection in the directory under geometry.py
. I'd suggest using that since it looks like you're using shapely only for polygon intersection area.
Edit: it looks like the native polygon was used originally before changing to shapely? The native polygon intersection function is quite slow since it's written in Python, but there's a speed up for convex polygons
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
few smaller comments. In addition, do we have a way of easily testing the correctness of the metrics in a test unit test? I think that would be great!
|
||
def __init__( | ||
self, | ||
enforce_label_match: bool = False, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This argument (and maybe confidence_threshold?) should be required in the constructor, and if we want to set defaults then set them in the child classes.
Added 3D IOU, BEV 2D IOU, and recall, precision metrics. Tested end-to-end with celery in
models
repo (see associated PR there).