Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HLA-1362: Modified the minimum RMS computation used as a discriminant for choos… #1908

Merged
merged 4 commits into from
Nov 21, 2024

Conversation

mdlpstsci
Copy link
Collaborator

@mdlpstsci mdlpstsci commented Nov 4, 2024

Resolves HLA-1362

Closes #

Modified the minimum RMS computation used as a discriminant for choosing the
background computation algorithm for source detection. The update is for the source code only as any detector-dependent configuration values have not yet been tuned.

Access to the updated source code is so statistical tests can be run for further evaluation of the change and data which can be used to tune the detector-dependent configuration values.

Checklist for maintainers

@mdlpstsci mdlpstsci self-assigned this Nov 4, 2024
@mdlpstsci mdlpstsci requested a review from a team as a code owner November 4, 2024 22:28
@mdlpstsci mdlpstsci added the Do Not Merge PR which should not be merged label Nov 4, 2024
Copy link
Collaborator

@rlwastro rlwastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Very clean implementation. You can certainly ignore my suggestion if you want!

if self.keyword_dict['detector'].upper() != 'SBC':
minimum_rms = np.sqrt(self.keyword_dict['numexp']) * self.keyword_dict['readnse'] / self.keyword_dict['texpo_time']
else:
minimum_rms = np.sqrt(np.clip(bkg_median * self.keyword_dict['texpo_time'], a_min=1.0, a_max=None)) / self.keyword_dict['texpo_time']
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
minimum_rms = np.sqrt(np.clip(bkg_median * self.keyword_dict['texpo_time'], a_min=1.0, a_max=None)) / self.keyword_dict['texpo_time']
minimum_rms = np.sqrt(
(bkg_median * self.keyword_dict['texpo_time']).clip(min=1.0)
) / self.keyword_dict['texpo_time']

I like using the ndarray.clip() method rather than the np.clip function. But if you have some reason to prefer the np.clip function version, your code will also work. In either case, you could omit the max value (or a_max, which seems the same as max), since it defaults to None.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no preference for either function/method, but since the function is already coded I will continue to use it :). I will note that one does have to specify both a_min and a_max (TypeError: clip() missing 1 required positional argument: 'a_max') as I discovered when I initially implemented the code!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, sounds OK to me. (You can omit the max value with ndarray.clip.)

I have one other suggestion. I think the version of the rms that is used in the "excessive zero background" case should also have the minimum applied. The code is a little simpler because the median value is known to be zero. Here's a patch:

index e9ceb8fe..dbadf50f 100755
--- a/drizzlepac/haputils/catalog_utils.py
+++ b/drizzlepac/haputils/catalog_utils.py
@@ -289,6 +289,12 @@ class CatalogImage:
 
                 is_zero_background_defined = True
                 log.info(f"Input image contains excessive zero values in the background. Median: {self.bkg_median:.6f} RMS: {self.bkg_rms_median:.6f}")
+                # Compute a minimum rms value based upon information directly from the data
+                minimum_rms = 1. / self.keyword_dict['texpo_time']
+                if np.nan_to_num(self.bkg_rms_median) < minimum_rms:
+                    self.bkg_rms_median = minimum_rms
+                    log.info(f"Minimum RMS of input based upon the total exposure time: {minimum_rms:.6f}")
+                    log.info(f"Median RMS has been updated - Median: {self.bkg_median:.6f} RMS: {self.bkg_rms_median:.6f}")
 
         # BACKGROUND COMPUTATION 2 (sigma_clipped_stats)
         # If the input data is not the unusual case of SBC "excessive zero background", compute

I included code that handles the issue you ran across where the computed rms is NaN.

…ing the

background computation algorithm.  The update is for the source code only as
any detector-dependent configuration values have not yet been tuned.
a lot of noise in the catalogs due to a low detection threshold.
@mdlpstsci mdlpstsci removed Do Not Merge PR which should not be merged in progress labels Nov 21, 2024
@mdlpstsci mdlpstsci merged commit 496db58 into spacetelescope:main Nov 21, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants