MS10/S1: Early Science Verification of full cross-match

[Onoddil]

Further verification of the Phase B WP3.11 cross-match software within Phase C (and hence a Phase C milestone), ensuring that results from the initial all-sky testing runs are scientifically sound and suitable for wider astrophysical-problem use.

[Onoddil]

See #57 for history of the verification aspect prior to 17/Jul/23 when original milestone was split into testing and verification separately.

The split results from change request C073 (https://lsst-uk.atlassian.net/wiki/download/attachments/751108134/C073_Change_B_WP3.11_MS16.pdf?api=v2).

[tms-epcc]

04/AUG/23

• updated ticket to be an epic in order to add due date of 01/APR/25 as listed in C073
• milestone added to Phase C milestones spreadsheet

[tms-epcc]

29/AUG/23
@Onoddil

• investigation of missing sources (as found by @MikeeRead) complete
• missing sources have been found
• the code to fix this will be added later but is not needed for science verification

[Onoddil]

Two bugfixes have been identified:
First, an issue with the likelihood ratios returning a 'capped' value for every source was found and fixed, in which sources were always returning photometric probabilities (c/f) for the brightest possible magnitude bin, and incorrectly capping very low non-match likelihoods (since all bright sources should be matched between the two catalogues) at unphysical values; the correct magnitude bins are now found and the threshold for "unphysical" likelihood ratios have both been corrected.
Second, for a small number of chunks, mostly in the Galactic bulge, the astrometric uncertainty correction code fails. In testing this seems to be down to two roots of failure: first, in very crowded regions our reference catalogue (an optical dataset) does not overlap with the to-be-corrected dataset due to large extinction, and our matches only return in the regime of bright, high signal-to-noise sources, failing to provide us data in the non-systematic-limited portion of the dynamic range of the input dataset; and second, in crowded regions the parameter being fit for is unconstrained (this is also true more generally at low SNR in all regions, but less of an issue), and our simple least-squares minimisation does not return uncertainties on the parameters that necessarily reflect this, and without a constraining hyper-parameter fit we do not use the "right" value for these data.
This second issue -- more general than perhaps "just" a bug to be fixed -- is currently not implemented in the active development branch as it currently doesn't affect SV (as above), and various competing timelines mean it may not take priority over other deadlines.

[tms-epcc]

11/JAN/24
The GAIA-WISE match for science verification

• Tom ran the entire GAIA-WISE crossmatch on CSD3.
• Mike has finished loading this onto the UK RSP
• Next step is for the Exeter MPhys student (George Foley) to use to do the verification.

[tms-epcc]

07/MAR/24

• Agreed with MSc student the tests to do

[tms-epcc]

23/MAY/24

• MSc did not have time to do this so Tim will try instead
• will initially validation use extreme objects
• Will do this on the RSP.

[tms-epcc]

01/AUG/24

• Tim has been using RSP to find missing objects/ extreme objects
• also need Andy to check once he is available

[tms-epcc]

17/OCT/24

• AndyW now available and will do this
• expect the milestone can be confirmed or not within the next month
• but note there is still a dependence on Tim