Lasair Cycle-11 Oct'24 to Mar'25

[tms-epcc]

See also notes from 25/SEP/24 Lasair fortnightly meeting.

Cycle 11 begins at the start of October 2024 (PM64) and finishes at the end of March 25 (PM69).

In order of priority, the activities, deliverables and milestones agreed as due in this cycle are as follows.

• 1. Cutout image ingestion (@genghisken / @gpfrancis / @thespacedoctor )

  • a. Reliability and optimisation
  • b. Reaper for cutout images
  • c. How to fetch cutout images from Rubin (subject to availability)

• 2. Sherlock: Ingest Delve, Gaia, catalogues ( @thespacedoctor )

  • Deep drilling fields from cosmology surveys i.e. ESO DECAM etc (see Josh W)

• 3. ZTF data release in the DAC ( @RoyWilliams )

  • a. Start with 18 fields RA 13h-14h and Dec -10 to 10.

• 4. Features

  • a. Revision of Features list ( @RoyWilliams )
  • b. Compare features for utility
  • c. Integrate Fastfinder ( @genghisken / @RoyWilliams /Michael Fulton )

• 5. Profile and improve database resilience and scalability ( @genghisken / @thespacedoctor / @gpfrancis )

• 6. Performance Evaluation ( @gpfrancis / @genghisken / @RoyWilliams / @thespacedoctor )

  • a. Integrate simulated alert distribution from USDF
  • b. see also Scaling & profile experiments ticket below

• 7. Add Fermi GBM alerts to GW infra – ingesting Healpix ( @RoyWilliams )

• 8. Oxford Lasair meeting, Oxford, January 2025

  • a. Preparations for ( @RoyWilliams )

• 9. Maintenance and continued operation of the ZTF pipeline, databases, and website

• 10. Lasair4 maintenance issue - Migrating everything that is still on Ubuntu 20.04 to24.04 before April 2025. This needs specifically to be addressed in the next 6 months. ( @gpfrancis )
  Note: if any items are left over from previous cycle then these are shown as tickets below

[tms-epcc]

25/SEP/24

• initial draft of ticket and plan (see description above) following Lasair fortnightly meeting
• @RoyWilliams - is this ok or are changes needed? When you are ok, I will add rest of the Lasair team to the assignees list.

[RoyWilliams]

Looks good Terry. Thank you as always for converting a sow's ear to a silk purse!

[RoyWilliams]

Terry, can we add another:
9. Maintenance and continued operation of the ZTF pipeline, databases, and website

[gpfrancis]

The lasair4 maintenance issue that we need to address specifically in the next 6 months is migrating everything that is still on Ubuntu 20.04 before April 2025.

[tms-epcc]

20/NOV/24

1. Cutout image ingestion - Cutouts will not be an issue after all, rather it is the catalogs
2. Sherlock - no news
3. ZTF in the DAC - on the test RSP but not yet on the production RSP. If tell astro community then it would mean them using the test RSP which is not what it is for. Roy will discuss more with Gareth.
4. Features - some progress with features; no movement yet on Fastfinder but Stephen ok with BBB if necessary; extinction map enables query on the extinction number(see dustmaps package lasair-lsst#210);
5. Profile and improve database resilience and scalability - some fixes made, timeout changes help with resilience but overall the approach feels arbitrary. Ken still going through the Data Stack tutorials to help with strategy.
6. Performance Evaluation - no news on simulated alerts since 30/SEP/24; will speak more with Eric in January.
7. Add Fermi GBM alerts to GW infra – Dave dealt with Healpix on Atlas so will discuss with him
8. Oxford meeting - need to send list to Lasair users
9. ZTF ops and maintenance - some PRs applied to Lasair ZTF (external brokers - Alerce/Fink updated for 2024 lasair4#230, URLs Special display when a column name is "url" lasair4#229). Need to apply these to Lasair-LSST too
10. Lasair4 and ubuntu - this is with Gareth

[tms-epcc]

04/DEC/24

1. Cutout image ingestion - Some issues
2. Sherlock - Deep drilling fields in progress
3. ZTF in the DAC- Now need astronomy community to use and provide feedback. Need to agree whom to contact first . Ask LUG.
4. Features - BBB being turned into a package - will get definitive confirmation from Stephen. No news on FF.
5. Profile and improve database resilience and scalability - need some news from Ken
6. Performance Evaluation - see above
7. Add Fermi GBM alerts to GW infra - getting more GBM alerts than originally expected. Had expected 1 per week but getting handful per day. Have meetings lined up with relevant experts to decide on way forward.
8. Oxford mtg - 32 registrations so far, most in person
9. ZTF ops and maintenance - to be discussed
10. Lasair4 & Ubuntu upgrade - to be discussed

[tms-epcc]

18/DEC/24
4. BBB is now included as package but Roy concerned users will be swamped by the number of available features