Automate monthly reports

[huddlej]

Context

We produce monthly reports describing seasonal influenza circulation patterns. A substantial portion of time spent on these reports is tedious copying and pasting or running commands manually that could be automated.

Description

The following specific tasks will minimize the manual work associated with reports and allow us to focus on the science.

• Schedule nextflu-private builds to run and deploy after data upload (like the public builds). This should be easy to do now. We need to add rules that produce dated Auspice JSONs and deploy these JSONs to the desired group.
  • Add GitHub Action workflow to run nextflu-private builds
  • Add rules to the nextflu-private workflow to rename Auspice JSONs based on the build date and deploy the JSONs to the Nextstrain group #139
  • Trigger nextflu-private builds from upload action #157
• Plot counts per lineage by region with matplotlib instead of altair #162
• Run pathogen-embed on HA/NA alignments to flag putative reassortant clades #174
• Use existing public 2y builds as “representative” trees in our reports #164
• Automatically generate Markdown tables of new sequences per clade and clade coverage by titer references. The workflow logic to do this was added in Make Markdown output for reports #129, but we still need to add explicit dependencies for tabulate to our Docker and Conda environments.
  • Summarize all new sequences per clade using Nextclade annotations from the full dataset instead of using subsampled data associated with representative or titers builds. Depends on the addition of an earlier step in the workflow that runs Nextclade on all data as mentioned in Filter low-quality sequences by Nextclade QC status #144.
  • Summarize haplotype coverage by titer references using frequencies per haplotype from all available data #173
• Use a narrative template file to stub out a date-specific narrative for a given set of dated builds #163
• Automatically add links for the latest builds to the nextflu-private group landing page after builds have been deployed.

[genehack]

@huddlej maybe when we meet next week we can discuss what needs to be done to get this particular issue wrapped up?

[huddlej]

@genehack Great idea! @j23414 and I had talked a while back about working on #163 together. This is the last big chunk of work to finish before we close this issue.