Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a streamlit interface to Wizard for anomaly detection #3369

Closed
8 of 10 tasks
Tracked by #3340
larsyencken opened this issue Oct 4, 2024 · 3 comments
Closed
8 of 10 tasks
Tracked by #3340

Add a streamlit interface to Wizard for anomaly detection #3369

larsyencken opened this issue Oct 4, 2024 · 3 comments
Assignees

Comments

@larsyencken
Copy link
Collaborator

larsyencken commented Oct 4, 2024

Rough proposal

  • In the Wizard, there is a page for anomaly detection
  • The page should present you with a list of all changed grapher indicators
    • It should read from local disk what indicators are upgrades of other indicators
    • You may optionally ask about any arbitrary ETL path for a grapher indicator
  • You click "Detect all anomalies"
  • It begins fetches data from the S3 API and finds things whilst you wait
    • It should be interactive speed, i.e. take less than one minute
    • If an indicator is an upgrade, it also detects within the context of the previous indicator
    • (optional) It would be dreamy if they appear one by one whilst you wait
  • The anomaly detector tries to return only the few "most important" anomalies
  • It stores them in some YAML file or SQLite DB; they're still there if you come back later
  • It provides a one-click link for any anomaly that takes you to the indicator (either in a chart, or in the admin)
  • There is a button to regenerate them all
  • There is a button to delete them all
  • They are ephemeral: when you merge your branch, and your staging server dies, they disappear
    • But you can still manually add something important to "What you should know about this indicator" for its data page
@lucasrodes
Copy link
Member

Streamlit app is stable and working. Needs some more testing, to ensure it works as expected.

We plan to dark-launch it and try it in various PRs.

@lucasrodes
Copy link
Member

Some things that we may add in the future:

  • Hiding anomalies
    • Each anomaly would have a button "hide anomaly" so that this anomaly is no longer shown
    • This should be done in sync with the filter-panel. A possible solution here is to add a new section to the filter pane which works as a st.popover. (idea below)
      Image
  • AI summary
    • Optimise how we handle the summaries to avoid unnecessary GPT calls.
  • Checksum detection
    • We should be able to infer if the anomalies in cache are out of sync (maybe using dataset checksums?)

@lucasrodes
Copy link
Member

@pabloarosado @Marigold I'll close this for now, since we released a first version of this already.

We can track improvements in #3436

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants