Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Streamlit app, Emap portal, and other fixes #72

Draft
wants to merge 34 commits into
base: jeremy/hf-data-vali-misc
Choose a base branch
from
Draft

Streamlit app, Emap portal, and other fixes #72

wants to merge 34 commits into from

Conversation

jeremyestein
Copy link
Collaborator

  • Make Streamlit app for browsing waveform data
  • Create "Emap Portal", an nginx server with a home page linking to various reverse proxied Emap admin services
  • Fix generator and reader bugs

@jeremyestein
Validation script to check for gaps
557e7f1
@jeremyestein
Allow disabling of periodic deletion (infinite retention period) and
cf4963a 
get rid of confusing test-only flag for retention.
@jeremyestein
Improve logging esp. in case of overlap, and hopefully at least isola…
576e812 
…te overlapping

data into its own row in the DB rather than collating it.
@jeremyestein
More logging tweaks
2837812
@jeremyestein
First attempt at data visualisation/validation. Fix generator bug that
7423371 
reset the sine wave at the beginning of each batch of messages.
@jeremyestein
convert validation scripts to jupytext
2250acb
@jeremyestein
Rename component to reflect more accurately its function
4497a36
@jeremyestein
Make waveforms a somewhat realistic frequency (but not shape). Fix
7b1fc8f 
generator bug that resulting in no data for alternating 5 second chunks
@jeremyestein
Use jupyter lab plus first attempt at streamlit app
2514238
@jeremyestein
Optimise DB calls for caching, cache a DB method I'd missed, and remo…
2718c3f 
…ve unnecessary columns from query
@jeremyestein
Do array unpacking and the related timestamp arithmetic in the database
ef826b3 
instead of in pandas because it's much quicker.
@jeremyestein
Put streamlit (and experimentally, glowroot) behind an nginx reverse
dd63efd 
proxy
@jeremyestein
Add HTTP basic auth
dcad7e5
@jeremyestein
Add various graph metadata; use streamlit caching with TTL to allow for
57b809a 
live updating.
@jeremyestein
Need network access during build
edbe0dc
@jeremyestein
A few settings tweaks to see if they solve the websocket proxying
89a80fc 
problem
@jeremyestein
Configure streamlit to run on different base URL rather than rewrite
656c206 
URLs in nginx; should be more robust
@jeremyestein
Streamlit config tweaks
d43076c
@jeremyestein
Get external hostname and external port into the streamlit app, as
32a31e8 
websockets don't work through nginx without this.
@jeremyestein
Bring variables (mostly waveform related) into the global config script.
2c6becb 
Rearrange some variables. Document my understanding of the config script.
@jeremyestein
Make various variables configurable
b119d40
@jeremyestein
Convert null values in global yaml to empty string in envs, not "None"
9b09d3c
@jeremyestein
Fix annoying off by one bug that could cause collation to not happen in
740e561 
a timely fashion even if there were plenty of samples.
@jeremyestein
Clear up db connections and tweak cache timeout
824d549
@jeremyestein
Use points instead of line and give tidier missing data error
70fd9a7
@jeremyestein
Slider memory, improve errors, call methods on containers rather than
30235e6 
using `with`
@jeremyestein
Lower cache TTLs to allow for live data to be displayed sooner
2630b32
@jeremyestein
Go back to line chart and make slider a bit more controllable hopefully
447e586
@jeremyestein
Put cache TTL back up but add button to manually invalidate it, move
ec79b58 
controls around
@jeremyestein
Check none
5454d13
@jeremyestein
Try to reproduce bug by not enabling all streams for all patients. Don't
a0b2f31 
continue drawing graph in case of error. More logging.
@jeremyestein
Only show streams in dropdown for which data exists. Fix upper bound
5b16aa9 
rounding up bug which could sometimes round down. Ensure we never
ask for more data than we know exists, to avoid putting incomplete data in
the cache.
@jeremyestein
Make into multi-page app
107c9bc
@jeremyestein
Point to right page
fe043a0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jeremyestein