Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 18 #54

Merged
merged 25 commits into from
Mar 29, 2021
Merged

Issue 18 #54

merged 25 commits into from
Mar 29, 2021

Conversation

singhish
Copy link

@singhish singhish commented Mar 10, 2021

bin/dump_data_to_file.py Outdated Show resolved Hide resolved
bin/dump_data_to_file.py Outdated Show resolved Hide resolved
bin/dump_data_to_file.py Outdated Show resolved Hide resolved
bin/dump_data_to_file.py Outdated Show resolved Hide resolved
emeval/input/spec_details.py Outdated Show resolved Hide resolved
emeval/input/phone_view.py Outdated Show resolved Hide resolved
bin/dump_data_to_file.py Outdated Show resolved Hide resolved
shankari and others added 5 commits March 21, 2021 23:45
Creates a SpecDetails object using the constructor and ensures that it calls
the correct methods to populate itself

Running is simple

```
$ python -m unittest
```
Add simple unit test with a mocking example
Copy link
Collaborator

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good overall. I'd like to see a "Testing done" section indicating how you tested it. Would also be good to pull out the keys into constants, and to change the notebooks to use the constants.

emeval/input/spec_details.py Outdated Show resolved Hide resolved
pv = PhoneView(sd)
for phone_os, phone_map in pv.map().items():
for phone_label, phone_detail_map in phone_map.items():
for key in [k for k in phone_detail_map.keys() if "/" in k]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as part of your documentation, you should highlight that this is how you identify that a particular key needs to be saved to a file

sd.eval_start_ts,
sd.eval_end_ts,
out_dir)
for ranges in [phone_detail_map["calibration_ranges"], phone_detail_map["evaluation_ranges"]]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we need two levels of dump_data_to_file to handle the transitions and then the data within each range? Again, add a comment to highlight that.

@shankari
Copy link
Collaborator

Next step is to change the notebooks to use FileSpecDetails and the constants for the entries (where applicable).
Once we have working notebooks again, I am happy to merge.

Copy link
Collaborator

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good overall. Just some minor fixes. I am open to merging with only this notebook, but you do need to change the other notebooks as well before starting on the tree flattening -> pandas shift

Data_exploration_template.ipynb Outdated Show resolved Hide resolved
Comment on lines 152 to 157
" #for tr in r[\"evaluation_trip_ranges\"]:\n",
" # print(12 * ' ', 30 * \"-\")\n",
" # print(12 * ' ',tr[\"trip_id\"], tr.keys())\n",
" # for sr in tr[\"evaluation_section_ranges\"]:\n",
" # print(16 * ' ', 30 * \"~\")\n",
" # print(16 * ' ',sr[\"trip_id\"], sr.keys())"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restore

bin/dump_data_to_file.py Show resolved Hide resolved
emeval/input/spec_details.py Show resolved Hide resolved
emeval/input/phone_view.py Show resolved Hide resolved
@singhish
Copy link
Author

Notebooks are mostly working now except for the ones that rely on create_analysed_view in emeval.analysed.phone_view. Here, the function relies on making calls to the server's analysis-oriented time series keys. Wasn't sure how to handle this -- we can pick up with this next week.

@shankari
Copy link
Collaborator

Confirmed that the generated maps don't change

Before After
Screen Shot 2021-03-26 at 6 17 03 PM Screen Shot 2021-03-26 at 8 20 01 PM

Comment on lines 142 to 144
"About to retrieve messages using {'user': 'ucb-sdb-android-1', 'key_list': ['manual/evaluation_transition'], 'start_time': 1563606000, 'end_time': 1588204800}\n",
"response = <Response [200]>\n",
"Found 86 entries\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are we still getting Response [200] here? Shouldn't we be using the FileSpecDetails?

Comment on lines 87 to 94
"execution_count": null,
"execution_count": 7,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"About to retrieve messages using {'user': '[email protected]', 'key_list': ['config/evaluation_spec'], 'start_time': 0, 'end_time': 1613685967}\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't check in outputs, at least for files that didn't have outputs to begin with.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't change the XXX_master files that had outputs just yet. As you point out, we haven't figured out how to run the algorithms on in-memory data instead.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try and keep the PR as minimal as possible so it is easy to review and understand.

@shankari
Copy link
Collaborator

shankari commented Mar 29, 2021

@singhish Can you also revert the changes to the master ipynb and file a separate issue to track it? Then I can merge this change and we can discuss the complicated part in greater detail after you are done with the outline.

@shankari
Copy link
Collaborator

Please make sure to handle the minor fixes at MobilityNet/mobilitynet.github.io#23

@shankari shankari merged commit a81d57b into MobilityNet:master Mar 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants