another pass at cleanup and some notes

README.rst

@@ -25,6 +25,21 @@
 Neon Recording API
 *******************************
 
-- TODO: fill in README, still testing code
-- needs error checking & some better formatting
-- see `initial discussion/design <https://www.notion.so/pupillabs/Neon-Recording-Python-Lib-5b247c33e1c74f638af2964fa78018ff?pvs=4>`_
+- As far as I can tell, between 'main' and 'multipart' branches, every requested feature in the notion doc, as well as all that was discussed at all meetings, is implemented and tested. Various implementations of some functionality were tried, to see what feels most ergonomic for a user, while balancing minimal code against efficiency, as well as explicitness.
+
+- See 'multipart' branch not only for code to load multipart recordings, but also for code to load different (appropriate) timestamps based on whether recording came from cloud or phone
+
+- Some examples and discussion in the Notion doc involved fixation data. It was not clear what we wanted to do with fixations, since I understood that this library will only be for Native Recording Data. Shall a fixation detector be run when loading the data?
+
+- I tried, but pl-recover-recording will complicate installation of this library. it requires building untrunc from source and instructions for windows and macOS are needed
+
+- Some functions from the original design are not possible without monkey patching routines deep within Python. For example, with the current generator approach to sampling, the following is not possible:
+gaze = gaze.sample(between_two_events).to_numpy()
+unless we monkey patch the code for generator objects. Earlier approaches to the implementation of Stream that supported this were rejected after evaluation.
+So there is a Stream.sampled_to_numpy() method to handle this request.
+
+- I tried one or two ways and searched/thought about it, but loading all frames at once into RAM does not seem feasible.
+
+- TODO: fill in rest of README
+
+- see `initial discussion/design <https://www.notion.so/pupillabs/Neon-Recording-Python-Lib-5b247c33e1c74f638af2964fa78018ff?pvs=4>`_

src/pupil_labs/neon_recording/calib.py

@@ -28,8 +28,6 @@ def parse_calib_bin(rec_dir: pathlib.Path):
 
     log.debug("NeonRecording: Parsing calibration data")
 
-    # obtained from @dom:
-    # https://github.com/pupil-labs/realtime-python-api/blob/main/src/pupil_labs/realtime_api/device.py#L178
     return np.frombuffer(
         calib_raw_data,
         np.dtype(

src/pupil_labs/neon_recording/stream/av_stream/video_stream.py

@@ -52,23 +52,6 @@ def _sample_linear_interp(self, sorted_ts):
             "NeonRecording: Video streams only support nearest neighbor interpolation."
         )
 
-    def _sample_nearest_rob(self, sorted_tses):
-        log.debug("NeonRecording: Sampling nearest timestamps.")
-
-        closest_idxs = [
-            np.argmin(np.abs(self._ts - curr_ts)) if not self._ts_oob(curr_ts) else None
-            for curr_ts in sorted_tses
-        ]
-
-        for idx in closest_idxs:
-            if idx is not None and not np.isnan(idx):
-                d = self._data[int(idx)]
-                setattr(d, "ts", self._ts[int(idx)])
-                setattr(d, "ts_rel", self._ts_rel[int(idx)])
-                yield d
-            else:
-                yield None
-
     # from stack overflow:
     # https://stackoverflow.com/questions/2566412/find-nearest-value-in-numpy-array
     def _sample_nearest(self, sorted_tses):

src/pupil_labs/neon_recording/stream/stream.py

@@ -1,7 +1,6 @@
 import abc
 import math
 from enum import Enum
-from typing import Optional
 
 import numpy as np
 
@@ -107,24 +106,9 @@ def sample(self, tstamps, method=InterpolationMethod.NEAREST):
                 "Only LINEAR and NEAREST methods are supported."
             )
 
-    def _sample_nearest_rob(self, sorted_tses):
-        log.debug("NeonRecording: Sampling timestamps with nearest neighbor method.")
-
-        closest_idxs = [
-            np.argmin(np.abs(self._ts - curr_ts)) if not self._ts_oob(curr_ts) else None
-            for curr_ts in sorted_tses
-        ]
-
-        for idx in closest_idxs:
-            if idx is not None and not np.isnan(idx):
-                yield self._data[int(idx)]
-            else:
-                yield None
-
     # from stack overflow:
     # https://stackoverflow.com/questions/2566412/find-nearest-value-in-numpy-array
     def _sample_nearest(self, sorted_tses):
-        # uggcf://jjj.lbhghor.pbz/jngpu?i=FRKKRF5i59b
         log.debug("NeonRecording: Sampling timestamps with nearest neighbor method.")
 
         closest_idxs = np.searchsorted(self._ts, sorted_tses, side="right")

todo.txt

@@ -6,12 +6,9 @@
 	- eyestate/pupillometry stream - coming soon to an android near you
 
 - clarify how to deal with TsNs in imu stream
-- file to notify that it has been checked
 - for multi-part video, concatenate raw then convert to np array
 
 specify default option for agg, and provide stream specific aggs
 
-see here:
-https://www.notion.so/pupillabs/Neon-Recording-Python-Lib-5b247c33e1c74f638af2964fa78018ff?pvs=4#6bf3848ff3ad435aa6ea8fe37e72ee9c
-
 - check if data is from cloud or direct from phone
+	- did it in 'multipart' branch