Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

missing .videoPositionTracking file gives error #543

Open
jihyunbak opened this issue Nov 12, 2020 · 0 comments · May be fixed by #547
Open

missing .videoPositionTracking file gives error #543

jihyunbak opened this issue Nov 12, 2020 · 0 comments · May be fixed by #547

Comments

@jihyunbak
Copy link

Prerequisite

These are (a subset of) files in the raw/[date] directory:

20170901_kf19_01_s1.1.h264
20170901_kf19_01_s1.1.videoTimeStamps
20170901_kf19_01_s1.1.videoTimeStamps.cameraHWFrameCount
20170901_kf19_01_s1.rec
20170901_kf19_02_r1.1.h264
20170901_kf19_02_r1.1.videoPositionTracking
20170901_kf19_02_r1.1.videoTimeStamps
20170901_kf19_02_r1.1.videoTimeStamps.cameraHWFrameCount
20170901_kf19_02_r1.rec
20170901_kf19_02_r1.stateScriptLog

Note that videoPositionTracking file is missing In the first epoch 01, where the animal is sleeping.

Expected behavior

Missing files should be skipped "gracefully".

Current behavior

PosDataManager tries to look for the missing file, but because the list of pos files for this epoch is empty, it leads to an IndexError. See below.

Error message

Traceback:

~/proj/rec_to_nwb/rec_to_nwb/processing/builder/nwb_file_builder.py in build(self)
    284             self.associated_files_originator.make(nwb_content)
    285 
--> 286         self.position_originator.make(nwb_content)
    287 
    288         valid_map_dict = self.__build_corrupted_data_manager()

~/proj/rec_to_nwb/rec_to_nwb/processing/tools/beartype/beartype.py in func_beartyped(__beartype_func, *args, **kwargs)

~/proj/rec_to_nwb/rec_to_nwb/processing/builder/originators/position_originator.py in make(self, nwb_content)
     25     def make(self, nwb_content: NWBFile):
     26         logger.info('Position: Building')
---> 27         fl_positions = self.fl_position_manager.get_fl_positions()
     28         logger.info('Position: Creating')
     29         position = self.position_creator.create_all(fl_positions)

~/proj/rec_to_nwb/rec_to_nwb/processing/tools/beartype/beartype.py in func_beartyped(__beartype_func, *args, **kwargs)

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/components/position/fl_position_manager.py in get_fl_positions(self)
     26         meters_per_pixels = self.__get_meters_per_pixels(cameras_ids, self.metadata)
     27 
---> 28         position_datas = self.fl_position_extractor.get_positions()
     29         columns_labels = self.fl_position_extractor.get_columns_labels()
     30         if self.process_timestamps:

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/components/position/fl_position_extractor.py in get_positions(self)
     34         pos_datas = [
     35             PosDataManager(directories=[single_pos])
---> 36             for single_pos in self.all_pos
     37         ]
     38         return [

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/components/position/fl_position_extractor.py in <listcomp>(.0)
     34         pos_datas = [
     35             PosDataManager(directories=[single_pos])
---> 36             for single_pos in self.all_pos
     37         ]
     38         return [

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/components/position/pos_data_manager.py in __init__(self, directories)
      7 class PosDataManager(DataManager):
      8     def __init__(self, directories):
----> 9         DataManager.__init__(self, directories)
     10 
     11     # override

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/common/data_manager.py in __init__(self, directories)
     10         self.number_of_datasets = self.get_number_of_datasets()
     11         self.number_of_files_per_dataset = self.get_number_of_files_per_dataset()
---> 12         self.number_of_rows_per_file = self._get_data_shape(0)[0]
     13         self.file_lenghts_in_datasets = self._get_file_length(self.number_of_datasets)
     14 

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/common/data_manager.py in _get_data_shape(self, dataset_num)
     21 
     22     def _get_data_shape(self, dataset_num):
---> 23         dim1 = np.shape(self.read_data(dataset_num, 0))[0]
     24         dim2 = np.shape(self.read_data(dataset_num, 0))[1]
     25         return dim1, dim2

~/proj/rec_to_nwb/rec_to_nwb/processing/nwb/components/position/pos_data_manager.py in read_data(self, dataset_id, file_id)
     12     def read_data(self, dataset_id, file_id):
     13         """extract data from POS files and build FlPos"""
---> 14         pos_online = readTrodesExtractedDataFile(self.directories[dataset_id][file_id])
     15         position = pd.DataFrame(pos_online['data'])
     16         labels = self.get_column_labels()

IndexError: list index out of range
@jihyunbak jihyunbak linked a pull request Nov 12, 2020 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant