Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

idmtools analyzer map seems missing read first line if first row if it is not label row #1582

Open
shchen-idmod opened this issue Jul 6, 2021 · 1 comment
Assignees
Labels
bug Something isn't working Core

Comments

@shchen-idmod
Copy link
Collaborator

shchen-idmod commented Jul 6, 2021

while I am working with Jillian's project: https://github.com/shchen-idmod/polio-calibration-emodpy/blob/dummy-site-calibra/Start_Here/Analysis_local/run_analysis_locally.py
in map function:
def map(self, data, simulation):
# retrieve sim data
df_all = self.retrieve_total_cases(data[self.filenames[0]], simulation, self.start_time, self.end_time)
return df_all

data[self.filenames[0]] contains 773 rows. but real csv contains 774 rows. the first row is missing in data[].
I think this is caused by first row in original csv file is not label row. idmtools may auto assume first row is label row and not read it. please look at this entry point:

data = {filename: FileParser.parse(filename, content)

experimentid in prod:
ca2b047f-c3db-eb11-a9ec-b88303911bc1

@issuelabeler issuelabeler bot added the Core label Jul 6, 2021
@shchen-idmod shchen-idmod added the bug Something isn't working label Jul 6, 2021
@shchen-idmod shchen-idmod changed the title idmtools analyzer map seems missing read first line if first line is not label row idmtools analyzer map seems missing read first line if first row if it is not label row Jul 6, 2021
@shchen-idmod
Copy link
Collaborator Author

we can use this entry script to test

import os
import sys

from Start_Here.Analysis_local.SpeedAnalyzer import SpeedAnalyzer
from Start_Here.Analysis_local.TotalCasesAnalyzer import TotalCasesAnalyzer
from Start_Here.Analysis_local.dummy_site import DummySite

sys.path.append(os.path.dirname(__file__))

from idmtools.analysis.analyze_manager import AnalyzeManager
from idmtools.core import ItemType
from idmtools.core.platform_factory import Platform

if __name__ == "__main__":
    platform = Platform("CALCULON")
    exp_id = 'ca2b047f-c3db-eb11-a9ec-b88303911bc1'
    site = DummySite()
    am = AnalyzeManager(platform, ids=[(exp_id, ItemType.EXPERIMENT)],
                        analyzers=[
                            TotalCasesAnalyzer(site=site, compute_likelihood=True, start_time=2007, end_time=2015,
                                               weight=1)])
    am.analyze()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Core
Projects
None yet
Development

No branches or pull requests

2 participants