deal with the malformed CSV text from IEA-EV-dataEVsalesCarsProjection-APS.csv

[cjyetman]

This causes a breaking change in workflow.scenario.preparation that will be fixed by RMI-PACTA/workflow.scenario.preparation#49

devtools::load_all()
#> ℹ Loading pacta.scenario.data.preparation

scenario_preparation_inputs_path <- "~/data/pactarawdata/scenario-sources"
scenario_preparation_outputs_path <- "~/data/pactadatadev/workflow-scenario-preparation-outputs"

config <- list(
  weo_2022_raw_path = "00Imported/WEO2022 extended data/WEO2022 extended data",
  weo_2022_ext_data_regions_raw_filepath = "raw_data_from_provider/used_in_pacta.scenario_preparation/WEO2022_Extended_Data_Regions.csv",
  weo_2022_ext_data_world_raw_filepath = "raw_data_from_provider/used_in_pacta.scenario_preparation/WEO2022_Extended_Data_World.csv",
  weo_2022_fossil_fuels_raw_filepath = "light_process/weo2022_fossilfuel_demand_supply.csv",
  weo_2022_nze_auto_raw_filepath = "processed_data/nze_may_2021_report_old_data/hard_process_auto/used/NZE2021_RawData_2050.xlsx",
  weo_2022_nze_steel_raw_filepath = "raw_data_from_provider/used_in_pacta.scenario_preparation/WEO2022_NZE_SteelData.csv",
  weo_2022_sales_aps_auto_raw_filepath = "raw_data_from_provider/used_in_pacta.scenario_preparation/SalesAPS_rawdata.csv",
  weo_2022_electric_sales_aps_auto_raw_filename = "raw_data_from_provider/used_in_pacta.scenario_preparation/IEA-EV-dataEV salesCarsProjection-APS.csv"
)

source("../workflow.scenario.preparation/process_scenario_weo_2022.R")

pacta.data.validation::validate_intermediate_scenario_output(weo_2022)

waldo::compare(
  dplyr::arrange(pacta.scenario.preparation::weo_2022, source, scenario, scenario_geography, sector, technology),
  weo_2022,
  tolerance = 1e-15
)
#> ✔ No differences

tail(weo_2022_electric_sales_aps_auto_raw_text)
#> [1] "World,Projection-APS,EV sales,Cars,BEV,2021,sales,4700000" 
#> [2] "World,Projection-APS,EV sales,Cars,PHEV,2021,sales,1900000"
#> [3] "World,Projection-APS,EV sales,Cars,BEV,2025,sales,14000000"
#> [4] "World,Projection-APS,EV sales,Cars,PHEV,2025,sales,4700000"
#> [5] "World,Projection-APS,EV sales,Cars,BEV,2030,sales,34000000"
#> [6] "World,Projection-APS,EV sales,Cars,PHEV,2030,sales,7000000"
length(weo_2022_electric_sales_aps_auto_raw_text)
#> [1] 49

[AlexAxthelm]

I think that this is a fine solution, but I think that RMI-PACTA/workflow.scenario.preparation#48 is a simpler option that we could consider.

[cjyetman]

The goal is to keep file i/o in workflows, and data munging in R pkgs, so this more accurately follows that strategy.

[jdhoffa]

@AlexAxthelm @cjyetman am I the only person that feels like this is an absurdly huge hoop to jump through just to reformat a file that was misformatted due to our manual error to begin with...?! lol

like just update the file!?

[cjyetman]

@AlexAxthelm @cjyetman am I the only person that feels like this is an absurdly huge hoop to jump through just to reformat a file that was misformatted due to manual error to begin with...?! lol

like just update the file!?

no... that was also my preferred solution

[AlexAxthelm]

If you both feel strongly that editing the file is the better call here, then we can do that. I'll make the changes and upload to AFS.

[jdhoffa]

Thanks @AlexAxthelm

Closing this PR then.