Skip to content

Commit

Permalink
BayDAG All Contributions (ActivitySim#834)
Browse files Browse the repository at this point in the history 
* get all disaggregate accessibility values

* updated settings to work with Pydantic

* Squashed commit of the following:

commit 1670dc9
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 15:53:37 2024 -0800

    updated settings to work with Pydantic

commit ea25a66
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 16:30:41 2023 -0800

    get all disaggregate accessibility values

* Squashed commit of the following:

commit d05f77c
Author: David Hensle <[email protected]>
Date:   Mon Mar 11 15:00:28 2024 -0700

    fixing person_rank call

commit f252006
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 16:35:37 2023 -0800

    CDAP estimation with joint component

* Squashed commit of the following:

commit 90b0b8c
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 14:05:43 2024 -0800

    Only want joint tours for joint tour dest model -- got dropped in rebase

* Squashed commit of the following:

commit 807b0af
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 16:40:26 2023 -0800

    Alt name for jtfc estimation mode

* Squashed commit of the following:

commit 5518ca4
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 16:41:59 2023 -0800

    error out of infinite loop

* Squashed commit of the following:

commit 0e6919b
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 13:23:13 2024 -0800

    blacken

commit 8071068
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 16:59:32 2023 -0800

    sampling in EDB for location choice

commit 150d593
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 16:59:32 2023 -0800

    sampling in EDB for location choice

* Squashed commit of the following:

commit 48be6f4
Author: David Hensle <[email protected]>
Date:   Thu Mar 7 13:56:38 2024 -0800

    replace orca inject with state object

commit 4201d36
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:44:06 2023 -0800

    landuse available in trip destination

commit 0630659
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:02:59 2023 -0800

    landuse and reindex in loc choice

* Squashed commit of the following:

commit d1f4db8
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 15:09:38 2024 -0800

    call ALT_DEST_COL_NAME as attribute

commit fe0c284
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 14:11:35 2024 -0800

    fix Nones to None typo

commit b955295
Author: David Hensle <[email protected]>
Date:   Mon Dec 18 11:56:54 2023 -0800

    blacken

commit cdcca91
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:12:02 2023 -0800

    mode choice logsum extraction

* Squashed commit of the following:

commit 504b974
Merge: ac69af8 a40fa16
Author: David Hensle <[email protected]>
Date:   Mon Mar 18 09:21:05 2024 -0700

    Merge branch 'BayDAG_cont10_nmtf_avail_time_windows' of https://github.com/SANDAG/activitysim into BayDAG_cont10_nmtf_avail_time_windows

commit ac69af8
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 14:15:31 2024 -0800

    remove bad path to annotate.py

commit a40fa16
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 14:15:31 2024 -0800

    remove bad path to annotate.py

commit 5a54dc5
Merge: 481238b 79980b8
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 13:38:45 2024 -0800

    Merge branch 'BayDAG_cont10_nmtf_avail_time_windows' of https://github.com/SANDAG/activitysim into BayDAG_cont10_nmtf_avail_time_windows

commit 481238b
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 13:37:37 2024 -0800

    blacken

commit 3aa3f46
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:16:51 2023 -0800

    NMTF person available periods

commit 79980b8
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:16:51 2023 -0800

    NMTF person available periods

* Squashed commit of the following:

commit c439405
Author: David Hensle <[email protected]>
Date:   Mon Mar 18 10:05:23 2024 -0700

    blacken

commit d9392f3
Author: David Hensle <[email protected]>
Date:   Mon Mar 18 09:33:09 2024 -0700

    adding missed columns necessary for no school escorting

commit 4470dd4
Author: David Hensle <[email protected]>
Date:   Mon Mar 11 14:10:42 2024 -0700

    updating to work with Pydantic and State object

commit cf0a663
Author: David Hensle <[email protected]>
Date:   Fri Mar 8 13:48:42 2024 -0800

    blacken

commit 5c1524f
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:42:15 2023 -0800

    School escorting estimation updates

    Most changes are needed to avoid crash if estimation run actually had no school escorting in the input data

* Squashed commit of the following:

commit f30a9ea
Author: David Hensle <[email protected]>
Date:   Tue Mar 12 10:28:11 2024 -0700

    settings in write_omnibus_table

commit e8144af
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:59:19 2023 -0800

    comments allowed in stop freq spec

commit ece02a8
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:53:12 2023 -0800

    estimation mode usability

* Squashed commit of the following:

commit 74212e4
Author: David Hensle <[email protected]>
Date:   Tue Mar 12 14:56:17 2024 -0700

    nmtf larch av arg

commit 6a3a332
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:54:47 2023 -0800

    decrease larch load time for nmtf

* Squashed commit of the following:

commit 2b248ce
Author: David Hensle <[email protected]>
Date:   Mon Dec 18 13:13:23 2023 -0800

    blacken

commit d8af5b8
Author: David Hensle <[email protected]>
Date:   Fri Dec 15 17:58:28 2023 -0800

    adding calls to larch for new simple simulate models

* KEEP_COLS setting

* working with sandag abm3 example

* adding required settings in cdap test

* Default value for KEEP_COLS

* parking locations in write trip matrices

* avoiding NaN in transit_pass_subsidy result

* fix bug in which joint utilities not in probabilities cdap

* putting origin and destination rewrite in correct location

* blacken

* removing arc test

* transit pass subsidy filter_col bug fix

* comment for persons_merged issue 853
  • Loading branch information
@dhensle
dhensle authored Apr 19, 2024
1 parent 751ee44 commit 8e1fbcd
Show file tree
Hide file tree
Showing 10 changed files with 43 additions and 25 deletions.
5 changes: 4 additions & 1 deletion activitysim/abm/models/auto_ownership.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ def auto_ownership_simulate(
state: workflow.State,
households: pd.DataFrame,
households_merged: pd.DataFrame,
# FIXME: persons_merged not used but included, see #853
persons_merged: pd.DataFrame,
model_settings: AutoOwnershipSettings | None = None,
model_settings_file_name: str = "auto_ownership.yaml",
trace_label: str = "auto_ownership_simulate",
Expand Down Expand Up @@ -76,6 +78,7 @@ def auto_ownership_simulate(
locals_d.update(constants)

expressions.assign_columns(
state,
df=choosers,
model_settings=preprocessor_settings,
locals_dict=locals_d,
Expand Down Expand Up @@ -119,7 +122,7 @@ def auto_ownership_simulate(
)

if model_settings.annotate_households:
annotate.annotate_households(model_settings, trace_label)
annotate.annotate_households(state, model_settings, trace_label)

if trace_hh_id:
state.tracing.trace_df(households, label="auto_ownership", warn_if_empty=True)
2 changes: 2 additions & 0 deletions activitysim/abm/models/cdap.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ class CdapSettings(PydanticReadable, extra="forbid"):
FIXED_RELATIVE_PROPORTIONS_SPEC: str = "cdap_fixed_relative_proportions.csv"
ADD_JOINT_TOUR_UTILITY: bool = False
JOINT_TOUR_COEFFICIENTS: str = "cdap_joint_tour_coefficients.csv"
JOINT_TOUR_USEFUL_COLUMNS: list[str] | None = None
"""Columns to include from the persons table that will be need to calculate household joint tour utility."""
annotate_persons: PreprocessorSettings | None = None
annotate_households: PreprocessorSettings | None = None
COEFFICIENTS: Path
Expand Down
4 changes: 4 additions & 0 deletions activitysim/abm/models/initialize.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,10 @@ def initialize_households(
suffixes = disaggregate_accessibility.disaggregate_suffixes(state)
shadow_pricing.add_size_tables(state, suffixes)

# create disaggregate_accessibility table if not model was run
if state.is_table("proto_disaggregate_accessibility"):
disaggregate_accessibility.disaggregate_accessibility(state)

# - preload person_windows
person_windows = state.get_dataframe("person_windows")
chunk_sizer.log_df(trace_label, "person_windows", person_windows)
Expand Down
3 changes: 3 additions & 0 deletions activitysim/abm/models/non_mandatory_tour_frequency.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,9 @@ def non_mandatory_tour_frequency(

choices_list.append(choices)

# FIXME only want to keep actual purposes, adding cols in alts will mess this up
# this is complicated by canonical_ids calculated based on alts if not specified explicitly
# thus, adding column to input alts will change IDs and break estimation mode....
del alternatives["tot_tours"] # del tot_tours column we added above

# The choice value 'non_mandatory_tour_frequency' assigned by interaction_simulate
Expand Down
13 changes: 11 additions & 2 deletions activitysim/abm/models/transit_pass_subsidy.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ class TransitPassSubsidySettings(LogitComponentSettings, extra="forbid"):
preprocessor: PreprocessorSettings | None = None
"""Setting for the preprocessor."""

CHOOSER_FILTER_COLUMN_NAME: str | None = None
"""Column name which selects choosers. If None, all persons are choosers."""


@workflow.step
def transit_pass_subsidy(
Expand All @@ -48,7 +51,6 @@ def transit_pass_subsidy(
)

choosers = persons_merged
logger.info("Running %s with %d persons", trace_label, len(choosers))

estimator = estimation.manager.begin_estimation(state, "transit_pass_subsidy")

Expand All @@ -69,6 +71,11 @@ def transit_pass_subsidy(
trace_label=trace_label,
)

filter_col = model_settings.CHOOSER_FILTER_COLUMN_NAME
if filter_col is not None:
choosers = choosers[choosers[filter_col]]
logger.info("Running %s with %d persons", trace_label, len(choosers))

model_spec = state.filesystem.read_model_spec(model_settings.SPEC)
coefficients_df = state.filesystem.read_model_coefficients(model_settings)
model_spec = simulate.eval_coefficients(
Expand Down Expand Up @@ -103,7 +110,9 @@ def transit_pass_subsidy(
estimator.write_override_choices(choices)
estimator.end_estimation()

persons["transit_pass_subsidy"] = choices.reindex(persons.index)
persons["transit_pass_subsidy"] = (
choices.reindex(persons.index).fillna(0).astype(int)
)

state.add_table("persons", persons)

Expand Down
2 changes: 2 additions & 0 deletions activitysim/abm/models/util/annotate.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ def annotate_households(
locals_dict = {}
households = state.get_dataframe("households")
expressions.assign_columns(
state,
df=households,
model_settings=model_settings.get("annotate_households"),
locals_dict=locals_dict,
Expand Down Expand Up @@ -64,6 +65,7 @@ def annotate_persons(
locals_dict = {}
persons = state.get_dataframe("persons")
expressions.assign_columns(
state,
df=persons,
model_settings=model_settings.get("annotate_persons"),
locals_dict=locals_dict,
Expand Down
29 changes: 10 additions & 19 deletions activitysim/abm/models/util/cdap.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -221,8 +221,10 @@ def individual_utilities(
indiv_utils[useful_columns] = persons[useful_columns]

# add attributes for joint tour utility
model_settings = state.filesystem.read_model_settings("cdap.yaml")
additional_useful_columns = model_settings.get("JOINT_TOUR_USEFUL_COLUMNS", None)
from activitysim.abm.models.cdap import CdapSettings

model_settings = CdapSettings.read_settings_file(state.filesystem, "cdap.yaml")
additional_useful_columns = model_settings.JOINT_TOUR_USEFUL_COLUMNS
if additional_useful_columns is not None:
indiv_utils[additional_useful_columns] = persons[additional_useful_columns]

Expand Down Expand Up @@ -850,8 +852,10 @@ def hh_choosers(state: workflow.State, indiv_utils, hhsize):
merge_cols = [_hh_id_, _ptype_, "M", "N", "H"]

# add attributes for joint tour utility
model_settings = state.filesystem.read_model_settings("cdap.yaml")
additional_merge_cols = model_settings.get("JOINT_TOUR_USEFUL_COLUMNS", None)
from activitysim.abm.models.cdap import CdapSettings

model_settings = CdapSettings.read_settings_file(state.filesystem, "cdap.yaml")
additional_merge_cols = model_settings.JOINT_TOUR_USEFUL_COLUMNS
if additional_merge_cols is not None:
merge_cols.extend(additional_merge_cols)

Expand Down Expand Up @@ -972,7 +976,6 @@ def household_activity_choices(
if len(utils.index) == 0:
return pd.Series(dtype="float64")

probs = logit.utils_to_probs(state, utils, trace_label=trace_label)
# calculate joint tour utility
if add_joint_tour_utility & (hhsize > 1):
# calculate joint utils
Expand All @@ -996,6 +999,8 @@ def household_activity_choices(
# add joint util to util
utils = utils.add(joint_tour_utils)

probs = logit.utils_to_probs(state, utils, trace_label=trace_label)

# select an activity pattern alternative for each household based on probability
# result is a series indexed on _hh_index_ with the (0 based) index of the column from probs
idx_choices, rands = logit.make_choices(state, probs, trace_label=trace_label)
Expand Down Expand Up @@ -1278,20 +1283,6 @@ def _run_cdap(
persons["cdap_activity"] = person_choices
chunk_sizer.log_df(trace_label, "persons", persons)

# return household joint tour flag
if add_joint_tour_utility:
hh_activity_choices = hh_activity_choices.to_frame(name="hh_choices")
hh_activity_choices["has_joint_tour"] = hh_activity_choices["hh_choices"].apply(
lambda x: 1 if "J" in x else 0
)

# return household joint tour flag
if add_joint_tour_utility:
hh_activity_choices = hh_activity_choices.to_frame(name="hh_choices")
hh_activity_choices["has_joint_tour"] = hh_activity_choices["hh_choices"].apply(
lambda x: 1 if "J" in x else 0
)

# return household joint tour flag
if add_joint_tour_utility:
hh_activity_choices = hh_activity_choices.to_frame(name="hh_choices")
Expand Down
3 changes: 3 additions & 0 deletions activitysim/abm/models/util/test/configs/cdap.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@ PERSON_TYPE_MAP:
- 6
- 7
- 8

INDIV_AND_HHSIZE1_SPEC: cdap_indiv_and_hhsize1.csv
COEFFICIENTS: cdap_coefficients.csv
3 changes: 2 additions & 1 deletion activitysim/abm/tables/persons.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ def persons_merged(
households,
left_on="household_id",
)
if disaggregate_accessibility is not None and not disaggregate_accessibility.empty:
if state.is_table("disaggregate_accessibility"):
disaggregate_accessibility = state.get_table("disaggregate_accessibility")
persons = simple_table_join(
persons,
disaggregate_accessibility,
Expand Down
4 changes: 2 additions & 2 deletions activitysim/core/configuration/logit.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -212,15 +212,15 @@ class TourLocationComponentSettings(LocationComponentSettings, extra="forbid"):
DEST_CHOICE_SAMPLE_TABLE_NAME: str | None = None
CHOOSER_TABLE_NAME: str | None = None
CHOOSER_SEGMENT_COLUMN_NAME: str | None = None
SEGMENT_IDS: dict[str, int] | None = None
SEGMENT_IDS: dict[str, int] | dict[str, str] | dict[str, bool] | None = None
SHADOW_PRICE_TABLE: str | None = None
MODELED_SIZE_TABLE: str | None = None
annotate_persons: PreprocessorSettings | None = None
annotate_households: PreprocessorSettings | None = None
SIMULATE_CHOOSER_COLUMNS: list[str] | None = None
ALT_DEST_COL_NAME: str
LOGSUM_TOUR_PURPOSE: str | dict[str, str] | None = None
MODEL_SELECTOR: Literal["workplace", "school", None] = None
MODEL_SELECTOR: str | None = None
SAVED_SHADOW_PRICE_TABLE_NAME: str | None = None
CHOOSER_ID_COLUMN: str = "person_id"

Expand Down

0 comments on commit 8e1fbcd

Please sign in to comment.