From f65da4970a828f846e7dabd6b4c7ae6846c9bbe5 Mon Sep 17 00:00:00 2001 From: Alex Higgs Date: Fri, 11 Jun 2021 17:27:54 +0000 Subject: [PATCH 1/2] Split some feature files for faster parallelism --- .../bridge/bridge_one_hub_one_link.feature | 180 ++++++ ...ure => bridge_one_hub_three_links.feature} | 539 +----------------- .../bridge/bridge_one_hub_two_links.feature | 361 ++++++++++++ .../sats/sats_period_mat_base.feature | 116 ++++ .../sats_period_mat_inferred_range.feature | 203 +++++++ .../sats/sats_period_mat_other.feature | 102 ++++ .../sats_period_mat_provided_range.feature | 215 +++++++ 7 files changed, 1178 insertions(+), 538 deletions(-) create mode 100644 test_project/features/bridge/bridge_one_hub_one_link.feature rename test_project/features/bridge/{bridge.feature => bridge_one_hub_three_links.feature} (65%) create mode 100644 test_project/features/bridge/bridge_one_hub_two_links.feature create mode 100644 test_project/features/sats/sats_period_mat_base.feature create mode 100644 test_project/features/sats/sats_period_mat_inferred_range.feature create mode 100644 test_project/features/sats/sats_period_mat_other.feature create mode 100644 test_project/features/sats/sats_period_mat_provided_range.feature diff --git a/test_project/features/bridge/bridge_one_hub_one_link.feature b/test_project/features/bridge/bridge_one_hub_one_link.feature new file mode 100644 index 000000000..5f41047ac --- /dev/null +++ b/test_project/features/bridge/bridge_one_hub_one_link.feature @@ -0,0 +1,180 @@ +@fixture.set_workdir +Feature: Bridge table - Base Bridge behaviour with one hub and one/two/three links + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with the AS_OF date and LDTS equal + Given the BRIDGE_CUSTOMER_ORDER table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-06-01 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | + | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | + | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | + | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | + | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with AS_OF dates in the past + Given the BRIDGE_CUSTOMER_ORDER table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-05-30 00:00:00.000 | + | 2018-05-31 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 12:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-02 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-02 12:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 00:00:00.000 | 2018-06-02 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 12:00:00.000 | 2018-06-02 12:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with AS_OF dates in the future + Given the BRIDGE_CUSTOMER_ORDER table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-06-03 00:00:00.000 | + | 2018-06-04 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 12:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-02 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-02 12:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 00:00:00.000 | 2018-06-02 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 12:00:00.000 | 2018-06-02 12:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | + | md5('1001') | 2018-06-03 00:00:00.000 | md5('1001\|\|100') | + | md5('1002') | 2018-06-03 00:00:00.000 | md5('1002\|\|200') | + | md5('1003') | 2018-06-03 00:00:00.000 | md5('1003\|\|300') | + | md5('1004') | 2018-06-03 00:00:00.000 | md5('1004\|\|400') | + | md5('1001') | 2018-06-04 00:00:00.000 | md5('1001\|\|100') | + | md5('1002') | 2018-06-04 00:00:00.000 | md5('1002\|\|200') | + | md5('1003') | 2018-06-04 00:00:00.000 | md5('1003\|\|300') | + | md5('1004') | 2018-06-04 00:00:00.000 | md5('1004\|\|400') | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with multiple loads and an encompassing range of AS OF dates + Given the BRIDGE_CUSTOMER_ORDER table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 201 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 301 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-05-31 00:00:00.000 | + | 2018-06-01 00:00:00.000 | + | 2018-06-02 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|201') | md5('1002') | md5('201') | 2018-06-01 12:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|301') | md5('1003') | md5('301') | 2018-06-01 12:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|201') | md5('1002') | md5('201') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|301') | md5('1003') | md5('301') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | + | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | + | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | + | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | + | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | + | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|201') | + | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | + | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|301') | \ No newline at end of file diff --git a/test_project/features/bridge/bridge.feature b/test_project/features/bridge/bridge_one_hub_three_links.feature similarity index 65% rename from test_project/features/bridge/bridge.feature rename to test_project/features/bridge/bridge_one_hub_three_links.feature index 076960add..8a9f41f0d 100644 --- a/test_project/features/bridge/bridge.feature +++ b/test_project/features/bridge/bridge_one_hub_three_links.feature @@ -1,543 +1,6 @@ @fixture.set_workdir Feature: Bridge table - Base Bridge behaviour with one hub and one/two/three links - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with the AS_OF date and LDTS equal - Given the BRIDGE_CUSTOMER_ORDER table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-06-01 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | - | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | - | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | - | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | - | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with AS_OF dates in the past - Given the BRIDGE_CUSTOMER_ORDER table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-05-30 00:00:00.000 | - | 2018-05-31 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 12:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-02 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-02 12:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 00:00:00.000 | 2018-06-02 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 12:00:00.000 | 2018-06-02 12:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with AS_OF dates in the future - Given the BRIDGE_CUSTOMER_ORDER table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-06-03 00:00:00.000 | - | 2018-06-04 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 12:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-02 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-02 12:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-02 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 00:00:00.000 | 2018-06-02 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-02 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-02 12:00:00.000 | 2018-06-02 12:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | - | md5('1001') | 2018-06-03 00:00:00.000 | md5('1001\|\|100') | - | md5('1002') | 2018-06-03 00:00:00.000 | md5('1002\|\|200') | - | md5('1003') | 2018-06-03 00:00:00.000 | md5('1003\|\|300') | - | md5('1004') | 2018-06-03 00:00:00.000 | md5('1004\|\|400') | - | md5('1001') | 2018-06-04 00:00:00.000 | md5('1001\|\|100') | - | md5('1002') | 2018-06-04 00:00:00.000 | md5('1002\|\|200') | - | md5('1003') | 2018-06-04 00:00:00.000 | md5('1003\|\|300') | - | md5('1004') | 2018-06-04 00:00:00.000 | md5('1004\|\|400') | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and one link with multiple loads and an encompassing range of AS OF dates - Given the BRIDGE_CUSTOMER_ORDER table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | BRIDGE_CUSTOMER_ORDER | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 201 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 301 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-05-31 00:00:00.000 | - | 2018-06-01 00:00:00.000 | - | 2018-06-02 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|201') | md5('1002') | md5('201') | 2018-06-01 12:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|301') | md5('1003') | md5('301') | 2018-06-01 12:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|201') | md5('1002') | md5('201') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|301') | md5('1003') | md5('301') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | - | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | - | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | - | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | - | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | - | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|201') | - | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | - | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|301') | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with the AS_OF date and LDTS equal - Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | - | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the RAW_ORDER_PRODUCT table contains data - | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_ORDER_PRODUCT stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-06-01 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the LINK_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | - | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | - | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | - | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | - | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with AS_OF dates in the past - Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | - | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the RAW_ORDER_PRODUCT table contains data - | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_ORDER_PRODUCT stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-05-30 00:00:00.000 | - | 2018-05-31 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the LINK_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with AS_OF dates in the future - Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | - | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the RAW_ORDER_PRODUCT table contains data - | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_ORDER_PRODUCT stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-06-02 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the LINK_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | - | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | - | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | - | md5('1004') | 2018-06-02 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with multiple loads and an encompassing range of AS_OF dates - Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | - | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1001 | 101 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the RAW_ORDER_PRODUCT table contains data - | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 101 | AAAA | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 101 | AAAB | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_ORDER_PRODUCT stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-05-31 00:00:00.000 | - | 2018-06-01 00:00:00.000 | - | 2018-06-02 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 12:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the LINK_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | - | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 12:00:00.000 | * | - | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 12:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | - | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | - | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | - | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | - | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|101') | md5('101\|\|AAAA') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|101') | md5('101\|\|AAAB') | - | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | - | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | - | md5('1004') | 2018-06-02 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | - - @fixture.bridge - Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with history and encompassing range of AS_OF dates - Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist - And the raw vault contains empty tables - | HUB | LINK | EFF_SAT | BRIDGE | - | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | - | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | - And the RAW_CUSTOMER_ORDER table contains data - | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 1001 | 100 | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | - | 1001 | 100 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1001 | 101 | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | - | 1001 | 101 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1003 | 300 | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | * | - | 1003 | 300 | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | * | - | 1003 | 300 | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_CUSTOMER_ORDER stage - And the RAW_ORDER_PRODUCT table contains data - | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | - | 100 | AAA | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | - | 100 | AAA | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 100 | AAB | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 101 | AAAA | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | - | 101 | AAAB | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | - | 101 | AAAA | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 300 | CCC | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | * | - | 300 | CCA | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | * | - | 300 | CCC | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | * | - | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | - And I create the STG_ORDER_PRODUCT stage - And the AS_OF_DATE table is created and populated with data - | AS_OF_DATE | - | 2018-05-31 00:00:00.000 | - | 2018-06-01 00:00:00.000 | - | 2018-06-02 00:00:00.000 | - When I load the vault - Then the HUB_CUSTOMER table should contain expected data - | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | - | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | - | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | - | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | - | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | - Then the LINK_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | - | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 00:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data - | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | - | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | - | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the LINK_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | - | md5('100\|\|AAB') | md5('100') | md5('AAB') | 2018-06-01 12:00:00.000 | * | - | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 00:00:00.000 | * | - | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 00:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCA') | md5('300') | md5('CCA') | 2018-06-01 09:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | - Then the EFF_SAT_ORDER_PRODUCT table should contain expected data - | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('100\|\|AAB') | md5('100') | md5('AAB') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | - | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - | md5('300\|\|CCA') | md5('300') | md5('CCA') | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | - | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | - | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | - Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | - | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | - | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAB') | - | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|101') | md5('101\|\|AAAA') | - | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | - | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | - | md5('1004') | 2018-06-02 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | - @fixture.bridge Scenario: [BASE-LOAD] Base load into a bridge table from one hub and three links with the AS_OF date and LDTS equal Given the BRIDGE_CUSTOMER_ORDER_PRODUCT_COMPONENT table does not exist @@ -696,7 +159,7 @@ Feature: Bridge table - Base Bridge behaviour with one hub and one/two/three lin | md5('CCC-0\|\|CCC') | md5('CCC') | md5('CCC-0') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | | md5('DDD-0\|\|DDD') | md5('DDD') | md5('DDD-0') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | Then the BRIDGE_CUSTOMER_ORDER_PRODUCT_COMPONENT table should contain expected data - | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | LINK_PRODUCT_COMPONENT_PK | + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | LINK_PRODUCT_COMPONENT_PK | @fixture.bridge Scenario: [BASE-LOAD] Base load into a bridge table from one hub and three links with the AS_OF dates in the future diff --git a/test_project/features/bridge/bridge_one_hub_two_links.feature b/test_project/features/bridge/bridge_one_hub_two_links.feature new file mode 100644 index 000000000..3d392d70d --- /dev/null +++ b/test_project/features/bridge/bridge_one_hub_two_links.feature @@ -0,0 +1,361 @@ +@fixture.set_workdir +Feature: Bridge table - Base Bridge behaviour with one hub and one/two/three links + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with the AS_OF date and LDTS equal + Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | + | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the RAW_ORDER_PRODUCT table contains data + | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_ORDER_PRODUCT stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-06-01 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the LINK_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | + | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | + | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | + | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | + | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with AS_OF dates in the past + Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | + | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the RAW_ORDER_PRODUCT table contains data + | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_ORDER_PRODUCT stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-05-30 00:00:00.000 | + | 2018-05-31 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the LINK_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with AS_OF dates in the future + Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | + | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the RAW_ORDER_PRODUCT table contains data + | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_ORDER_PRODUCT stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-06-02 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the LINK_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | + | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | + | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | + | md5('1004') | 2018-06-02 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with multiple loads and an encompassing range of AS_OF dates + Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | + | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1001 | 101 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the RAW_ORDER_PRODUCT table contains data + | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 100 | AAA | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 101 | AAAA | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 101 | AAAB | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 300 | CCC | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_ORDER_PRODUCT stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-05-31 00:00:00.000 | + | 2018-06-01 00:00:00.000 | + | 2018-06-02 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 12:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the LINK_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | + | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 12:00:00.000 | * | + | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 12:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | + | md5('1001') | 2018-06-01 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | + | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | + | md5('1003') | 2018-06-01 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | + | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|101') | md5('101\|\|AAAA') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|101') | md5('101\|\|AAAB') | + | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | + | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | + | md5('1004') | 2018-06-02 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | + + @fixture.bridge + Scenario: [BASE-LOAD] Base load into a bridge table from one hub and two links with history and encompassing range of AS_OF dates + Given the BRIDGE_CUSTOMER_ORDER_PRODUCT table does not exist + And the raw vault contains empty tables + | HUB | LINK | EFF_SAT | BRIDGE | + | HUB_CUSTOMER | LINK_ORDER_PRODUCT | EFF_SAT_ORDER_PRODUCT | BRIDGE_CUSTOMER_ORDER_PRODUCT | + | | LINK_CUSTOMER_ORDER | EFF_SAT_CUSTOMER_ORDER | | + And the RAW_CUSTOMER_ORDER table contains data + | CUSTOMER_ID | ORDER_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 1001 | 100 | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | + | 1001 | 100 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1001 | 101 | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | + | 1001 | 101 | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1002 | 200 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1003 | 300 | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | * | + | 1003 | 300 | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | * | + | 1003 | 300 | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 1004 | 400 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_CUSTOMER_ORDER stage + And the RAW_ORDER_PRODUCT table contains data + | ORDER_ID | PRODUCT_ID | LOAD_DATETIME | END_DATE | SOURCE | + | 100 | AAA | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | + | 100 | AAA | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 100 | AAB | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 101 | AAAA | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | + | 101 | AAAB | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | * | + | 101 | AAAA | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 200 | BBB | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 300 | CCC | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | * | + | 300 | CCA | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | * | + | 300 | CCC | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | * | + | 400 | DDD | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | * | + And I create the STG_ORDER_PRODUCT stage + And the AS_OF_DATE table is created and populated with data + | AS_OF_DATE | + | 2018-05-31 00:00:00.000 | + | 2018-06-01 00:00:00.000 | + | 2018-06-02 00:00:00.000 | + When I load the vault + Then the HUB_CUSTOMER table should contain expected data + | CUSTOMER_PK | CUSTOMER_ID | LOAD_DATETIME | SOURCE | + | md5('1001') | 1001 | 2018-06-01 00:00:00.000 | * | + | md5('1002') | 1002 | 2018-06-01 00:00:00.000 | * | + | md5('1003') | 1003 | 2018-06-01 00:00:00.000 | * | + | md5('1004') | 1004 | 2018-06-01 00:00:00.000 | * | + Then the LINK_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | * | + | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 00:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data + | CUSTOMER_ORDER_PK | CUSTOMER_FK | ORDER_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | + | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | + | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the LINK_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | * | + | md5('100\|\|AAB') | md5('100') | md5('AAB') | 2018-06-01 12:00:00.000 | * | + | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 00:00:00.000 | * | + | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 00:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCA') | md5('300') | md5('CCA') | 2018-06-01 09:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | * | + Then the EFF_SAT_ORDER_PRODUCT table should contain expected data + | ORDER_PRODUCT_PK | ORDER_FK | PRODUCT_FK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('100\|\|AAA') | md5('100') | md5('AAA') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('100\|\|AAB') | md5('100') | md5('AAB') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('101\|\|AAAB') | md5('101') | md5('AAAB') | 2018-06-01 00:00:00.000 | 2018-06-01 11:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('101\|\|AAAA') | md5('101') | md5('AAAA') | 2018-06-01 12:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 12:00:00.000 | 2018-06-01 12:00:00.000 | * | + | md5('200\|\|BBB') | md5('200') | md5('BBB') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 00:00:00.000 | 2018-06-01 08:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + | md5('300\|\|CCA') | md5('300') | md5('CCA') | 2018-06-01 09:00:00.000 | 2018-06-01 17:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | + | md5('300\|\|CCC') | md5('300') | md5('CCC') | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | + | md5('400\|\|DDD') | md5('400') | md5('DDD') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | + Then the BRIDGE_CUSTOMER_ORDER_PRODUCT table should contain expected data + | CUSTOMER_PK | AS_OF_DATE | LINK_CUSTOMER_ORDER_PK | LINK_ORDER_PRODUCT_PK | + | md5('1002') | 2018-06-01 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | + | md5('1004') | 2018-06-01 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAA') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|100') | md5('100\|\|AAB') | + | md5('1001') | 2018-06-02 00:00:00.000 | md5('1001\|\|101') | md5('101\|\|AAAA') | + | md5('1002') | 2018-06-02 00:00:00.000 | md5('1002\|\|200') | md5('200\|\|BBB') | + | md5('1003') | 2018-06-02 00:00:00.000 | md5('1003\|\|300') | md5('300\|\|CCC') | + | md5('1004') | 2018-06-02 00:00:00.000 | md5('1004\|\|400') | md5('400\|\|DDD') | \ No newline at end of file diff --git a/test_project/features/sats/sats_period_mat_base.feature b/test_project/features/sats/sats_period_mat_base.feature new file mode 100644 index 000000000..667f8eb49 --- /dev/null +++ b/test_project/features/sats/sats_period_mat_base.feature @@ -0,0 +1,116 @@ +@fixture.set_workdir +Feature: Satellites Loaded using Period Materialization for base loads + + @fixture.satellite + Scenario: [SAT-PERIOD-MAT-BASE] Base load of a satellite with one value in rank column loads first rank + Given the SATELLITE table does not exist + And the RAW_STAGE table contains data + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | CUSTOMER_PHONE | LOAD_DATE | SOURCE | + | 1001 | Alice | 1997-04-24 | 17-214-233-1214 | 1993-01-01 | * | + | 1001 | Alice | 1997-04-24 | 17-214-233-1214 | 1993-01-02 | * | + | 1002 | Bob | 2006-04-17 | 17-214-233-1215 | 1993-01-01 | * | + | 1003 | Chad | 2013-02-04 | 17-214-233-1216 | 1993-01-01 | * | + | 1004 | Dom | 2018-04-13 | 17-214-233-1217 | 1993-01-01 | * | + | 1004 | Dom | 2018-04-13 | 17-214-233-1217 | 1993-01-02 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 1993-01-01 to 1993-01-03 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_PHONE | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1997-04-24\|\|1001\|\|ALICE\|\|17-214-233-1214') | Alice | 17-214-233-1214 | 1997-04-24 | 1993-01-01 | 1993-01-01 | * | + | md5('1002') | md5('2006-04-17\|\|1002\|\|BOB\|\|17-214-233-1215') | Bob | 17-214-233-1215 | 2006-04-17 | 1993-01-01 | 1993-01-01 | * | + | md5('1003') | md5('2013-02-04\|\|1003\|\|CHAD\|\|17-214-233-1216') | Chad | 17-214-233-1216 | 2013-02-04 | 1993-01-01 | 1993-01-01 | * | + | md5('1004') | md5('2018-04-13\|\|1004\|\|DOM\|\|17-214-233-1217') | Dom | 17-214-233-1217 | 2018-04-13 | 1993-01-01 | 1993-01-01 | * | + + @fixture.satellite + Scenario: [SAT-PERIOD-MAT-BASE] Incremental load of a satellite with one value in rank column loads all records + Given the SATELLITE table does not exist + And the RAW_STAGE table contains data + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | CUSTOMER_PHONE | LOAD_DATE | SOURCE | + | 1001 | Alice | 1997-04-24 | 17-214-233-1214 | 1993-01-01 | * | + | 1002 | Bob | 2006-04-17 | 17-214-233-1215 | 1993-01-01 | * | + | 1003 | Chad | 2013-02-04 | 17-214-233-1216 | 1993-01-01 | * | + | 1004 | Dom | 2018-04-13 | 17-214-233-1217 | 1993-01-01 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 1993-01-01 to 1993-01-03 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_PHONE | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1997-04-24\|\|1001\|\|ALICE\|\|17-214-233-1214') | Alice | 17-214-233-1214 | 1997-04-24 | 1993-01-01 | 1993-01-01 | * | + | md5('1002') | md5('2006-04-17\|\|1002\|\|BOB\|\|17-214-233-1215') | Bob | 17-214-233-1215 | 2006-04-17 | 1993-01-01 | 1993-01-01 | * | + | md5('1003') | md5('2013-02-04\|\|1003\|\|CHAD\|\|17-214-233-1216') | Chad | 17-214-233-1216 | 2013-02-04 | 1993-01-01 | 1993-01-01 | * | + | md5('1004') | md5('2018-04-13\|\|1004\|\|DOM\|\|17-214-233-1217') | Dom | 17-214-233-1217 | 2018-04-13 | 1993-01-01 | 1993-01-01 | * | + + @fixture.satellite + Scenario: [SAT-PERIOD-MAT-BASE] Incremental load of a satellite should not load PK NULLs + Given the SATELLITE table does not exist + And the RAW_STAGE table contains data + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | CUSTOMER_PHONE | LOAD_DATE | SOURCE | + | 1001 | Alice | 1997-04-24 | 17-214-233-1214 | 1993-01-01 | * | + | 1002 | Bob | 2006-04-17 | 17-214-233-1215 | 1993-01-01 | * | + | 1003 | Chad | 2013-02-04 | 17-214-233-1216 | 1993-01-01 | * | + | 1004 | Dom | 2018-04-13 | 17-214-233-1217 | 1993-01-01 | * | + | | Emily | 2018-04-11 | 17-214-233-1218 | 1993-01-01 | * | + | | Fred | 2018-06-11 | 17-214-233-1219 | 1993-01-01 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 1993-01-01 to 1993-01-03 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_PHONE | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1997-04-24\|\|1001\|\|ALICE\|\|17-214-233-1214') | Alice | 17-214-233-1214 | 1997-04-24 | 1993-01-01 | 1993-01-01 | * | + | md5('1002') | md5('2006-04-17\|\|1002\|\|BOB\|\|17-214-233-1215') | Bob | 17-214-233-1215 | 2006-04-17 | 1993-01-01 | 1993-01-01 | * | + | md5('1003') | md5('2013-02-04\|\|1003\|\|CHAD\|\|17-214-233-1216') | Chad | 17-214-233-1216 | 2013-02-04 | 1993-01-01 | 1993-01-01 | * | + | md5('1004') | md5('2018-04-13\|\|1004\|\|DOM\|\|17-214-233-1217') | Dom | 17-214-233-1217 | 2018-04-13 | 1993-01-01 | 1993-01-01 | * | + + @fixture.enable_full_refresh + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT-BASE] Base load of a satellite using full refresh and start and end dates should only contain first period records + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 2019-05-05 to 2019-05-06 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT-BASE] Base load of a satellite should not load PK NULLs + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | | Emily | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 2019-05-05 to 2019-05-06 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + + @fixture.enable_full_refresh + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT-BASE] Base load of a satellite using full refresh and only start date should only contain first period records + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with start date: 2019-05-05 + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | \ No newline at end of file diff --git a/test_project/features/sats/sats_period_mat_inferred_range.feature b/test_project/features/sats/sats_period_mat_inferred_range.feature new file mode 100644 index 000000000..911b31f3e --- /dev/null +++ b/test_project/features/sats/sats_period_mat_inferred_range.feature @@ -0,0 +1,203 @@ +@fixture.set_workdir +Feature: Satellites Loaded using Period Materialization and inferred date ranges + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into non-existent satellite + Given the SATELLITE table does not exist + And the RAW_STAGE table contains data + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day + And I insert by period into the SATELLITE sat by day + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into empty satellite. + Given the RAW_STAGE stage is empty + And the SATELLITE sat is empty + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into populated satellite, with partial duplicates. + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + | 1013 | Zach | 1995-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + | md5('1013') | md5('1995-06-16\|\|1013\|\|ZACH') | Zach | 1995-06-16 | 2019-05-07 | 2019-05-07 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into populated satellite, with all duplicates. + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | \ No newline at end of file diff --git a/test_project/features/sats/sats_period_mat_other.feature b/test_project/features/sats/sats_period_mat_other.feature new file mode 100644 index 000000000..ac6512c85 --- /dev/null +++ b/test_project/features/sats/sats_period_mat_other.feature @@ -0,0 +1,102 @@ +@fixture.set_workdir +Feature: Satellites Loaded using Period Materialization for other scenarios + + # ABORTED LOADS + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Simulate a restart of an aborted load + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + + # INFERRED DATE RANGE (MONTHLY) + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several monthly cycles with insert_by_period into empty satellite. + Given the RAW_STAGE stage is empty + And the SATELLITE sat is empty + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-06-05 | 2019-06-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-06-05 | 2019-06-05 | * | + | 1004 | David | 1992-01-30 | 2019-06-05 | 2019-06-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-06-05 | 2019-06-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-07-06 | 2019-07-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-07-06 | 2019-07-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-07-06 | 2019-07-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-07-06 | 2019-07-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-08-07 | 2019-08-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-08-07 | 2019-08-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-08-07 | 2019-08-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-08-07 | 2019-08-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-08-07 | 2019-08-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by month + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-06-05 | 2019-06-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-06-05 | 2019-06-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-06-05 | 2019-06-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-06-05 | 2019-06-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-07-06 | 2019-07-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-07-06 | 2019-07-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-07-06 | 2019-07-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-07-06 | 2019-07-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-08-07 | 2019-08-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-08-07 | 2019-08-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-08-07 | 2019-08-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-08-07 | 2019-08-07 | * | \ No newline at end of file diff --git a/test_project/features/sats/sats_period_mat_provided_range.feature b/test_project/features/sats/sats_period_mat_provided_range.feature new file mode 100644 index 000000000..72cee4450 --- /dev/null +++ b/test_project/features/sats/sats_period_mat_provided_range.feature @@ -0,0 +1,215 @@ +@fixture.set_workdir +Feature: Satellites Loaded using Period Materialization for provided date ranges + +# TODO: Work out a way to stop this becoming a larger test as time goes on (will always do until current date) +# # PROVIDED DATE RANGE [START-ONLY] (DAILY) +# +# @fixture.satellite_cycle +# Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into non-existent satellite, with start date only. +# Given the SATELLITE table does not exist +# And the RAW_STAGE table contains data +# | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | +# | 1002 | Beth | 1995-08-07 | 2021-01-11 | 2021-01-11 | * | +# | 1003 | Charley | 1990-02-03 | 2021-01-11 | 2021-01-11 | * | +# | 1010 | Jenny | 1991-03-21 | 2021-01-11 | 2021-01-11 | * | +# | 1012 | Albert | 1990-02-03 | 2021-01-11 | 2021-01-11 | * | +# | 1002 | Beah | 1995-08-07 | 2021-01-12 | 2021-01-12 | * | +# | 1003 | Chris | 1990-02-03 | 2021-01-12 | 2021-01-12 | * | +# | 1004 | David | 1992-01-30 | 2021-01-12 | 2021-01-12 | * | +# | 1010 | Jenny | 1991-03-25 | 2021-01-12 | 2021-01-12 | * | +# And I create the STG_CUSTOMER stage +# And I insert by period into the SATELLITE sat by day with start date: 2021-01-11 +# And I insert by period into the SATELLITE sat by day with start date: 2021-01-11 +# Then the SATELLITE table should contain expected data +# | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | +# | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2021-01-11 | 2021-01-11 | * | +# | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2021-01-11 | 2021-01-11 | * | +# | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2021-01-11 | 2021-01-11 | * | +# | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2021-01-11 | 2021-01-11 | * | +# | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2021-01-12 | 2021-01-12 | * | +# | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2021-01-12 | 2021-01-12 | * | +# | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2021-01-12 | 2021-01-12 | * | +# | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2021-01-12 | 2021-01-12 | * | + + # PROVIDED DATE RANGE [START-AND-STOP] (DAILY) + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into non-existent satellite, with date range. + Given the SATELLITE table does not exist + And the RAW_STAGE table contains data + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 2019-05-05 to 2019-05-06 and LDTS LOAD_DATE + And I insert by period into the SATELLITE sat by day with date range: 2019-05-05 to 2019-05-06 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into empty satellite, with date range. + Given the RAW_STAGE stage is empty + And the SATELLITE sat is empty + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 2019-05-04 to 2019-05-06 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into populated satellite, with partial duplicates and date range + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + | 1013 | Zach | 1995-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 2019-05-04 to 2019-05-06 and LDTS LOAD_DATE + + # =============== CHECKS =================== + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CLAIRE') | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | md5('1005') | md5('2001-07-23\|\|1005\|\|ELWYN') | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | md5('1006') | md5('1960-01-01\|\|1006\|\|FREIA') | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1007') | md5('1990-02-03\|\|1007\|\|GEOFF') | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | md5('1011') | md5('1978-06-16\|\|1011\|\|KAREN') | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + + @fixture.satellite_cycle + Scenario: [SAT-PERIOD-MAT] Satellite load over several daily cycles with insert_by_period into populated satellite, with all duplicates and date range. + Given the RAW_STAGE stage is empty + And the SATELLITE sat is already populated with data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + When the RAW_STAGE is loaded + | CUSTOMER_ID | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | 1001 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1010 | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | 1012 | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | 1003 | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | 1004 | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | + | 1002 | Beth | 1995-08-07 | 2019-05-06 | 2019-05-06 | * | + | 1003 | Claire | 1990-02-03 | 2019-05-06 | 2019-05-06 | * | + | 1005 | Elwyn | 2001-07-23 | 2019-05-06 | 2019-05-06 | * | + | 1006 | Freia | 1960-01-01 | 2019-05-06 | 2019-05-06 | * | + | 1002 | Beah | 1995-08-07 | 2019-05-07 | 2019-05-07 | * | + | 1003 | Charley | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1007 | Geoff | 1990-02-03 | 2019-05-07 | 2019-05-07 | * | + | 1010 | Jenny | 1991-03-25 | 2019-05-07 | 2019-05-07 | * | + | 1011 | Karen | 1978-06-16 | 2019-05-07 | 2019-05-07 | * | + And I create the STG_CUSTOMER stage + And I insert by period into the SATELLITE sat by day with date range: 2019-05-04 to 2019-05-05 and LDTS LOAD_DATE + Then the SATELLITE table should contain expected data + | CUSTOMER_PK | HASHDIFF | CUSTOMER_NAME | CUSTOMER_DOB | EFFECTIVE_FROM | LOAD_DATE | SOURCE | + | md5('1001') | md5('1990-02-03\|\|1001\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BETH') | Beth | 1995-08-07 | 2019-05-04 | 2019-05-04 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHARLEY') | Charley | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1010') | md5('1991-03-21\|\|1010\|\|JENNY') | Jenny | 1991-03-21 | 2019-05-04 | 2019-05-04 | * | + | md5('1012') | md5('1990-02-03\|\|1012\|\|ALBERT') | Albert | 1990-02-03 | 2019-05-04 | 2019-05-04 | * | + | md5('1002') | md5('1995-08-07\|\|1002\|\|BEAH') | Beah | 1995-08-07 | 2019-05-05 | 2019-05-05 | * | + | md5('1003') | md5('1990-02-03\|\|1003\|\|CHRIS') | Chris | 1990-02-03 | 2019-05-05 | 2019-05-05 | * | + | md5('1004') | md5('1992-01-30\|\|1004\|\|DAVID') | David | 1992-01-30 | 2019-05-05 | 2019-05-05 | * | + | md5('1010') | md5('1991-03-25\|\|1010\|\|JENNY') | Jenny | 1991-03-25 | 2019-05-05 | 2019-05-05 | * | \ No newline at end of file From e21a9d1d3476286cab6f5d0e9437f9709cebfeef Mon Sep 17 00:00:00 2001 From: Alex Higgs Date: Fri, 11 Jun 2021 17:32:24 +0000 Subject: [PATCH 2/2] Cleanup --- ...=> eff_sat_auto_end_dating_detail.feature} | 26 +------------------ 1 file changed, 1 insertion(+), 25 deletions(-) rename test_project/features/eff_sats/{eff_sat_must_pass_tests.feature => eff_sat_auto_end_dating_detail.feature} (99%) diff --git a/test_project/features/eff_sats/eff_sat_must_pass_tests.feature b/test_project/features/eff_sats/eff_sat_auto_end_dating_detail.feature similarity index 99% rename from test_project/features/eff_sats/eff_sat_must_pass_tests.feature rename to test_project/features/eff_sats/eff_sat_auto_end_dating_detail.feature index 2d98917cf..a674a13a9 100644 --- a/test_project/features/eff_sats/eff_sat_must_pass_tests.feature +++ b/test_project/features/eff_sats/eff_sat_auto_end_dating_detail.feature @@ -1,6 +1,6 @@ @fixture.set_workdir Feature: Effectivity Satellites - Exploring the auto end dating of effectivity satellite in the light of the bridge feature + Further depth of testing for the auto-end-dating of effectivity satellite ################## ORDER_FK is DRIVING KEY ################## @@ -81,7 +81,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Three loads; adding (completely) new relationships in each load Given the EFF_SAT_ORDER_CUSTOMER table does not exist -# First load... And the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -94,7 +93,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_ORDER_CUSTOMER table should contain expected data | ORDER_CUSTOMER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1002 | 200 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -115,7 +113,6 @@ Feature: Effectivity Satellites | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1003\|\|300') | md5('1003') | md5('300') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1004\|\|400') | md5('1004') | md5('400') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | -# Third load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1005 | 500 | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | @@ -141,7 +138,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Three loads; the same CUSTOMER placing a varying number of ORDERS at different times Given the EFF_SAT_ORDER_CUSTOMER table does not exist -# First load... And the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -154,7 +150,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_ORDER_CUSTOMER table should contain expected data | ORDER_CUSTOMER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 101 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -172,7 +167,6 @@ Feature: Effectivity Satellites | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1001\|\|102') | md5('1001') | md5('102') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | -# Third load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 103 | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | @@ -196,7 +190,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [BASE-LOAD] Three loads; going from an empty table to 1 CUSTOMER per ORDER + flip-flop situation Given the EFF_SAT_ORDER_CUSTOMER table does not exist -# First load... And the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -209,7 +202,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_ORDER_CUSTOMER table should contain expected data | ORDER_CUSTOMER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1002 | 100 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -225,7 +217,6 @@ Feature: Effectivity Satellites | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1002\|\|100') | md5('1002') | md5('100') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | -# Third load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | @@ -248,7 +239,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Three loads; the last load will bring new open, new reopen and closed records in the eff sat Given the EFF_SAT_ORDER_CUSTOMER table does not exist -# First load... And the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -261,7 +251,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_ORDER_CUSTOMER table should contain expected data | ORDER_CUSTOMER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1011 | 100 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -280,7 +269,6 @@ Feature: Effectivity Satellites | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1011\|\|100') | md5('1011') | md5('100') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1002\|\|200') | md5('1002') | md5('200') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | -# Third load... Given the RAW_STAGE_ORDER_CUSTOMER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | @@ -341,7 +329,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Two loads; going from 1 ORDER to another (new) ORDER Given the EFF_SAT_CUSTOMER_ORDER table does not exist -# First load... And the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -354,7 +341,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data | CUSTOMER_ORDER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 101 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -375,7 +361,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Two loads; changing the ORDER to another ORDER Given the EFF_SAT_CUSTOMER_ORDER table does not exist -# First load... And the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -388,7 +373,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data | CUSTOMER_ORDER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 101 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -409,7 +393,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Two loads; going from 1 ORDER to 3 (new) ORDERS Given the EFF_SAT_CUSTOMER_ORDER table does not exist -# First load... And the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -422,7 +405,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data | CUSTOMER_ORDER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 101 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -449,7 +431,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Three loads; going from 1 ORDER to 3 (new) ORDERS and then back to 1 (new) ORDER Given the EFF_SAT_CUSTOMER_ORDER table does not exist -# First load... And the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -462,7 +443,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data | CUSTOMER_ORDER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 101 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -484,7 +464,6 @@ Feature: Effectivity Satellites | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1001\|\|102') | md5('1001') | md5('102') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1001\|\|103') | md5('1001') | md5('103') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | -# Third load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 104 | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * | @@ -514,7 +493,6 @@ Feature: Effectivity Satellites @fixture.eff_satellite_testing_auto_end_dating Scenario: [INCR-LOAD] Three loads; going from 1 ORDER to 3 (new) ORDERS and then back to the initial ORDER Given the EFF_SAT_CUSTOMER_ORDER table does not exist -# First load... And the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | @@ -527,7 +505,6 @@ Feature: Effectivity Satellites Then the EFF_SAT_CUSTOMER_ORDER table should contain expected data | CUSTOMER_ORDER_PK | CUSTOMER_PK | ORDER_PK | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | md5('1001\|\|100') | md5('1001') | md5('100') | 2018-06-01 00:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 00:00:00.000 | 2018-06-01 00:00:00.000 | * | -# Second load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 101 | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | @@ -549,7 +526,6 @@ Feature: Effectivity Satellites | md5('1001\|\|101') | md5('1001') | md5('101') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1001\|\|102') | md5('1001') | md5('102') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | | md5('1001\|\|103') | md5('1001') | md5('103') | 2018-06-01 09:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 09:00:00.000 | 2018-06-01 09:00:00.000 | * | -# Third load... Given the RAW_STAGE_CUSTOMER_ORDER table contains data | CUSTOMER_ID | ORDER_ID | START_DATE | END_DATE | EFFECTIVE_FROM | LOAD_DATETIME | SOURCE | | 1001 | 100 | 2018-06-01 18:00:00.000 | 9999-12-31 23:59:59.999 | 2018-06-01 18:00:00.000 | 2018-06-01 18:00:00.000 | * |