DATA-1361 Add ChainID to all tables

[jasadams-imtbl]

Description

Changes to include chain ID in all tables and new isERC1155 attribute in contracts table

Jira

DATA-1361

Type of change

• Bug fix
• New feature
• Documentation
• Configuration

How Has This Been Tested?

Tested against dev-im-data

[github-actions]

Terraform Initialization ⚙️success

Terraform Validation 🤖success

Terraform Plan 📖success

Show Plan

google_dataflow_flex_template_job.flex_template_job: Refreshing state... [id=2023-12-11_15_36_23-15159653732800287599]
google_storage_bucket_object.blockchain_etl_dataflow: Refreshing state... [id=prod-im-data-imx-resource-dataflow-templates/blockchain-etl-dataflow/metadata.json]
google_dataflow_flex_template_job.zkevm_imtbl_mainnet_job: Refreshing state... [id=2023-12-11_15_36_17-13269957987267613837]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  - destroy
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # google_dataflow_flex_template_job.flex_template_job will be destroyed
  # (because google_dataflow_flex_template_job.flex_template_job is not in configuration)
  - resource "google_dataflow_flex_template_job" "flex_template_job" {
      - additional_experiments       = [] -> null
      - container_spec_gcs_path      = "gs://prod-im-data-imx-resource/dataflow-templates/blockchain-etl-dataflow/metadata.json" -> null
      - id                           = "2023-12-11_15_36_23-15159653732800287599" -> null
      - job_id                       = "2023-12-11_15_36_23-15159653732800287599" -> null
      - labels                       = {} -> null
      - machine_type                 = "n1-standard-1" -> null
      - max_workers                  = 0 -> null
      - name                         = "zkevm-imtbl-testnet-etl-dataflow-20231211-233623" -> null
      - num_workers                  = 0 -> null
      - on_delete                    = "drain" -> null
      - parameters                   = {
          - "allowedTimestampSkewSeconds" = "5184000"
          - "chainConfigFile"             = "/template/blockchain_zkevm_imtbl_testnet_prod.json"
          - "diskSizeGb"                  = "30"
          - "enableStreamingEngine"       = "true"
          - "gcpTempLocation"             = "gs://prod-im-data-imx-resource/ethereum-etl/temp"
          - "maxNumWorkers"               = "1"
          - "project"                     = "prod-im-data"
          - "region"                      = "us-central1"
          - "runner"                      = "DataflowRunner"
          - "tempLocation"                = "gs://prod-im-data-imx-resource/ethereum-etl/temp"
          - "workerMachineType"           = "n1-standard-1"
          - "zone"                        = "us-central1-a"
        } -> null
      - project                      = "prod-im-data" -> null
      - region                       = "us-central1" -> null
      - service_account_email        = "[email protected]" -> null
      - skip_wait_on_job_termination = true -> null
      - staging_location             = "gs://dataflow-staging-us-central1-692943280739/staging" -> null
      - state                        = "JOB_STATE_RUNNING" -> null
      - temp_location                = "gs://prod-im-data-imx-resource/ethereum-etl/temp" -> null
      - type                         = "JOB_TYPE_STREAMING" -> null
    }

  # google_dataflow_flex_template_job.zkevm_imtbl_mainnet_job must be replaced
-/+ resource "google_dataflow_flex_template_job" "zkevm_imtbl_mainnet_job" {
      ~ additional_experiments       = [] -> (known after apply)
      ~ id                           = "2023-12-11_15_36_17-13269957987267613837" -> (known after apply)
      ~ job_id                       = "2023-12-11_15_36_17-13269957987267613837" -> (known after apply)
      - labels                       = {} -> null
      - machine_type                 = "n1-standard-1" -> null
      - max_workers                  = 0 -> null
      ~ name                         = "zkevm-imtbl-mainnet-etl-dataflow-20231211-233616" # forces replacement -> (known after apply) # forces replacement
      - num_workers                  = 0 -> null
      ~ service_account_email        = "[email protected]" -> (known after apply)
      ~ staging_location             = "gs://dataflow-staging-us-central1-692943280739/staging" -> (known after apply)
      ~ state                        = "JOB_STATE_RUNNING" -> (known after apply)
      ~ temp_location                = "gs://prod-im-data-imx-resource/ethereum-etl/zkevm-imtbl-mainnet-streaming/temp" -> (known after apply)
      ~ type                         = "JOB_TYPE_STREAMING" -> (known after apply)
        # (6 unchanged attributes hidden)
    }

  # google_dataflow_flex_template_job.zkevm_imtbl_testnet_job will be created
  + resource "google_dataflow_flex_template_job" "zkevm_imtbl_testnet_job" {
      + additional_experiments       = (known after apply)
      + container_spec_gcs_path      = "gs://prod-im-data-imx-resource/dataflow-templates/blockchain-etl-dataflow/metadata.json"
      + id                           = (known after apply)
      + job_id                       = (known after apply)
      + name                         = (known after apply)
      + on_delete                    = "drain"
      + parameters                   = {
          + "allowedTimestampSkewSeconds" = "5184000"
          + "chainConfigFile"             = "/template/blockchain_zkevm_imtbl_testnet_prod.json"
          + "diskSizeGb"                  = "30"
          + "enableStreamingEngine"       = "true"
          + "gcpTempLocation"             = "gs://prod-im-data-imx-resource/ethereum-etl/zkevm-imtbl-testnet-streaming/temp"
          + "maxNumWorkers"               = "1"
          + "project"                     = "prod-im-data"
          + "region"                      = "us-central1"
          + "runner"                      = "DataflowRunner"
          + "tempLocation"                = "gs://prod-im-data-imx-resource/ethereum-etl/zkevm-imtbl-testnet-streaming/temp"
          + "workerMachineType"           = "n1-standard-1"
          + "zone"                        = "us-central1-a"
        }
      + project                      = "prod-im-data"
      + region                       = "us-central1"
      + service_account_email        = (known after apply)
      + skip_wait_on_job_termination = true
      + staging_location             = (known after apply)
      + state                        = (known after apply)
      + temp_location                = (known after apply)
      + type                         = (known after apply)
    }

Plan: 2 to add, 0 to change, 2 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Pusher: @jasadams-imtbl, Action: pull_request, Workflow: Deploy blockchain etl datafow into GCP

[github-actions]

Terraform Initialization ⚙️success

Terraform Validation 🤖success

Terraform Plan 📖success

Show Plan

google_dataflow_flex_template_job.flex_template_job: Refreshing state... [id=2023-12-11_15_36_23-15159653732800287599]
google_storage_bucket_object.blockchain_etl_dataflow: Refreshing state... [id=prod-im-data-imx-resource-dataflow-templates/blockchain-etl-dataflow/metadata.json]
google_dataflow_flex_template_job.zkevm_imtbl_mainnet_job: Refreshing state... [id=2023-12-11_15_36_17-13269957987267613837]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  - destroy
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # google_dataflow_flex_template_job.flex_template_job will be destroyed
  # (because google_dataflow_flex_template_job.flex_template_job is not in configuration)
  - resource "google_dataflow_flex_template_job" "flex_template_job" {
      - additional_experiments       = [] -> null
      - container_spec_gcs_path      = "gs://prod-im-data-imx-resource/dataflow-templates/blockchain-etl-dataflow/metadata.json" -> null
      - id                           = "2023-12-11_15_36_23-15159653732800287599" -> null
      - job_id                       = "2023-12-11_15_36_23-15159653732800287599" -> null
      - labels                       = {} -> null
      - machine_type                 = "n1-standard-1" -> null
      - max_workers                  = 0 -> null
      - name                         = "zkevm-imtbl-testnet-etl-dataflow-20231211-233623" -> null
      - num_workers                  = 0 -> null
      - on_delete                    = "drain" -> null
      - parameters                   = {
          - "allowedTimestampSkewSeconds" = "5184000"
          - "chainConfigFile"             = "/template/blockchain_zkevm_imtbl_testnet_prod.json"
          - "diskSizeGb"                  = "30"
          - "enableStreamingEngine"       = "true"
          - "gcpTempLocation"             = "gs://prod-im-data-imx-resource/ethereum-etl/temp"
          - "maxNumWorkers"               = "1"
          - "project"                     = "prod-im-data"
          - "region"                      = "us-central1"
          - "runner"                      = "DataflowRunner"
          - "tempLocation"                = "gs://prod-im-data-imx-resource/ethereum-etl/temp"
          - "workerMachineType"           = "n1-standard-1"
          - "zone"                        = "us-central1-a"
        } -> null
      - project                      = "prod-im-data" -> null
      - region                       = "us-central1" -> null
      - service_account_email        = "[email protected]" -> null
      - skip_wait_on_job_termination = true -> null
      - staging_location             = "gs://dataflow-staging-us-central1-692943280739/staging" -> null
      - state                        = "JOB_STATE_RUNNING" -> null
      - temp_location                = "gs://prod-im-data-imx-resource/ethereum-etl/temp" -> null
      - type                         = "JOB_TYPE_STREAMING" -> null
    }

  # google_dataflow_flex_template_job.zkevm_imtbl_mainnet_job must be replaced
-/+ resource "google_dataflow_flex_template_job" "zkevm_imtbl_mainnet_job" {
      ~ additional_experiments       = [] -> (known after apply)
      ~ id                           = "2023-12-11_15_36_17-13269957987267613837" -> (known after apply)
      ~ job_id                       = "2023-12-11_15_36_17-13269957987267613837" -> (known after apply)
      - labels                       = {} -> null
      - machine_type                 = "n1-standard-1" -> null
      - max_workers                  = 0 -> null
      ~ name                         = "zkevm-imtbl-mainnet-etl-dataflow-20231211-233616" # forces replacement -> (known after apply) # forces replacement
      - num_workers                  = 0 -> null
      ~ service_account_email        = "[email protected]" -> (known after apply)
      ~ staging_location             = "gs://dataflow-staging-us-central1-692943280739/staging" -> (known after apply)
      ~ state                        = "JOB_STATE_RUNNING" -> (known after apply)
      ~ temp_location                = "gs://prod-im-data-imx-resource/ethereum-etl/zkevm-imtbl-mainnet-streaming/temp" -> (known after apply)
      ~ type                         = "JOB_TYPE_STREAMING" -> (known after apply)
        # (6 unchanged attributes hidden)
    }

  # google_dataflow_flex_template_job.zkevm_imtbl_testnet_job will be created
  + resource "google_dataflow_flex_template_job" "zkevm_imtbl_testnet_job" {
      + additional_experiments       = (known after apply)
      + container_spec_gcs_path      = "gs://prod-im-data-imx-resource/dataflow-templates/blockchain-etl-dataflow/metadata.json"
      + id                           = (known after apply)
      + job_id                       = (known after apply)
      + name                         = (known after apply)
      + on_delete                    = "drain"
      + parameters                   = {
          + "allowedTimestampSkewSeconds" = "5184000"
          + "chainConfigFile"             = "/template/blockchain_zkevm_imtbl_testnet_prod.json"
          + "diskSizeGb"                  = "30"
          + "enableStreamingEngine"       = "true"
          + "gcpTempLocation"             = "gs://prod-im-data-imx-resource/ethereum-etl/zkevm-imtbl-testnet-streaming/temp"
          + "maxNumWorkers"               = "1"
          + "project"                     = "prod-im-data"
          + "region"                      = "us-central1"
          + "runner"                      = "DataflowRunner"
          + "tempLocation"                = "gs://prod-im-data-imx-resource/ethereum-etl/zkevm-imtbl-testnet-streaming/temp"
          + "workerMachineType"           = "n1-standard-1"
          + "zone"                        = "us-central1-a"
        }
      + project                      = "prod-im-data"
      + region                       = "us-central1"
      + service_account_email        = (known after apply)
      + skip_wait_on_job_termination = true
      + staging_location             = (known after apply)
      + state                        = (known after apply)
      + temp_location                = (known after apply)
      + type                         = (known after apply)
    }

Plan: 2 to add, 0 to change, 2 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.

Pusher: @jasadams-imtbl, Action: pull_request, Workflow: Deploy blockchain etl datafow into GCP