Failing test(s): TestAccDataflowJob*

[melinath]

Impacted tests

• TestAccDataflowJob_basic
• TestAccDataflowJob_withProviderDefaultLabels
• TestAccDataflowJob_withKmsKey
• TestAccDataflowJobSkipWait_basic

Affected Resource(s)

• google_dataflow_job

Failure rates

• 100% since 2024-03-20

Message(s)

vcr_utils.go:152: Step 1/2 error: Check failed: Check 1/1 error: could not confirm Dataflow Job "2024-05-09_00_30_15-8505904879367105719" exists: googleapi: Error 404: (46a5a60f263a7f99): Information about job 2024-05-09_00_30_15-8505904879367105719 could not be found in our system. Please double check that the API being used is projects.locations.jobs.get (https://cloud.google.com/dataflow/docs/reference/rest/v1b3/projects.locations.jobs/get). If the API being used is projects.locations.jobs.get, please double check the id (2024-05-09_00_30_15-8505904879367105719) is correct. If it is please contact customer support., notFound

 Error: Cannot import non-existent remote object
        While attempting to import an existing object to
        "google_dataflow_job.big_data", the provider detected that no object exists
        with the given id. Only pre-existing objects can be imported; check that the
        id is correct and that it is associated with the provider's configured region
        or endpoint, or use "terraform apply" to create a new remote object for this
        resource.

Nightly build test history

• https://hashicorp.teamcity.com/test/-4521318455747298733?currentProjectId=TerraformProviders_GoogleCloud_GOOGLE_NIGHTLYTESTS&expandTestHistoryChartSection=true

b/339718340

[melinath]

@damondouglas FYI

[shuyama1]

I did a quick investigation on these failures, I believe the issue is we're not able to create zonal resource but always regional resource even if "zone" is explicitly set in this case.

terraform-provider-google/google/services/dataflow/resource_dataflow_job.go

Lines 571 to 574 in b528f87

	if region == "" {
	return config.NewDataflowClient(userAgent).Projects.Templates.Create(project, request).Do()
	}
	return config.NewDataflowClient(userAgent).Projects.Locations.Templates.Create(project, region, request).Do()

terraform-provider-google/google/tpgresource/field_helpers.go

Line 466 in b528f87

return GetRegionFromZone(zone), nil

[damondouglas]

Now that I figured out how to troubleshoot these tests, I'll apply this method to the all the Dataflow-related tests and apply PRs fixes where relevant.

[damondouglas]

Created GoogleCloudPlatform/magic-modules#12798