bug: the actor information has not been broadcasted to the affected CNs

[yezizp2012]

Describe the bug

ERROR:  Failed to run the query

Caused by these errors (recent errors listed first):
1: gRPC request to meta service failed: Internal error
2: gRPC request to stream service failed: Internal error
3: actor 105 not found in info table

Reported in one of our customer and discovered by @shanicky , this is a corner for creating streaming jobs. It only appears under the following three situations:

1. the streaming job is create sink into table.
2. the sink is performing temporary joins on multiple tables.
3. The set of CNs that all joined tables distributed on is a subset of the set of CNs that the target table distributed on. In other words, some actors in the target table are distributed on CNs that are not present in the join tables.

Currently we only broadcast the info of new actors and all upstream actors to compute nodes that only have some building actors during creation of streaming job.

risingwave/src/meta/src/stream/stream_manager.rs

Lines 400 to 409 in e05015a

	let worker_node = building_locations.worker_locations.get(worker_id).unwrap();
	let actor_infos_to_broadcast = &actor_infos_to_broadcast;
	async move {
	let client = self.env.stream_client_pool().get(worker_node).await?;
	client
	.broadcast_actor_info_table(BroadcastActorInfoTableRequest {
	info: actor_infos_to_broadcast.clone(),
	})
	.await?;

It works for streaming jobs except for sink into table, because we broadcast actor infos of the sink and new target table jobs separately. Under the situation mentioned above, Some actors of the sink on some CNs do not know the information of the upstream sink actors when building.

risingwave/src/meta/src/stream/stream_manager.rs

Lines 493 to 505 in e05015a

	self.build_actors(&table_fragments, &building_locations, &existing_locations)
	.await?;
	if let Some((_, context, table_fragments)) = replace_table_job_info {
	let MetadataManager::V1(mgr) = &self.metadata_manager else {
	unimplemented!("support create sink into table in v2");
	};
	self.build_actors(
	&table_fragments,
	&context.building_locations,
	&context.existing_locations,
	)
	.await?;

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

[yezizp2012]

Fixed by #18270 .