Use different labels other than job and instance when building the dev dashboards

[arkbriar]

Is your feature request related to a problem? Please describe.

According to the Prometheus doc here, job and instance are two labels attached by the collector automatically:

• job: The configured job name that the target belongs to.
• instance: The : part of the target's URL that was scraped.

It works good for local dashboards but it doesn't for the Kubernetes'. In Kubernetes, there's only one overall job for all RisingWave processes, no matter the roles. Thus the job loses its meaning of which role or component an instance belongs to. Similarly, the instance can not be treated as a typical identifier of a process although it is a unique identifier at a certain point. The reason is Kubernetes will reclaim and reuse IPs. Pod name is a better one in this case.

Describe the solution you'd like

There are two additional labels that can be used for the same purposes in Kubernetes (at least in the operator deployment):

• risingwave_component, to replace the job
• pod, to replace the instance

It would be great if we could build the dashboard for Kubernetes with these two labels instead. It may require parameterization for these two labels and substitution everywhere they occur.

Describe alternatives you've considered

1. Overwrite and tweak these two labels in Kubernetes. However, though these labels are not widely used, it's not a good solution to tweak the meanings.
2. Attach the two labels locally. But pod is meaningless outside k8s.

Additional context

No response