Support cluster and replica labels for remote write deduplication

[balbirthomas]

Bug Description

In order to ensure high availability when using Prometheus remote write multiple remote write agents (Grafana Agent or Prometheus) need to scrape the same targets. On the other end of the remote write relation the metrics aggregating charm (for example a Mimir charm) needs to be able to de-duplicate these redundant metrics. In the case of Mimir such de-duplication requires cluster and __replica__ labels to be associated with each time series. The cluster label must identify a group of related remote write consumer agents that scrape the same set of targets. The __replica__ label must identify an individual consumer in this cluster. Both these labels may need to be specified as global: and external_labels: since they must be associated with each metric time series.

The names of the two labels are only suggested. The suggested names are defaults in Grafana Mimir. Changing these two label names for the remote write consumer also requires configuring Grafana Mimir with the new set of label names to be used for de-duplication.

To Reproduce

NA

Environment

NA

Relevant log output

NA

Additional context

NA