Patch existing resources by matching labels, not resource template name

[negz]

One neat thing about function-patch-and-transform is that you can patch composed resources that are produced by another Function, earlier in the pipeline. For example here we P&T some resources produced by function-go-templates:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: dynamo-with-bucket
spec:
  compositeTypeRef:
    apiVersion: database.example.com/v1alpha1
    kind: NoSQL
  # This pipeliene renders some Go templates, then passes them to P&T
  pipeline:
  - step: render-go-templates
    functionRef:
      name: function-go-templates
    input: {} # Omitted for brevity :)
  - step: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:
        # Notice that my-cool-bucket doesn't have a base template. As long as
        # the render-go-templates step above rendered a composed resource with
        # this name, this Function will patch it.
      - name: my-cool-bucket
        patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "location"
          toFieldPath: "spec.forProvider.region"
          transforms:
          - type: map
            map: 
              EU: "eu-north-1"
              US: "us-east-2"

One shortcoming with this is that you have to know the names (as in composition template names, not object metadata names) of the composed resources some other function produced in order to patch them.

@vfarcic noted that it would be useful to be able to patch resources without knowing their name in advance. Consider for example a Function that produced an arbitrary number of resources, where the number of resources was derived from an XR field. It wouldn't be possible to know what (or more specifically how many) names that Function would produce.

I think this would be pretty easy to implement if we can find a good API to describe it.

[negz]

Perhaps something like this could work:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: dynamo-with-bucket
spec:
  compositeTypeRef:
    apiVersion: database.example.com/v1alpha1
    kind: NoSQL
  # This pipeliene renders some Go templates, then passes them to P&T
  pipeline:
  - step: render-go-templates
    functionRef:
      name: function-go-templates
    input: {} # Omitted for brevity :)
  - step: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      # This can be specified instead of or in addition to the
      # resources array. It doesn't have base templates. Instead it
      # only matches existing resources.
      matchResources:
      # Do we need a 'name' field? A map key would be helpful, but
      # name could easily be misunderstood to mean resource name.
      - match:
          apiVersion: example.org/v1
          kind: SomeComposedResource
          labels:
            # This isn't a special label, just a suggested convention.
            crossplane.io/composition-resource-type: patch-me
        patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "location"
          toFieldPath: "spec.forProvider.region"
          transforms:
          - type: map
            map: 
              EU: "eu-north-1"
              US: "us-east-2"

[negz]

I personally kinda prefer this API that extends the existing resources array:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: dynamo-with-bucket
spec:
  compositeTypeRef:
    apiVersion: database.example.com/v1alpha1
    kind: NoSQL
  # This pipeliene renders some Go templates, then passes them to P&T
  pipeline:
  - step: render-go-templates
    functionRef:
      name: function-go-templates
    input: {} # Omitted for brevity :)
  - step: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:
      - name: match-some-existing-resources
        # What would the default type be? The default behaviour is
        # currently to either render the base template, or if there is
        # none to try patch a resource with the same name.
        type: MatchResources
        match:
          apiVersion: example.org/v1
          kind: SomeComposedResource
          labels:
            # This isn't a special label, just a suggested convention.
            crossplane.io/composition-resource-type: patch-me
        patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "location"
          toFieldPath: "spec.forProvider.region"
          transforms:
          - type: map
            map: 
              EU: "eu-north-1"
              US: "us-east-2"

[stevendborrelli]

It makes sense in the case of generated resources higher up in the pipeline. Would it make sense to also support something like matchControllerRef?

I wonder if it makes sense to have an alpha channel for f-p-and-t where users can experiment.

[negz]

I wonder if it makes sense to have an alpha channel for f-p-and-t where users can experiment.

Yeah that sounds like a good idea.

[negz]

A good point was raised on Slack - how would this work with a ToCompositeFieldPath patch? It inherently becomes a many-to-one patch if multiple composed resources match.

[tmarton]

Hey folks, is there a possibility that this feature will be implemented? It would be great to have it.

[negz]

I'm not planning to work on it, but if someone wants to give it a shot I can review.