diff --git a/teps/0140-producing-results-in-matrix.md b/teps/0140-producing-results-in-matrix.md new file mode 100644 index 000000000..d67eea211 --- /dev/null +++ b/teps/0140-producing-results-in-matrix.md @@ -0,0 +1,525 @@ +--- +status: 'implementable' +title: Producing Results in Matrix +creation-date: '2023-07-31' +last-updated: '2023-08-07' +authors: +- '@emmamunley' +- '@pritidesai' +- '@jerop' +see-also: +- TEP-0075 +- TEP-0076 +- TEP-0090 +- TEP-0118 +--- + +# TEP-0140: Producing Results in Matrix + + +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) + - [Background](#background) + - [String Results](#string-results) + - [Array Results](#array-results) + - [Object Results](#object-results) + - [Requirements](#requirements) + - [Use Cases](#use-cases) + - [1. Building Images](#1-building-images) + - [2. Platforms and Browsers](#2-platforms-and-browsers) +- [Proposal](#proposal) +- [Design Details](#design-details) + - [Results Cache](#results-cache) + - [Context Variables to Access Aggregated Results Length](#context-variables-to-access-aggregated-results-length) + - [Failed Task With Results](#failed-task-with-results) +- [Examples](#examples) + - [1. Building Images](#1-building-images-1) + - [2. Platforms and Browsers](#2-platforms-and-browsers-1) +- [Design Evaluation](#design-evaluation) +- [References](#references) + + +## Summary + +Today, we do not support producing `Results` from `PipelineTasks` that have been fanned out using `Matrix`. This TEP aims to enable producing `Results` from Matrixed `PipelineTasks` and would enable users to: +- Declare a Matrixed `TaskRun` that emits `Results` of type `String` that are fanned out over multiple TaskRuns and aggregated into an array of `Results` that can then be consumed by another `PipelineTask` +- Consume an entire `Array` of `Results` produced by a referenced Matrixed `PipelineTask` +- Declare a Matrixed `TaskRun` that emits `Results` of type `Array` or `Object` as long as those `Results` are not consumed by another `PipelineTask` + +In summary, we propose, each fanned out `TaskRun` that produces `Results` of type `string` will be aggregated into an `Array` of `Results` during reconciliation, in which the entire aggregated `Array` of `Results` can be consumed by another `PipelineTask` using the star notion [*]. + +We will not limit producing `Results` of type `Array` or `Object` from a Matrixed `PipelineTask`. However, we will validate that any `Results` produced from a fanned out `PipelineTask` can only be emitted as a `String` type *IF* that `Result` is also being consumed by another `PipelineTask`. This is because we currently don't support arrays of type `Array` or arrays of type `Object`. + +## Motivation + +We currently support emitting `Results` from non-matrixed `PipelineTasks` which can be easily referenced by the `Result` name of the `PipelineTask`. Before `Matrix` was introduced, there was only one `TaskRun` from a given `PipelineTask` so the variable has this constraint: `tasks..results..` +In the example below, we have a PipelineTask “get-platforms” which produces a `Results` “platforms”. It will execute in a TaskRun named “pr-get-platforms” and its `Results` can be accessed via the variable `$(tasks.get-platforms.results.platforms[*])`. + +```yaml +tasks: +... +- name: get-platforms + taskSpec: + results: + - name: platforms + type: array + steps: + - name: write-array + image: bash:latest + script: | + #!/usr/bin/env bash + echo -n "[\"linux\",\"mac\",\"windows\"]" | tee $(results.platforms.path) +``` + +TaskRun Created +``` +pr-get-platforms - [\"linux\",\"mac\",\"windows\"] +``` + +A `Matrix` when fanned out creates multiple `TaskRuns` so for a `Matrix` to emit results, it is unclear how each `Results` would map back to the original matrixed `PipelineTask`. In the example below, the following `Matrix` will produce 9 `TaskRuns` that will each produce a string `Results` named “report-url” for each combination of platform and browser. Since the order of the `TaskRuns` is deterministic, it will always produce `Results` in the same order. +```yaml +Tasks: +... +kind: Task +metadata: + name: taskwithresults +spec: + params: + - name: platform + default: "" + - name: browser + default: "" + results: + - name: report-url + type: string + steps: + - name: produce-report-url + image: alpine + script: | + #!/usr/bin/env bash + echo "https://api.example/get-report/$(params.platform)-$(params.browser)" | tee $(results.report-url.path) +--- +kind: PipelineRun +metadata: + generateName: platforms-with-results +spec: + serviceAccountName: "default" + pipelineSpec: + tasks: + - name: matrix-emitting-results + matrix: + params: + - name: platform + value: + - linux + - mac + - windows + - name: browser + value: + - chrome + - safari + - firefox + taskRef: + name: taskwithresults + kind: Task +``` + +TaskRuns Created - “report-url” Result Produced +``` +pr-produce-platforms-and-browsers-0 - “path/to/report/linux-chrome” +pr-produce-platforms-and-browsers-1 - “path/to/report/mac-chrome” +pr-produce-platforms-and-browsers-2 - “path/to/report/windows-chrome” +pr-produce-platforms-and-browsers-3 - “path/to/report/linux-safari” +pr-produce-platforms-and-browsers-4 - “path/to/report/mac-safari” +pr-produce-platforms-and-browsers-5 - “path/to/report/windows-safari” +pr-produce-platforms-and-browsers-6 - “path/to/report/linux-firefox” +pr-produce-platforms-and-browsers-7 - “path/to/report/mac-firefox” +pr-produce-platforms-and-browsers-8 - “path/to/report/windows-firefox” +``` + +### Goals + +The main goal of this TEP is to enable executing a `Matrixed` `PipelineTask` that can emit `Results` and support consuming all of the `Results` that were produced in another `PipelineTask` as long as the type of `Result` emitted was `String`. + +### Non-Goals + +The following are out of scope for this TEP: + +1. Support a matrixed `PipelineTask` that produces `Results` of type `Array` or `Object` that are then consumed by another `PipelineTask`. + - A matrixed `PipelineTask` can support `Results` of any type as long as they aren’t consumed by another `PipelineTask`. However, only matrixed `PipelineTask` that produces `Results` of type `String` can be consumed by another `PipelineTask`. +2. Support consuming a specific instance or combination(s) of `Results` produced by a fanned out `PipelineTask`. + - At this time, we propose only supporting whole `Array` `Result` replacements from a Matrixed `PipelineTask` + +### Background + +Consuming `Results` from previous `TaskRuns` or `Runs` in a `Matrix`, which would dynamically generate `TaskRuns` from the fanned out `Results`, is currently supported for string `Results`, array `Results`, and object `Results`. Note that the underlying `Results` type in each must be string. + +#### String Results +String `Results` is a stable feature and is referred to as `$(tasks..results.)` and can be passed into the `Matrix` from previous `TaskRuns`. +```yaml +tasks: +... +- name: task-1 + taskRef: + name: task-1 + matrix: + - name: values + value: + - $(tasks.task-1.results.foo) # string +``` +#### Array Results +Array `Results` is a beta feature and is referred to as `$(tasks..results.[*])`. String replacements from arrays are supported through Array indexing and are referred to as `$(tasks..results.[i])` where i is the index. Array `Results` from previous `TaskRuns` can be passed into the Matrix: +```yaml +tasks: +... +- name: task-2 + taskRef: + name: task-2 + matrix: + - name: values + value: $(tasks.task-4.results.bar[*]) # array +``` + +#### Object Results +Object `Results` is a beta feature and is referred to as `$(tasks..results..[*])`. String replacements from objects are supported and are referred to as $(tasks..results..key) where key is the object key. Strings from Object `Results` from previous `TaskRuns` can be passed into the `Matrix`: +tasks: + +```yaml +... +- name: task-3 + taskRef: + name: task-3 + matrix: + - name: values + value: $(tasks.task-4.results.rad.key) # string replacement from object result +``` + + +### Requirements + +1. A `matrix` of `Parameters` can only produce `Results` of type `string` if the `Results` are being consumed by another `PipelineTask`. +2. A `PipelineTask` that consumes `Results` produced by a `Matrix` must consume the entire aggregated `Array` of `Results` produced during fanning out. + +### Use Cases + +#### 1. Building Images + +In TEP-0090: Matrix, we described use cases for `Matrix` that involve building images - kaniko and monorepos. When the fanned out PipelineTasks produce the images as `Results`, the users would need to pass them to subsequent `PipelineTasks` to scan and deploy the images, among other operations. +To be specific, the kaniko use case uses the kaniko `Task` from the Tekton Catalog which produces an IMAGE-DIGEST `Results`. + +In TEP-0118, we expanded this with explicit combinations where the user needs to specify explicit +mapping between `IMAGE` and `DOCKERFILE`. + +```yaml + - IMAGE: "image-1" + DOCKERFILE: "path/to/Dockerfile1" + + - IMAGE: "image-2" + DOCKERFILE: "path/to/Dockerfile2" + + - IMAGE: "image-3" + DOCKERFILE: "path/to/Dockerfile3" +``` + +When the `Results` has a Matrix, it will be fanned out to multiple TaskRuns to execute that Task - each of which will produce an IMAGE-DIGEST `Results`. A user may want to use this IMAGE-DIGEST to deploy images. + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: matrix-building-images +spec: + tasks: + - ... + - name: matrix-emitting-results + matrix: + include: + - name: build-1 + params: + - name: IMAGE + value: image-1 + - name: DOCKERFILE + value: path/to/Dockerfile1 + - name: build-2 + params: + - name: IMAGE + value: image-2 + - name: DOCKERFILE + value: path/to/Dockerfile2 + - name: build-3 + params: + - name: IMAGE + value: image-3 + - name: DOCKERFILE + value: path/to/Dockerfile3 + taskSpec: + params: + - name: IMAGE + - name: DIGEST + default: "" + results: + - name: IMAGE-DIGEST + steps: + - name: produce-image-digest + image: bash:latest + script: | + #!/usr/bin/env bash + echo "Building image for $(params.IMAGE)" + echo -n "$(params.DIGEST)" | sha256sum | tee $(results.IMAGE-DIGEST.path) +``` + + +#### 2. Platforms and Browsers + +As a `Pipeline` author, I need to run tests on a combination of platforms and browsers. This will fan out into 9 different `TaskRuns` that will produce a `Result` “report-url” for each combination which can be used in a subsequent `PipelineTask` to fetch the report for each platform-browser combination. + +```text +# platforms +linux +windows +mac + +# browsers +chrome +firefox +safari +``` + + +``` + clone + | + v + -------------------------------------------------------------------------------------------------------------------------- + | | | | | | | | | + v v v v v v v v v +linux-chrome linux-firefox linux-safari windows-chrome windows-firefox windows-safari mac-chrome mac-firefox mac-safari + | | | | | | | | | + v v v v v v v v v +report-url-0 report-url-1 report-url-2 report-url-3 report-url-4 report-url-5 report-url-6 report-url-7 report-url-8 +``` + +```yaml +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: matrix-testing-platform-and-browsers +spec: + tasks: + - name: test-platforms-and-browsers + matrix: + params: + - name: platform + value: + - linux + - mac + - windows + - name: browser + value: + - chrome + - safari + - firefox + taskSpec: + params: + - name: platform + type: string + - name: browser + type: string + results: + - name: report-url + type: string + steps: + - name: produce-report-url + image: alpine + script: | + echo "Running tests on $(params.platform)-$(params.browser)" + echo -n "https://api.example/get-report/$(params.platform)-$(params.browser)" | tee $(results.report-url.path) +``` + +## Proposal + +To support enabling a Matrixed `PipelineTask` to produce `Results`, we propose, each fanned out `TaskRun` that produces `Results` of type `string` will be aggregated into an `Array` of `Results` during reconciliation, in which the whole `Array` of `Results` can be consumed by another `PipelineTask` using the star notion [*]. + +## Design Details + +### Results Cache + +With this proposal, we add a `ResultsCache` to `ResolvedPipelineTask` that enables caching of `Results` from a matrixed `PipelineTask` in order to prevent resolving the result references for the referenced `Matrixed PipelineTask` on every single reconcile loop. +```go +t.ResultsCache[result.Name] = []string{result.Value.StringVal} +``` +```go +type ResolvedPipelineTask struct { + TaskRunNames []string + TaskRuns []*v1.TaskRun + CustomTask bool + CustomRunNames []string + CustomRuns []*v1beta1.CustomRun + PipelineTask *v1.PipelineTask + ResolvedTask *resources.ResolvedTask + ResultsCache map[string][]string +} +``` + +### Context Variables to Access Aggregated Results Length + +We propose enabling `context variables` to allow users to access the length of the array of aggregated results that were actually produced using the syntax: `context..matrix.`. This will allow users to loop over the results produced. + +### Failed Task With Results + + + + + +## Examples + +### 1. Building Images +In the example below, a user is able to produce and deploy images using specific combinations of images and dockerfiles using the task ”matrix-emitting-results” which produces the result ”IMAGE-DIGEST”, which is used by another matrixed PipelineTask “task-deploy-images” to deploy the images. +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + generateName: matrix-emitting-results +spec: + serviceAccountName: "default" + pipelineSpec: + tasks: + - name: matrix-emitting-results + matrix: + include: + - name: build-1 + params: + - name: IMAGE + value: image-1 + - name: DOCKERFILE + value: path/to/Dockerfile1 + - name: build-2 + params: + - name: IMAGE + value: image-2 + - name: DOCKERFILE + value: path/to/Dockerfile2 + - name: build-3 + params: + - name: IMAGE + value: image-3 + - name: DOCKERFILE + value: path/to/Dockerfile3 + taskSpec: + params: + - name: IMAGE + - name: DIGEST + default: "" + results: + - name: IMAGE-DIGEST + steps: + - name: produce-image-digest + image: bash:latest + script: | + echo "Building image for $(params.IMAGE)" + echo -n "$(params.IMAGE)" | sha256sum | tee $(results.IMAGE-DIGEST.path) + - name: task-deploy-images + params: + - name: DIGEST + value: $(tasks.matrix-emitting-results.results.IMAGE-DIGEST[*]) + taskSpec: + params: + - name: DIGEST + type: string + steps: + - name: echo + image: alpine + script: | + echo "deploying image: $(params.DIGEST)" +``` + + +### 2. Platforms and Browsers +In the example below, a user wants to run tests on a combination of different platforms and browsers and then fetch the reports for all combinations +```yaml +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + generateName: platforms-with-results +spec: + serviceAccountName: "default" + pipelineSpec: + tasks: + - name: matrix-emitting-results + matrix: + params: + - name: platform + value: + - linux + - mac + - windows + - name: browser + value: + - chrome + - safari + - firefox + taskSpec: + params: + - name: platform + default: "" + - name: browser + default: "" + results: + - name: report-url + type: string + steps: + - name: produce-report-url + image: alpine + script: | + echo "Running tests on $(params.platform)-$(params.browser)" + echo -n "https://api.example/get-report/$(params.platform)-$(params.browser)" | tee $(results.report-url.path) + - name: task-consuming-results + taskRef: + name: stringtask + kind: Task + params: + - name: url + Value: $(tasks.matrix-emitting-results.results.report-url[*]) + taskSpec: + params: + - name: url + type: string + steps: + - name: echo + image: alpine + script: | + for arg in "$@"; do + echo "Arg: $arg" + done +``` + +## Design Evaluation + +* [Reusability](https://github.com/tektoncd/community/blob/main/design-principles.md#reusability): + * Pro: This will improve the reusability of Tekton components by enabling the scenario of a matrixed + `PipelineTask` aggregating each of the `Results` produced by the fanned `TaskRuns` into an array + `Result` and then a Pipeline being able to loop over those values for subsequent Tasks +* [Simplicity](https://github.com/tektoncd/community/blob/main/design-principles.md#simplicity) + * Pro: This proposal reuses the existing array or string concept for params + * Pro: This proposal continues [the precedent of using JSONPath syntax in variable replacement](https://github.com/tektoncd/pipeline/issues/1393#issuecomment-561476075) +* [Flexibility](https://github.com/tektoncd/community/blob/main/design-principles.md#flexibility) + * Con: Although there is a precedent for including JSONPath syntax, this is a step toward including more hard coded + expression syntax in the Pipelines API (without the ability to choose other language options) +* [Conformance](https://github.com/tektoncd/community/blob/main/design-principles.md#conformance) + * Supporting array results and indexing syntax would be included in the conformance surface + + +## References +* Tekton Enhancement Proposals: + * [TEP-0075:Object Parameter and Results][tep-0075] + * [TEP-0076:Object Parameter and Results][tep-0076] + * [TEP-0090: Matrix][tep-0090] + * [TEP-0118: Matrix with Explicit Combinations][tep-0118] + +[tep-0075]: ./0075-object-param-and-result-types.md +[tep-0076]: ./0076-array-result-types.md +[tep-0090]: ./0090-matrix.md +[tep-0118]: ./0118-matrix-with-explicit-combinations-of-parameters.md +[support-failed-taskrun]: https://github.com/tektoncd/pipeline/pull/6510 diff --git a/teps/README.md b/teps/README.md index 7b4cf5c92..b81f4f5af 100644 --- a/teps/README.md +++ b/teps/README.md @@ -128,3 +128,4 @@ This is the complete list of Tekton TEPs: |[TEP-0136](0136-capture-traces-for-task-pod-events.md) | Capture traces for task pod events | proposed | 2023-06-14 | |[TEP-0137](0137-cloudevents-controller.md) | CloudEvents controller | proposed | 2023-06-19 | |[TEP-0138](0138-decouple-api-and-feature-versioning.md) | Decouple api and feature versioning | proposed | 2023-07-27 | +|[TEP-0140](0140-producing-results-in-matrix.md) | Producing Results in Matrix | implementable | 2023-08-07 |