Add support for consuming whole array results in matrix #6603

EmmaMunley · 2023-04-28T21:53:58Z

Changes

This PR adds support for referencing whole array results produced by another PipelineTask as a matrix parameter value.

This feature request is necessary before promoting Matrix to beta.
For me details please see Issue #6110.

This closes issue #6602.

Co-authored-by: Yongxuan Zhang [email protected]
/kind feature

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
Has Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

TEP-0090: Pipeline Tasks may now reference whole array results in a Matrix. See https://github.com/tektoncd/pipeline/blob/09d422cff057f67170b4c2f76097ac6ffded33ef/docs/matrix.md?specifying-results-in-a-matrix#specifying-results-in-a-matrix docs for more information.

tekton-robot · 2023-04-28T21:59:47Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.2%	0.0
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	95.8%	-0.6

tekton-robot · 2023-04-28T22:01:01Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.2%	0.0
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	95.8%	-0.6

jerop · 2023-05-01T15:42:28Z

/assign

lbernick

nice!!
Suggested release notes: "TEP-0090: Pipeline Tasks may now reference whole array results in a Matrix. See [link to docs] for more information."

docs/matrix.md

pkg/reconciler/pipelinerun/pipelinerun.go

pkg/apis/pipeline/v1/matrix_types.go

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go

tekton-robot · 2023-05-03T22:44:50Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.3%	0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	96.3%	-0.1

tekton-robot · 2023-05-03T22:45:52Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.3%	0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	96.3%	-0.1

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go

lbernick

LGTM, just have the same question that Jerop does (everything else is minor). Also, the release notes formatting is a bit weird-- can you make sure the doc link is included between the backticks so it gets into the release notes?

pkg/reconciler/pipelinerun/pipelinerun_test.go

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go

pkg/reconciler/pipelinerun/resources/pipelinerunresolution_test.go

tekton-robot · 2023-05-04T15:19:16Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.3%	0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	96.3%	-0.1

tekton-robot · 2023-05-04T15:21:53Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.3%	0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	96.3%	-0.1

EmmaMunley · 2023-05-04T15:26:24Z

/retest

tekton-robot · 2023-05-04T20:32:26Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.3%	0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	96.3%	-0.1

tekton-robot · 2023-05-04T20:34:28Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	89.2%	89.3%	0.1
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	96.4%	96.3%	-0.1

tekton-robot · 2023-05-22T17:43:47Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.7%	99.7%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	99.4%	99.4%	-0.0
pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go	97.0%	93.9%	-3.1

EmmaMunley · 2023-06-21T14:22:39Z

/unhold

tekton-robot · 2023-06-21T14:26:31Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.3%	91.5%	0.2
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	96.2%	3.8

tekton-robot · 2023-06-21T14:29:34Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.3%	91.5%	0.2
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	95.2%	2.9

EmmaMunley · 2023-06-21T14:58:03Z

/retest-required
#6772

pkg/reconciler/pipelinerun/resources/apply.go

pkg/reconciler/pipelinerun/resources/resultrefresolution_test.go

pkg/reconciler/pipelinerun/resources/pipelinerunresolution_test.go

pkg/reconciler/pipelinerun/pipelinerun_test.go

Yongxuanzhang · 2023-06-21T15:42:22Z

pkg/reconciler/pipelinerun/pipelinerun_test.go

@@ -10443,6 +10600,373 @@ spec:
 	}
 }

+func TestReconciler_FailedPipelineTaskMatrixIdxResultsOutOfBounds(t *testing.T) {


Should this test be in this PR? 🤔

Hmm, probably should have been in the last one. Opened a separate PR here just to add the test coverage: #6852

cc: @dibyom

tekton-robot · 2023-06-21T16:03:31Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	96.2%	3.8

tekton-robot · 2023-06-21T16:03:41Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.3%	91.5%	0.2
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	96.2%	3.8

tekton-robot · 2023-06-21T16:05:44Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	95.2%	2.9

dibyom · 2023-06-21T17:21:26Z

pkg/reconciler/pipelinerun/pipelinerun_test.go

+    kind: TaskRun
+    name: pr-echo-platforms-2
+    pipelineTaskName: echo-platforms
+  provenance:


do we really need to compare the provenance/feature flags section here? Or can we ignore it in the comparison? (Otherwise, anytime we update a feature flag value we'd have to update this test making it brittle)

@dibyom Added an ignoreProvenance flag
cc: @Yongxuanzhang

tekton-robot · 2023-06-21T18:06:22Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dibyom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dibyom]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This commit adds support for referencing whole array results produced by another PipelineTask as a matrix parameter value. This feature request is necessary before promoting Matrix to beta. For more details please see issue tektoncd#6110. This closes issue tektoncd#6602. Co-authored-by: Yongxuan Zhang <[email protected]>

Yongxuanzhang

/lgtm
Thanks!! @EmmaMunley

Yongxuanzhang · 2023-06-21T19:18:56Z

/meow

tekton-robot · 2023-06-21T19:18:58Z

@Yongxuanzhang:

In response to this:

/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tekton-robot · 2023-06-21T19:22:11Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	96.2%	3.8

tekton-robot · 2023-06-21T19:33:29Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.1%	99.1%	-0.0
pkg/apis/pipeline/v1beta1/pipeline_validation.go	98.8%	98.8%	-0.0
pkg/reconciler/pipelinerun/resources/resultrefresolution.go	92.4%	96.2%	3.8

tekton-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 28, 2023

tekton-robot requested review from bobcatfish and dibyom April 28, 2023 21:54

tekton-robot assigned jerop May 1, 2023

lbernick reviewed May 1, 2023

View reviewed changes

lbernick self-assigned this May 1, 2023

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch from ecdf383 to 4e707a2 Compare May 3, 2023 22:39

tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 3, 2023

jerop reviewed May 3, 2023

View reviewed changes

pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go Outdated Show resolved Hide resolved

lbernick reviewed May 4, 2023

View reviewed changes

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch from 4e707a2 to 4f1cdbe Compare May 4, 2023 15:13

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch from 4f1cdbe to 7e5f686 Compare May 4, 2023 20:26

lbernick mentioned this pull request May 5, 2023

Clean up pipelinerun resolution code #6628

Closed

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 16, 2023

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch from 7e5f686 to 97eba02 Compare May 22, 2023 17:38

tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 22, 2023

tekton-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 21, 2023

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch from c164f71 to 9bb9226 Compare June 21, 2023 14:21

tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 21, 2023

tekton-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 21, 2023

Yongxuanzhang reviewed Jun 21, 2023

View reviewed changes

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch 2 times, most recently from 5ea6147 to b02bd60 Compare June 21, 2023 15:58

dibyom approved these changes Jun 21, 2023

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 21, 2023

EmmaMunley force-pushed the whole-array-result-replacements-matrix branch from b02bd60 to f0e02ef Compare June 21, 2023 19:15

Yongxuanzhang reviewed Jun 21, 2023

View reviewed changes

tekton-robot assigned Yongxuanzhang Jun 21, 2023

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 21, 2023

tekton-robot merged commit 381a420 into tektoncd:main Jun 21, 2023

EmmaMunley deleted the whole-array-result-replacements-matrix branch June 21, 2023 19:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for consuming whole array results in matrix #6603

Add support for consuming whole array results in matrix #6603

EmmaMunley commented Apr 28, 2023 •

edited

Loading

tekton-robot commented Apr 28, 2023

tekton-robot commented Apr 28, 2023

jerop commented May 1, 2023

lbernick left a comment

tekton-robot commented May 3, 2023

tekton-robot commented May 3, 2023

lbernick left a comment

tekton-robot commented May 4, 2023

tekton-robot commented May 4, 2023

EmmaMunley commented May 4, 2023

tekton-robot commented May 4, 2023

tekton-robot commented May 4, 2023

tekton-robot commented May 22, 2023

EmmaMunley commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

EmmaMunley commented Jun 21, 2023

Yongxuanzhang Jun 21, 2023

EmmaMunley Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

dibyom Jun 21, 2023

EmmaMunley Jun 21, 2023

tekton-robot commented Jun 21, 2023

Yongxuanzhang left a comment

Yongxuanzhang commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

Add support for consuming whole array results in matrix #6603

Add support for consuming whole array results in matrix #6603

Conversation

EmmaMunley commented Apr 28, 2023 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Apr 28, 2023

tekton-robot commented Apr 28, 2023

jerop commented May 1, 2023

lbernick left a comment

Choose a reason for hiding this comment

tekton-robot commented May 3, 2023

tekton-robot commented May 3, 2023

lbernick left a comment

Choose a reason for hiding this comment

tekton-robot commented May 4, 2023

tekton-robot commented May 4, 2023

EmmaMunley commented May 4, 2023

tekton-robot commented May 4, 2023

tekton-robot commented May 4, 2023

tekton-robot commented May 22, 2023

EmmaMunley commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

EmmaMunley commented Jun 21, 2023

Yongxuanzhang Jun 21, 2023

Choose a reason for hiding this comment

EmmaMunley Jun 21, 2023

Choose a reason for hiding this comment

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

dibyom Jun 21, 2023

Choose a reason for hiding this comment

EmmaMunley Jun 21, 2023

Choose a reason for hiding this comment

tekton-robot commented Jun 21, 2023

Yongxuanzhang left a comment

Choose a reason for hiding this comment

Yongxuanzhang commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

tekton-robot commented Jun 21, 2023

EmmaMunley commented Apr 28, 2023 •

edited

Loading