Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEP-0144] Validate PipelineRun for Param Enum #7338

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion config/config-feature-flags.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -127,5 +127,4 @@ data:
# This feature is in preview mode and not implemented yet. Please check #7259 for updates.
enable-step-actions: "false"
# Setting this flag to "true" will enable the built-in param input validation via param enum.
# NOTE (#7270): this feature is still under development and not yet functional.
enable-param-enum: "false"
1 change: 1 addition & 0 deletions docs/additional-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ Features currently in "alpha" are:
| [Coschedule](./affinityassistants.md) | [TEP-0135](https://github.com/tektoncd/community/blob/main/teps/0135-coscheduling-pipelinerun-pods.md) | N/A |`coschedule` |
| [keep pod on cancel](./taskruns.md#cancelling-a-taskrun) | N/A | v0.52 | keep-pod-on-cancel |
| [CEL in WhenExpression](./taskruns.md#cancelling-a-taskrun) | [TEP-0145](https://github.com/tektoncd/community/blob/main/teps/0145-cel-in-whenexpression.md) | N/A | enable-cel-in-whenexpression |
| [Param Enum](./taskruns.md#parameter-enums) | [TEP-0144](https://github.com/tektoncd/community/blob/main/teps/0144-param-enum.md) | N/A | `enable-param-enum` |

### Beta Features

Expand Down
3 changes: 3 additions & 0 deletions docs/pipeline-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1997,6 +1997,9 @@ associated Pipeline is an invalid graph (a.k.a wrong order, cycle, …)</p>
</tr><tr><td><p>&#34;InvalidMatrixParameterTypes&#34;</p></td>
<td><p>ReasonInvalidMatrixParameterTypes indicates a matrix contains invalid parameter types</p>
</td>
</tr><tr><td><p>&#34;InvalidParamValue&#34;</p></td>
<td><p>PipelineRunReasonInvalidParamValue indicates that the PipelineRun Param input value is not allowed.</p>
</td>
</tr><tr><td><p>&#34;InvalidTaskResultReference&#34;</p></td>
<td><p>ReasonInvalidTaskResultReference indicates a task result was declared
but was not initialized by that task</p>
Expand Down
12 changes: 12 additions & 0 deletions docs/pipelineruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,18 @@ case is when your CI system autogenerates `PipelineRuns` and it has `Parameters`
provide to all `PipelineRuns`. Because you can pass in extra `Parameters`, you don't have to
go through the complexity of checking each `Pipeline` and providing only the required params.

#### Parameter Enums

> :seedling: **`enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.

If a `Parameter` is guarded by `Enum` in the `Pipeline`, you can only provide `Parameter` values in the `PipelineRun` that are predefined in the `Param.Enum` in the `Pipeline`. The `PipelineRun` will fail with reason `InvalidParamValue` otherwise.

Tekton will also the validate the `param` values passed to any referenced `Tasks` (via `taskRef`) if `Enum` is specified for the `Task`. The `PipelineRun` will fail with reason `InvalidParamValue` if `Enum` validation is failed for any of the `PipelineTask`.

You can also specify `Enum` in an embedded `Pipeline` in a `PipelineRun`. The same `Param` validation will be executed in this scenario.

See more details in [Param.Enum](./pipelines.md#param-enum).

#### Propagated Parameters

When using an inlined spec, parameters from the parent `PipelineRun` will be
Expand Down
72 changes: 69 additions & 3 deletions docs/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,11 +272,77 @@ spec:
```

#### Param enum
> :seedling: **Specifying `enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.
> :seedling: **`enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.

> :seedling: This feature is WIP and not yet supported/implemented. Documentation to be completed.
Parameter declarations can include `enum` which is a predefine set of valid values that can be accepted by the `Pipeline` `Param`. If a `Param` has both `enum` and default value, the default value must be in the `enum` set. For example, the valid/allowed values for `Param` "message" is bounded to `v1` and `v2`:

Parameter declarations can include `enum` which is a predefine set of valid values that can be accepted by the `Pipeline`.
``` yaml
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: pipeline-param-enum
spec:
params:
- name: message
enum: ["v1", "v2"]
default: "v1"
jerop marked this conversation as resolved.
Show resolved Hide resolved
tasks:
- name: task1
params:
- name: message
value: $(params.message)
steps:
- name: build
image: bash:3.2
script: |
echo "$(params.message)"
```

If the `Param` value passed in by `PipelineRun` is **NOT** in the predefined `enum` list, the `PipelineRun` will fail with reason `InvalidParamValue`.

If a `PipelineTask` references a `Task` with `enum`, the `enums` specified in the Pipeline `spec.params` (pipeline-level `enum`) must be
a **subset** of the `enums` specified in the referenced `Task` (task-level `enum`). Note that an empty pipeline-level `enum` is invalid
in this scenario since an empty `enum` set indicates a "universal set" which allows all possible values. In the below example, the referenced `Task` accepts `v1` and `v2` as valid values, the `Pipeline` further restricts the valid values to `v1`.

``` yaml
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: param-enum-demo
spec:
params:
- name: message
type: string
enum: ["v1", "v2"]
steps:
- name: build
image: bash:latest
script: |
echo "$(params.message)"
```

``` yaml
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: pipeline-param-enum
spec:
params:
- name: message
enum: ["v1"] # note that an empty enum set is invalid
tasks:
- name: task1
params:
- name: message
value: $(params.message)
taskRef:
name: param-enum-demo
```

Tekton validates user-provided values in a `PipelineRun` against the `enum` specified in the `PipelineSpec.params`. Tekton also validates
any resolved `param` value against the `enum` specified in each `PipelineTask` before creating the `TaskRun`.

See usage in this [example](../examples/v1/pipelineruns/alpha/param-enum.yaml)

## Adding `Tasks` to the `Pipeline`

Expand Down
4 changes: 1 addition & 3 deletions docs/taskruns.md
Original file line number Diff line number Diff line change
Expand Up @@ -404,9 +404,7 @@ go through the complexity of checking each `Task` and providing only the require

#### Parameter Enums

> :seedling: **Specifying `enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.

> :seedling: This feature is WIP and not yet supported/implemented. Documentation to be completed.
> :seedling: **`enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.

If a `Parameter` is guarded by `Enum` in the `Task`, you can only provide `Parameter` values in the `TaskRun` that are predefined in the `Param.Enum` in the `Task`. The `TaskRun` will fail with reason `InvalidParamValue` otherwise.

Expand Down
7 changes: 3 additions & 4 deletions docs/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -713,11 +713,9 @@ spec:
```

#### Param enum
> :seedling: **Specifying `enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.
> :seedling: **`enum` is an [alpha](additional-configs.md#alpha-features) feature.** The `enable-param-enum` feature flag must be set to `"true"` to enable this feature.

> :seedling: This feature is WIP and not yet supported/implemented. Documentation to be completed.

Parameter declarations can include `enum` which is a predefine set of valid values that can be accepted by the `Param`. For example, the valid/allowed values for `Param` "message" is bounded to `v1`, `v2` and `v3`:
Parameter declarations can include `enum` which is a predefine set of valid values that can be accepted by the `Param`. If a `Param` has both `enum` and default value, the default value must be in the `enum` set. For example, the valid/allowed values for `Param` "message" is bounded to `v1`, `v2` and `v3`:

``` yaml
apiVersion: tekton.dev/v1
Expand All @@ -729,6 +727,7 @@ spec:
- name: message
type: string
enum: ["v1", "v2", "v3"]
default: "v1"
steps:
- name: build
image: bash:latest
Expand Down
42 changes: 42 additions & 0 deletions examples/v1/pipelineruns/alpha/param-enum.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: task-param-enum
spec:
params:
- name: message
type: string
enum: ["v1", "v2", "v3"]
steps:
- name: build
image: bash:latest
script: |
echo "$(params.message)"
---
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: pipeline-param-enum
spec:
params:
- name: message
enum: ["v1", "v2"]
default: "v1"
tasks:
- name: task1
params:
- name: message
value: $(params.message)
taskRef:
name: task-param-enum
---
apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
name: pipelinerun-param-enum
spec:
pipelineRef:
name: pipeline-param-enum
params:
- name: message
value: "v1"
2 changes: 2 additions & 0 deletions pkg/apis/pipeline/v1/pipelinerun_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,8 @@ const (
PipelineRunReasonCreateRunFailed PipelineRunReason = "CreateRunFailed"
// ReasonCELEvaluationFailed indicates the pipeline fails the CEL evaluation
PipelineRunReasonCELEvaluationFailed PipelineRunReason = "CELEvaluationFailed"
// PipelineRunReasonInvalidParamValue indicates that the PipelineRun Param input value is not allowed.
PipelineRunReasonInvalidParamValue PipelineRunReason = "InvalidParamValue"
)

func (t PipelineRunReason) String() string {
Expand Down
42 changes: 40 additions & 2 deletions pkg/reconciler/pipelinerun/pipelinerun.go
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,16 @@ func (c *Reconciler) reconcile(ctx context.Context, pr *v1.PipelineRun, getPipel
return controller.NewPermanentError(err)
}

if config.FromContextOrDefaults(ctx).FeatureFlags.EnableParamEnum {
if err := taskrun.ValidateEnumParam(ctx, pr.Spec.Params, pipelineSpec.Params); err != nil {
logger.Errorf("PipelineRun %q Param Enum validation failed: %v", pr.Name, err)
pr.Status.MarkFailed(v1.PipelineRunReasonInvalidParamValue.String(),
"PipelineRun %s/%s parameters have invalid value: %s",
pr.Namespace, pr.Name, err)
return controller.NewPermanentError(err)
}
}

// Ensure that the keys of an object param declared in PipelineSpec are not missed in the PipelineRunSpec
if err = resources.ValidateObjectParamRequiredKeys(pipelineSpec.Params, pr.Spec.Params); err != nil {
// This Run has failed, so we need to mark it as failed and stop reconciling it
Expand Down Expand Up @@ -521,6 +531,12 @@ func (c *Reconciler) reconcile(ctx context.Context, pr *v1.PipelineRun, getPipel
return controller.NewPermanentError(err)
}

// Make a deep copy of the Pipeline and its Tasks before value substution.
// This is used to find referenced pipeline-level params at each PipelineTask when validate param enum subset requirement
originalPipeline := pipelineSpec.DeepCopy()
originalTasks := originalPipeline.Tasks
originalTasks = append(originalTasks, originalPipeline.Finally...)

// Apply parameter substitution from the PipelineRun
pipelineSpec = resources.ApplyParameters(ctx, pipelineSpec, pr)
pipelineSpec = resources.ApplyContexts(pipelineSpec, pipelineMeta.Name, pr)
Expand Down Expand Up @@ -615,14 +631,22 @@ func (c *Reconciler) reconcile(ctx context.Context, pr *v1.PipelineRun, getPipel
pipelineRunFacts.TimeoutsState.PipelineTimeout = &pipelineTimeout
}

for _, rpt := range pipelineRunFacts.State {
for i, rpt := range pipelineRunFacts.State {
if !rpt.IsCustomTask() {
err := taskrun.ValidateResolvedTask(ctx, rpt.PipelineTask.Params, rpt.PipelineTask.Matrix, rpt.ResolvedTask)
if err != nil {
logger.Errorf("Failed to validate pipelinerun %q with error %v", pr.Name, err)
pr.Status.MarkFailed(v1.PipelineRunReasonFailedValidation.String(), err.Error())
return controller.NewPermanentError(err)
}

if config.FromContextOrDefaults(ctx).FeatureFlags.EnableParamEnum {
if err := resources.ValidateParamEnumSubset(originalTasks[i].Params, pipelineSpec.Params, rpt.ResolvedTask); err != nil {
logger.Errorf("Failed to validate pipelinerun %q with error %v", pr.Name, err)
pr.Status.MarkFailed(v1.PipelineRunReasonFailedValidation.String(), err.Error())
return controller.NewPermanentError(err)
}
}
}
}

Expand Down Expand Up @@ -864,10 +888,24 @@ func (c *Reconciler) createTaskRuns(ctx context.Context, rpt *resources.Resolved
defer span.End()
var taskRuns []*v1.TaskRun
var matrixCombinations []v1.Params

if rpt.PipelineTask.IsMatrixed() {
matrixCombinations = rpt.PipelineTask.Matrix.FanOut()
}
// validate the param values meet resolved Task Param Enum requirements before creating TaskRuns
if config.FromContextOrDefaults(ctx).FeatureFlags.EnableParamEnum {
for i := range rpt.TaskRunNames {
var params v1.Params
if len(matrixCombinations) > i {
params = matrixCombinations[i]
}
params = append(params, rpt.PipelineTask.Params...)
if err := taskrun.ValidateEnumParam(ctx, params, rpt.ResolvedTask.TaskSpec.Params); err != nil {
err = fmt.Errorf("Invalid param value from PipelineTask \"%s\": %w", rpt.PipelineTask.Name, err)
pr.Status.MarkFailed(v1.PipelineRunReasonInvalidParamValue.String(), err.Error())
return nil, controller.NewPermanentError(err)
}
}
}
for i, taskRunName := range rpt.TaskRunNames {
var params v1.Params
if len(matrixCombinations) > i {
Expand Down
Loading
Loading