-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(sdk): Backport fixes in kubeflow/pipelines#11075 #11392
base: master
Are you sure you want to change the base?
Conversation
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
df40f68
to
b8b72fe
Compare
02daafe
to
e11eb4c
Compare
e11eb4c
to
bc62bfd
Compare
bc62bfd
to
848b614
Compare
293792f
to
c669a7b
Compare
container_spec.resources.cpu_request = compiler_utils.validate_cpu_request_limit_to_float( | ||
cpu=convert_to_placeholder( | ||
task.container_spec.resources.cpu_request)) | ||
if task.container_spec.resources.cpu_limit is not None: | ||
container_spec.resources.resource_cpu_limit = ( | ||
convert_to_placeholder(task.container_spec.resources.cpu_limit)) | ||
container_spec.resources.cpu_limit = compiler_utils.validate_cpu_request_limit_to_float( | ||
cpu=convert_to_placeholder( | ||
task.container_spec.resources.cpu_limit)) | ||
if task.container_spec.resources.memory_request is not None: | ||
container_spec.resources.resource_memory_request = ( | ||
convert_to_placeholder( | ||
task.container_spec.resources.memory_request)) | ||
container_spec.resources.memory_request = compiler_utils.validate_memory_request_limit_to_float( | ||
memory=convert_to_placeholder( | ||
task.container_spec.resources.memory_request)) | ||
if task.container_spec.resources.memory_limit is not None: | ||
container_spec.resources.resource_memory_limit = ( | ||
convert_to_placeholder( | ||
task.container_spec.resources.memory_limit)) | ||
container_spec.resources.memory_limit = compiler_utils.validate_memory_request_limit_to_float( | ||
memory=convert_to_placeholder( | ||
task.container_spec.resources.memory_limit)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is altering the initial logic and when validation happens. I would prefer it if we kept the logic of how we validate the deprecated fields similar to how it used to be, and keep it in class PipelineTask:
the way I suggest doing this is:
Return these fields as float types, but rename them as:
cpu_request_deprecated: Optional[float] = None
cpu_limit_deprecated: Optional[float] = None
memory_request_deprecated: Optional[float] = None
memory_limit_deprecated: Optional[float] = None
Then move validate_cpu_request_limit_to_float
& validate_memory_request_limit_to_float
back in to pipeline_task.py
, and when set_cpu_request
, set_cpu_limit
, set_memory_request
, set_memory_limit
is called, we set the deprecated fields the same as we did before the breaking change, and validate right before assignment. This keeps the logic functioning similarly as it used to, and it keeps all the logic close to gether so it's easier to remove it when we sunset these fields.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I only followed the way it's done for Accelerator fields. So Accelerator should be changed as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The suggestion is to update the non accelerator fields and re-add them as deprecated values to resourcespec
Thinking more on this, I don't feel very strongly about this. @chensun do you have a preference here?
If not then this lgtm.
Thanks @rimolive can you also add this change to: https://github.com/kubeflow/pipelines/blob/master/sdk/RELEASE.md#bug-fixes-and-other-changes? otherwise lgtm |
c669a7b
to
fe17f23
Compare
Introduced back the functions to convert k8s size values to float, but moved to kfp.dsl.utils Signed-off-by: Ricardo M. Oliveira <[email protected]>
fe17f23
to
8ea2606
Compare
@chensun PTAL when you have the chance, this is another api breaking change that was introduced in 2.10 |
Introduced back the functions to convert k8s size values to float, but moved to kfp.dsl.utils
Description of your changes:
fixes #11390
Checklist: