Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infer FaaS runtime from interpreter constraints, when unambiguous #19314

Merged
merged 8 commits into from
Jun 16, 2023

Conversation

huonw
Copy link
Contributor

@huonw huonw commented Jun 15, 2023

This allows the runtime argument to python_aws_lambda_function, python_aws_lambda_layer and python_google_cloud_function to be inferred from the relevant interpreter constraints, when they cover only one major/minor version. For instance, having ==3.9.* will infer runtime="python3.9" for AWS Lambda.

The inference is powered by checking for two patterns of interpreter constraints that limit to a single major version: equality ==3.9.* (implies 3.9) and range >=3.10,<3.11 (implies 3.10). This inference doesn't always work: when it doesn't work, the user gets an error message to clarify by providing the runtime field explicitly. Failure cases:

  • if the interpreter constraints are too wide (for instance, >=3.7,<3.9 covering 2 versions, or >=3.11 that'll eventually include many versions), we can't be sure which is meant
  • if the interpreter constraints limit the patch versions (for instance, ==3.8.9 matching a specific version, or ==3.9.*,!=3.9.10 excluding one), we can't be sure the cloud environment runs that version, so inferring the runtime would be misleading
  • if the interpreter constraints are non-obvious (for instance, >=3.7,<3.10,!=3.9.* is technically 3.8 only), we don't try too hard to handle it. We can expand the inference if required in future.

For instance, if one has set [python].interpreter_constraints = ["==3.9.*"] in pants.toml, one can build a lambda artefact like (and similarly for a GCF artifact):

python_sources()
python_aws_lambda_function(name="func", entry_point="./foo.py:handler")

This is the final piece* of my work to improve the FaaS backends in Pants 2.18:

(* The fixed complete platform files are currently only provided for AWS Lambda, not GCF. #18195.)

The commits are individually reviewable.

Fixes #19304

@huonw huonw requested a review from benjyw June 15, 2023 22:48
Copy link
Contributor

@benjyw benjyw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it!

@huonw
Copy link
Contributor Author

huonw commented Jun 16, 2023

Final commit just cleans out the remnants of the old approach that I had missed earlier (including updating the comment).

@huonw huonw enabled auto-merge (squash) June 16, 2023 02:14
@huonw huonw merged commit dc293a3 into pantsbuild:main Jun 16, 2023
huonw added a commit that referenced this pull request Aug 7, 2024
This updates the Lambda and GCF docs (very belatedly) for the "new"
behaviours related to choosing platform specific code, like inference of
the `runtime` field (#19314), and associated complete platforms
(#19253).

This includes stripping out the `runtime` field from the "main" flow,
and adding an separate subsection for how to specify the runtime
explicitly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Infer appropriate FaaS runtime from interpreter constraints
2 participants