Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] CLI-227: add implicit context_mount to Image docker methods #2582

Closed
wants to merge 34 commits into from

Conversation

kramstrom
Copy link
Contributor

@kramstrom kramstrom commented Nov 27, 2024

Adds deprecation warning when using context_mount parameter on Image docker methods as part of the effort to remove mounts. This parameter which used to be necessary for COPY commands to work.

This PR instead creates an implicit context mount from the dockerfile/docker commands by parsing the COPY source patterns and use them as mount filters for the current directory.

Also adds support for finding a dockerignore file that matches the given dockerfile or if there's a generic one if using docker commands

@kramstrom kramstrom force-pushed the kramstrom/cli-227-remove-context_mount-from branch from dcacc4a to 10f0df6 Compare November 28, 2024 14:04
@kramstrom kramstrom force-pushed the kramstrom/cli-227-remove-context_mount-from branch from bffa50d to a650752 Compare November 28, 2024 14:05
@kramstrom kramstrom requested a review from freider December 4, 2024 09:25
Copy link
Contributor

@freider freider left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't looked at the details too much, but looks good on the surface!

I just realized we might also want to (separately) add more fine grained control mechanisms/filter support to give people a way to exclude particular files they don't want to include (dockerignore style) before we can forcibly push people to use this.

As a start, maybe we could just start exposing condition in a more user-friendly way. Will discuss separately in slack

modal/image.py Outdated Show resolved Hide resolved
@kramstrom kramstrom changed the title [WIP] CLI-227: remove context_mount from Image docker methods [WIP] CLI-227: add implicit context_mount to Image docker methods Dec 16, 2024
@kramstrom kramstrom changed the title [WIP] CLI-227: add implicit context_mount to Image docker methods [WIP] CLI-227: add implicit context_mount to Image docker methods Dec 16, 2024
modal/_utils/match.py Outdated Show resolved Hide resolved
modal/image.py Outdated Show resolved Hide resolved
@@ -1162,6 +1181,24 @@ def dockerfile_commands(
if not cmds:
return self

if context_mount is not None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to add the ignore argument here as well if people want manual inline ignore patterns without external files, as a replacement for context_mount. And the pending deprecation could point people towards that argument

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or dockerignore obviously)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, does dockerfile_commands support dockerignore with this? We only call find_dockerignore_file in from_dockerfile right? 🤔
No biggie if it doesn't - as long as we support ignore for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added to dockerfile_commands as well -- currently both methods will try to find a dockerignore file unless a custom ignore argument is passed

return True


AUTO_DOCKERIGNORE = _AutoDockerIgnore().with_repr(f"{__name__}.AUTO_DOCKERIGNORE")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's all this repr magic used for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the repr inside the class, but it's just a placeholder value for type hints & docs to make it simpler to understand what will happen if you don't pass in a custom ignore pattern-list or callable --> then we default to trying to find a matching dockerignore to your dockerfile (or just a generic one if you're using docker_commands) -- same as we have for add_local_python_source

Screenshot 2024-12-20 at 13 49 59

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TLDR for why it's needed (this was my suggestion)

It's so that we can use these as default values in signatures and still emit valid python signatures in our type stubs, since things like inspect.signature(...) will just use the repr of all default values, and can't figure out stuff like import path of the specific constant we use as a default.

It's definitely a bit of a hack, but not something users should see. An alternative would be to do a bunch of ast magic to extract the signatures.

@kramstrom kramstrom closed this Jan 7, 2025
@kramstrom kramstrom deleted the kramstrom/cli-227-remove-context_mount-from branch January 7, 2025 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants