Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paw/ct 1844 log params take 2 #6994

Merged
merged 14 commits into from
Feb 24, 2023
Merged

Conversation

peterallenwebb
Copy link
Contributor

@peterallenwebb peterallenwebb commented Feb 16, 2023

resolves #6639

Description

Adds the following cli parameters for finer grained control of console and file log behavior, as described in #6639:

--log-format
--log-format-file
--log-level
--log-level-file
--use-colors
--use-colors-file
--no-use-colors
--no-use-colors-file the

Checklist

@cla-bot cla-bot bot added the cla:yes label Feb 16, 2023
@github-actions
Copy link
Contributor

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@peterallenwebb peterallenwebb marked this pull request as ready for review February 16, 2023 20:04
@peterallenwebb peterallenwebb requested review from a team as code owners February 16, 2023 20:04
@gshank
Copy link
Contributor

gshank commented Feb 16, 2023

This doesn't touch UserConfig (in core/dbt/contracts/project.py) where users can set log_format and use_colors. So I'm wondering how those settings integrate with these cli params, and whether we shouldn't allow setting the new params (somehow) in user config also. If I have a logging configuration that I want to use all the time, I would prefer not to have to pass in cli params every time, but have the settings persistent in user config.

@peterallenwebb
Copy link
Contributor Author

@gshank Yes, that is an interesting point which I did not consider.

@jtcohen6 Do you agree that we should let users set these same flags in their user config, following the pattern of log_format and use_colors? I worry that the interaction patterns between all the different places these get set could start to become complicated, but Gerda's observations do make sense to me.

On the other hand, maybe Gerda's use case could be supported by the work we are considering for #6993.

@jtcohen6
Copy link
Contributor

jtcohen6 commented Feb 21, 2023

@peterallenwebb @gshank Two-part answer:

  1. Every "global config" that is settable via CLI flag + env var should also be settable in UserConfig. I know that is not the case today, but it is the documented behavior, and we ought to make it so. Let's do this for these logging parameters as well.
  2. We should move UserConfig out of profiles.yml, and into its own standalone config file, in order to simplify the initialization flow for dbt-core. I agree it's a tangled web we weave today. My latest proposal for this: [CT-1470] Whither UserConfig? #6207 (comment)

@peterallenwebb
Copy link
Contributor Author

@gshank Still waiting for the checks to pass here, but let me know whether the latest changes address your concerns.

@@ -172,6 +177,12 @@ def assign_params(ctx, params_assigned_from_default):
for param in params:
object.__setattr__(self, param.lower(), getattr(self, param))

# If the value of the lead parameter was set explicitly, apply the value to follow,
# unless follow was also set explicitly.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny nit: documenting the method in a docstring below the signature would be more pythonic + consistent with the rest of this file.

log_cache_events: bool,
quiet: bool,
) -> None:
def setup_event_logger(flags) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we able to annotate the type of flags here?

)
else:
if flags.LOG_LEVEL != "none":
log_format = _line_format_from_str(flags.LOG_FORMAT, LineFormat.PlainText)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a difference between log_format and line_format? If not, line_format would be more intuitive given the naming of _line_format_from_str + parameter name of _get_stdout_config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, the variable being assigned here should be line_format. There is no difference between the two things, and the difference in naming is unfortunate.

Copy link
Contributor

@MichelleArk MichelleArk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple small nits, otherwise LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-1844] Improve Granularity of Log Configuration
5 participants