Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#9791] Fix datetime.datetime.utcnow() is deprecated as of Python 3.12 #9839

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

slothkong
Copy link
Contributor

@slothkong slothkong commented Apr 1, 2024

Resolves #9791

Problem

Any usage of datetime.datetime.utcnow() throws a deprecation warning in Python 3.12 when unittests are executed.

Solution

To get some results firsts, I just want to help with:

  1. the execution of your recommendations @dbeatty10, e.g. datetime.utcnow() -> datetime.now(timezone.utc).replace(tzinfo=None)
  2. bringing a bit of homogeneity to the ways datetime and its classes are being invoked, such that future development would require less effort

Code review checklist

  • Confirm that there are no breaking changes to dbt/artifacts
    • Otherwise, remove the artifact_minor_upgrade label. See #9839 (comment) for context.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

Additional Context

  • I successfully ran unit and integration tests on Python 3.12.0, Ubuntu 22.04
  • For good measure, I also built some models with both dbt-postgres and dbt-bigguery (1.8.0b1), then browsed over the resulting manifest.json files and verified that time references were accurate.
  • Currently, the only deprecation warnings raised during unittesting are related to dbt-common and logbook dependencies:
tests/unit/task/test_base.py: 2 warnings
  /path/to/my/dbt-core/.tox/py/lib/python3.12/site-packages/dbt_common/events/helpers.py:12: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    ts = datetime.utcnow()
tests/unit/task/test_base.py: 2 warnings
  /path/to/my/dbt-core/.tox/py/lib/python3.12/site-packages/dbt_common/events/logger.py:141: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    ts: str = datetime.utcnow().strftime("%H:%M:%S")
tests/unit/test_events.py::TestAdapterLogger::test_set_adapter_dependency_log_level
  /path/to/my/dbt-core/.tox/py/lib/python3.12/site-packages/logbook/compat.py:143: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
    return datetime.utcfromtimestamp(timestamp)

@slothkong slothkong requested a review from a team as a code owner April 1, 2024 11:27
@cla-bot cla-bot bot added the cla:yes label Apr 1, 2024
@github-actions github-actions bot added the community This PR is from a community member label Apr 1, 2024
Copy link

codecov bot commented Apr 1, 2024

Codecov Report

Attention: Patch coverage is 76.47059% with 8 lines in your changes missing coverage. Please review.

Project coverage is 88.05%. Comparing base (71f3519) to head (205a69e).
Report is 203 commits behind head on main.

Files with missing lines Patch % Lines
core/dbt/contracts/sql.py 0.00% 3 Missing ⚠️
core/dbt/parser/manifest.py 71.42% 2 Missing ⚠️
core/dbt/logger.py 50.00% 1 Missing ⚠️
core/dbt/task/sql.py 0.00% 1 Missing ⚠️
core/dbt/utils.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9839      +/-   ##
==========================================
- Coverage   88.12%   88.05%   -0.08%     
==========================================
  Files         178      178              
  Lines       22458    22458              
==========================================
- Hits        19792    19775      -17     
- Misses       2666     2683      +17     
Flag Coverage Δ
integration 85.41% <76.47%> (-0.17%) ⬇️
unit 61.78% <47.05%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@slothkong
Copy link
Contributor Author

slothkong commented Apr 1, 2024

Judging by what is noted in the README, about changing code of dbt artifacts, it should be acceptable to add the artifact_minor_upgrade tag to this PR, no?

Changes made to core/dbt/artifacts/schemas/base.py, core/dbt/artifacts/schemas/results.py and core/dbt/artifacts/schemas/run/v5/run.py do not fall into the braking changes category.

Just for the record, the returned type of datetime.utcnow() is equivalent to that of datetime.now(timezone.utc).replace(tzinfo=None) (e.g. <class 'datetime.datetime'>), and both methods work across python versions, from 3.12 all the way to 3.8.

@slothkong
Copy link
Contributor Author

I will try to submit an additional commit this week, to raise the patch coverage, specifically for core/dbt/contracts/sql.py~

@dbeatty10 dbeatty10 added ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering artifact_minor_upgrade To bypass the CI check by confirming that the change is not breaking labels Apr 15, 2024
@dbeatty10
Copy link
Contributor

Judging by what is noted in the README, about changing code of dbt artifacts, it should be acceptable to add the artifact_minor_upgrade tag to this PR, no?

@slothkong Since I'm not aware of any breaking changes, I've added the artifact_minor_upgrade label.

But to cover our bases, I've added this checklist item for the code review:

image

@edgarrmondragon
Copy link
Contributor

I will try to submit an additional commit this week, to raise the patch coverage, specifically for core/dbt/contracts/sql.py~

Is that a blocker for merging this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
artifact_minor_upgrade To bypass the CI check by confirming that the change is not breaking cla:yes community This PR is from a community member ready_for_review Externally contributed PR has functional approval, ready for code review from Core engineering
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] datetime.datetime.utcnow() is deprecated as of Python 3.12
3 participants