Skip to content
This repository has been archived by the owner on Nov 30, 2022. It is now read-only.

Make log send async #1174

Merged
merged 11 commits into from
Aug 31, 2022
Merged

Make log send async #1174

merged 11 commits into from
Aug 31, 2022

Conversation

sanders41
Copy link
Contributor

@sanders41 sanders41 commented Aug 29, 2022

Purpose

Resolve the issue listed in ethyca/fideslog#71

Changes

  • Use the async send from fideslog

Checklist

  • Update CHANGELOG.md file
    • Merge in main so the most recent CHANGELOG.md file is being appended to
    • Add description within the Unreleased section in an appropriate category. Add a new category from the list at the top of the file if the needed one isn't already there.
    • Add a link to this PR at the end of the description with the PR number as the text. example: #1
  • Applicable documentation updated (guides, quickstart, postman collections, tutorial, fidesdemo, database diagram.
  • If docs updated (select one):
    • documentation complete, or draft/outline provided (tag docs-team to complete/review on this branch)
    • documentation issue created (tag docs-team to complete issue separately)
  • Good unit test/integration test coverage
  • This PR contains a DB migration. If checked, the reviewer should confirm with the author that the down_revision correctly references the previous migration before merging
  • The Run Unsafe PR Checks label has been applied, and checks have passed, if this PR touches any external services

Ticket

Fixes #

@sanders41
Copy link
Contributor Author

@eastandwestwind I got all the tests passing now.

@sanders41 sanders41 added the run unsafe ci checks Triggers running of unsafe CI checks label Aug 31, 2022
Copy link
Contributor

@eastandwestwind eastandwestwind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work on this @sanders41! Kinda glad we tackled this groundwork first, since now we'll only have to replace our _AnalyticsClient__send with using send again, once fideslog is released.

Just one question and 1 blocker to address.

fidesops.toml Outdated Show resolved Hide resolved


@pytest.fixture(scope="session", autouse=True)
def event_loop():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you explain why this is needed specifically for tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It happens to be the same issue we are solving with the async log. When running the async tests it will sometimes start up multiple event loops and cause errors. I have found that using this fixture in my other async code bases has prevent the error.

from typing import Any, Callable


def sync(func: Callable) -> Any:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh nice workaround!

fidesops.toml Outdated
@@ -41,6 +41,7 @@ worker_enabled = false

[root_user]
analytics_opt_out = false
analytics_id = "bc03820cc476fd0bb87322950f07b295"
Copy link
Contributor

@seanpreston seanpreston Aug 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to remove this.

@sanders41 — which containers are you running? I wonder if the cause is the same this time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran lots of the nox commands on this one. Unfortunately I was watching for this and still somehow missed it so I'm not sure which one triggered it.

@eastandwestwind eastandwestwind merged commit 4e59d69 into main Aug 31, 2022
@eastandwestwind eastandwestwind deleted the ps-fideslog-loop-error branch August 31, 2022 16:20
sanders41 added a commit that referenced this pull request Sep 22, 2022
* Make log send async

* Update CHANGELOG

* Add async to additional fideslog calls

* WIP

* Fix issue with async function is celery

* Make __send work with name mangling

* Remove extra await

* Await coroutines in tests

* Remove analytics id

Co-authored-by: Paul Sanders <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
run unsafe ci checks Triggers running of unsafe CI checks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants