Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save hosts and tickets to datalake #250

Merged
merged 6 commits into from
May 15, 2024
Merged

Conversation

DJensen94
Copy link
Contributor

Save hosts and tickets to datalake

🗣 Description

This PR pull hosts and tickets collections from VS and saves them to the mini datalake, allowing us to provide VS data to stakeholders via crossfeed

💭 Motivation and context

This data allows us to recreate the VS scorecard data in Crossfeed

🧪 Testing

Tested Locally

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All future TODOs are captured in issues, which are referenced
    in code comments.
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • Tests have been added and/or modified to cover the changes in this PR.
  • All new and existing tests pass.

✅ Pre-merge checklist

  • Revert dependencies to default branches.
  • Finalize version.

✅ Post-merge checklist

  • Create a release.

Save hosts and tickets to datalake
DJensen94 added 2 commits May 13, 2024 15:05
run linter
remove unnecessary conditional
Copy link
Collaborator

@cduhn17 cduhn17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Adding a variable that was left null
Copy link
Collaborator

@rapidray12 rapidray12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing this makes Prisma look so much better

DJensen94 added 2 commits May 15, 2024 09:31
update connection logic
remove idna
@schmelz21 schmelz21 merged commit 4f19718 into develop May 15, 2024
18 of 19 checks passed
@schmelz21 schmelz21 deleted the DJ_extract_cyhy_data_WIP branch May 15, 2024 15:30
@DJensen94 DJensen94 restored the DJ_extract_cyhy_data_WIP branch May 16, 2024 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants