Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GSProcessing&GSPartition] Support Hard Negative #1080

Merged
merged 56 commits into from
Nov 15, 2024
Merged

[GSProcessing&GSPartition] Support Hard Negative #1080

merged 56 commits into from
Nov 15, 2024

Conversation

jalencato
Copy link
Collaborator

@jalencato jalencato commented Oct 29, 2024 •
edited
Loading

Issue #, if available:

Description of changes:

  • Add feature support for hard negative for distributed graph construction pipeline. For partition stage, users does not need to do any additional step for hard negative feature transformation itself.
  • Implementation with the way we agreed on with the design doc.
  • Create additional steps for distributed partition as we can not fully reused the previous script because different ways for mapping files. I create a separate one in the gspartition code repo to easy maintain.
  • Tested with 1M nodes and 1M edges.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jalencato jalencato added the ready able to trigger the CI label Nov 4, 2024
thvasilo
thvasilo requested changes Nov 12, 2024
View reviewed changes
...rm_processing/data_transformations/dist_transformations/dist_hard_negative_transformation.py Outdated Show resolved Hide resolved
...rm_processing/data_transformations/dist_transformations/dist_hard_negative_transformation.py Outdated Show resolved Hide resolved
...rm_processing/data_transformations/dist_transformations/dist_hard_negative_transformation.py Outdated Show resolved Hide resolved
...rm_processing/data_transformations/dist_transformations/dist_hard_negative_transformation.py Outdated Show resolved Hide resolved
...rm_processing/data_transformations/dist_transformations/dist_hard_negative_transformation.py Show resolved Hide resolved
python/graphstorm/gpartition/post_hard_negative.py Outdated Show resolved Hide resolved
python/graphstorm/model/utils.py Show resolved Hide resolved
tests/unit-tests/gpartition/test_hard_negative_post_partition.py Outdated Show resolved Hide resolved
tests/unit-tests/gpartition/test_hard_negative_post_partition.py Outdated Show resolved Hide resolved
tests/unit-tests/gpartition/test_hard_negative_post_partition.py Outdated Show resolved Hide resolved
@awslabs awslabs deleted a comment from thvasilo Nov 12, 2024
@jalencato jalencato requested a review from thvasilo November 13, 2024 00:41
@awslabs awslabs deleted a comment from thvasilo Nov 13, 2024
thvasilo
thvasilo approved these changes Nov 14, 2024
View reviewed changes
Copy link
Contributor

@thvasilo thvasilo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now, ideally we want to verify correctness with some regression tests.

@jalencato jalencato merged commit c964cc4 into awslabs:main Nov 15, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thvasilo thvasilo thvasilo approved these changes

@classicsong classicsong Awaiting requested review from classicsong

Assignees

@jalencato jalencato

Labels
ready able to trigger the CI
Projects
None yet
Milestone
0.4 release
Development

Successfully merging this pull request may close these issues.
3 participants
@jalencato @classicsong @thvasilo