Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GSProcessing] Add support for numerical and multi-numerical transformations. #575

Merged
merged 7 commits into from
Oct 27, 2023
Merged

[GSProcessing] Add support for numerical and multi-numerical transformations. #575

merged 7 commits into from
Oct 27, 2023

Conversation

thvasilo
Copy link
Contributor

Issue #, if available:
Closes #572

Description of changes:

  • Adds a single-column and multi-column numerical transformations.
  • We support missing data imputation, and value normalization.
    • For missing data we support mean, median and most_frequent imputation.
    • For normalization we support mix-max and standard which divides all values
      by their column sum.
  • We update the GConstruct config converter to allow pass-through of the min-max transformation.
  • We change the GSProcessing feature config key that describes the transformation from transform to transformation.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@thvasilo thvasilo added ready able to trigger the CI 0.2.1 gsprocessing For issues and PRs related the the GSProcessing library labels Oct 17, 2023
@thvasilo thvasilo requested a review from jalencato October 17, 2023 18:19
@thvasilo thvasilo self-assigned this Oct 17, 2023
@thvasilo thvasilo force-pushed the numerical-transform branch from 8551c37 to 291d368 Compare October 17, 2023 18:20
@thvasilo thvasilo added draft label only to be used by dev team - skips CI for small changes and removed ready able to trigger the CI labels Oct 17, 2023
@thvasilo thvasilo force-pushed the numerical-transform branch from 291d368 to 27d89b4 Compare October 17, 2023 18:31
@thvasilo thvasilo added ready able to trigger the CI and removed draft label only to be used by dev team - skips CI for small changes labels Oct 17, 2023
@thvasilo thvasilo added this to the 0.2.1 Release Plan. milestone Oct 17, 2023
@thvasilo thvasilo requested a review from classicsong October 17, 2023 18:34
@thvasilo thvasilo force-pushed the numerical-transform branch 3 times, most recently from 4922852 to 64b6888 Compare October 17, 2023 18:41
@thvasilo thvasilo force-pushed the numerical-transform branch from 64b6888 to 9f5ee5d Compare October 17, 2023 18:46
classicsong
classicsong reviewed Oct 19, 2023
View reviewed changes
docs/source/gs-processing/developer/input-configuration.rst Outdated Show resolved Hide resolved
docs/source/gs-processing/developer/input-configuration.rst Show resolved Hide resolved
graphstorm-processing/graphstorm_processing/config/config_conversion/gconstruct_converter.py Show resolved Hide resolved
graphstorm-processing/graphstorm_processing/config/numerical_configs.py Outdated Show resolved Hide resolved
graphstorm-processing/graphstorm_processing/config/numerical_configs.py Outdated Show resolved Hide resolved
...hstorm_processing/data_transformations/dist_transformations/dist_numerical_transformation.py Outdated Show resolved Hide resolved
...hstorm_processing/data_transformations/dist_transformations/dist_numerical_transformation.py Show resolved Hide resolved
graphstorm-processing/tests/test_dist_heterogenous_loader.py Show resolved Hide resolved
graphstorm-processing/tests/test_dist_numerical_transformation.py Show resolved Hide resolved
graphstorm-processing/tests/test_dist_numerical_transformation.py Outdated Show resolved Hide resolved
@thvasilo thvasilo merged commit d6d5819 into awslabs:main Oct 27, 2023
8 checks passed
@thvasilo thvasilo deleted the numerical-transform branch October 27, 2023 23:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jalencato jalencato jalencato left review comments

@classicsong classicsong classicsong approved these changes

Assignees

@thvasilo thvasilo

Labels
0.2.1 gsprocessing For issues and PRs related the the GSProcessing library ready able to trigger the CI
Projects
None yet
Milestone
0.2.1 Release Plan.
Development

Successfully merging this pull request may close these issues.

[GSProcessing] Numerical transformation
3 participants
@thvasilo @classicsong @jalencato