Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add null check for tables stats collector when collecting table policies #235

Merged

Conversation

chenselena
Copy link
Collaborator

@chenselena chenselena commented Oct 22, 2024

Summary

This PR fixes the issue where tables that had a sharing policy set without a retention policy set would throw a null pointer exception when trying to collect retention stats.

java.lang.NullPointerException
	at com.linkedin.openhouse.jobs.util.TableStatsCollectorUtil.addEntriesToMap(TableStatsCollectorUtil.java:271)
	at com.linkedin.openhouse.jobs.util.TableStatsCollectorUtil.getTablePolicies(TableStatsCollectorUtil.java:250)
	at com.linkedin.openhouse.jobs.util.TableStatsCollectorUtil.populateTableMetadata(TableStatsCollectorUtil.java:155)
	at com.linkedin.openhouse.jobs.util.TableStatsCollector.collectTableStats(TableStatsCollector.java:25)
	at com.linkedin.openhouse.jobs.spark.Operations.collectTableStats(Operations.java:445)
	at com.linkedin.openhouse.jobs.spark.OperationsTest.testCollectTablePolicyStats(OperationsTest.java:603)

A null check is added to getTablePolicies so if sharing or retention policies do not exist on the table it will not collect those stats.

Changes

  • Client-facing API Changes
  • Internal API Changes
  • Bug Fixes
  • New Features
  • Performance Improvements
  • Code Style
  • Refactoring
  • Documentation
  • Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

  • Manually Tested on local docker setup. Please include commands ran, and their output.
  • Added new tests for the changes made.
  • Updated existing tests to reflect the changes made.
  • No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
  • Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Added unit tests for testing stat collection on tables without a retention policy.

Additional Information

  • Breaking Changes
  • Deprecations
  • Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

@chenselena chenselena marked this pull request as ready for review October 22, 2024 18:49
@chenselena chenselena merged commit 79e2a9f into linkedin:main Oct 22, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants