WIP: [GA Merge-on-Read] enabling compaction for MoR related delete files

[cbb330]

Summary

developing in public, if reviewing keep in mind that everything here will change

Changes

• Client-facing API Changes
• Internal API Changes
• Bug Fixes
• New Features
• Performance Improvements
• Code Style
• Refactoring
• Documentation
• Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

• Manually Tested on local docker setup. Please include commands ran, and their output.
• Added new tests for the changes made.
• Updated existing tests to reflect the changes made.
• No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
• Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

• Breaking Changes
• Deprecations
• Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

[maluchari]

getContentSummary can cause namenode issues. It obtains a lock throughout the process of this computation and this will have an impact on namenode performance for larger directories. It is as expensive as ls but atleast ls drops the lock in between some batching it does internally but getContentSummary will not. This will cause a performance hit and also could be throttled since cost assigned to this could be higher while serving this request.