Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Make '--platform' argument mandatory in CLI to prevent incorrect behavior #1462

Closed
parthosa opened this issue Dec 13, 2024 · 0 comments · Fixed by #1463
Closed

[FEA] Make '--platform' argument mandatory in CLI to prevent incorrect behavior #1462

parthosa opened this issue Dec 13, 2024 · 0 comments · Fixed by #1463
Assignees
Labels
bug Something isn't working user_tools Scope the wrapper module running CSP, QualX, and reports (python)

Comments

@parthosa
Copy link
Collaborator

Description

The platform detection logic currently relies on the event log file’s protocol (e.g. s3://, gs://, hdfs://) to identify the platform. This approach can lead to incorrect platform detection in certain cases:

  • S3 Protocol: Event logs with the s3:// protocol are always identified as EMR, which is incorrect since Databricks AWS logs can also use S3 storage.
  • File Protocol or Local Filesystem: Event logs stored locally or with the file:// protocol are always identified as onprem. This might not be the actual platform on which the user had run the application.

Impact

Incorrect platform detection leads to invalid speedup estimations, as QualX uses platform-specific models for its calculations.

Proposed Change

Until the platform detection logic is improved, make the --platform argument mandatory. This ensures users explicitly specify the platform, avoiding incorrect behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working user_tools Scope the wrapper module running CSP, QualX, and reports (python)
Projects
None yet
1 participant