Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HPCC4J-672 Include hpcc-spark source code into the hpcc4j project #779

Conversation

Michael-Gardner
Copy link
Contributor

@Michael-Gardner Michael-Gardner commented Dec 16, 2024

Type of change:

  • This change is a bug fix (non-breaking change which fixes an issue).
  • This change is a new feature (non-breaking change which adds functionality).
  • This change is a breaking change (fix or feature that will cause existing behavior to change).

Checklist:

  • I have created a corresponding JIRA ticket for this submission
  • My code follows the code style of this project.
    • I have applied the Eclipse code-format template provided.
  • My change requires a change to the documentation.
    • I have updated the documentation accordingly, or...
    • I have created a JIRA ticket to update the documentation.
    • Any new interfaces or exported functions are appropriately commented.
  • I have read the HPCC Systems CONTRIBUTORS document (https://github.com/hpcc-systems/HPCC-Platform/wiki/Guide-for-contributors).
  • The change has been fully tested:
    • This change does not cause any existing JUnits to fail.
    • I have include JUnit coverage to test this change
    • I have performed system test and covered possible regressions and side effects.
  • I have given due consideration to all of the following potential concerns:
    • Scalability
    • Performance
    • Security
    • Thread-safety
    • Premature optimization
    • This change fixes the problem, not just the symptom

Testing:

Copy link

Jira Issue: https://hpccsystems.atlassian.net/browse/HPCC4J-672

Jirabot Action Result:
Workflow Transition To: Merge Pending
Updated PR

@Michael-Gardner
Copy link
Contributor Author

This is just adding spark-hpcc:master source code to the hpcc4j/spark-hpcc directory. Working on fixing the pom and getting local builds working in a second pull request for HPCC4J-673

@rpastrana
Copy link
Member

@Michael-Gardner Looks good. However, we realized this is a great opportunity to remove the top-level DataAccess directory. Can we move everything up a level and remove 'DataAccess'?

@jpmcmu what do you think about the /Examples folder? Where could we put that?

@Michael-Gardner
Copy link
Contributor Author

Agreed. My 673 PR moves the code from spark-hpcc/DataAccess over to spark-hpcc with examples alongside. Then fixes all the poms so that spark-hpcc is a module of the top level poms project.

@jpmcmu
Copy link
Contributor

jpmcmu commented Dec 17, 2024

@rpastrana @Michael-Gardner I think the Examples folder can hang out in the Spark-HPCC folder for now. If we need to move it later that will be easy todo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants