Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Improve Scala 2.13 build process to not require building against unstaged git changes #9331

Closed
NVnavkumar opened this issue Sep 28, 2023 · 1 comment · Fixed by #8592
Assignees
Labels
build Related to CI / CD or cleanly building Scala 2.13 Issues related to Scala 2.13 binaries

Comments

@NVnavkumar
Copy link
Collaborator

NVnavkumar commented Sep 28, 2023

This is a follow on to #1525. The changes in #8592 do not address a concern in the build process in that the CI will build against unstaged git changes. See this thread:

#8592 (comment)

We should generate the pom files directly, and we can put the generated pom files in gitignore, so the CI will be able to build against what it sees as a clean working copy.

Note the CI will not be building Scala 2.13 artifacts until this issue is addressed.

@NVnavkumar NVnavkumar added feature request New feature or request ? - Needs Triage Need team to review and classify labels Sep 28, 2023
@NVnavkumar NVnavkumar self-assigned this Sep 28, 2023
@gerashegalov
Copy link
Collaborator

gerashegalov commented Sep 28, 2023

archetype could be an interesting option for pom generation. We may even be able to use it for generating classes like


/*** spark-rapids-shim-json-lines
{"spark": "331"}
spark-rapids-shim-json-lines ***/
package com.nvidia.spark.rapids.spark331

import org.apache.spark.SparkConf
import org.apache.spark.sql.rapids.shims.spark331.ProxyRapidsShuffleInternalManager

/** A shuffle manager optimized for the RAPIDS Plugin for Apache Spark. */
sealed class RapidsShuffleManager(
    conf: SparkConf,
    isDriver: Boolean) extends ProxyRapidsShuffleInternalManager(conf, isDriver) {
}

@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Sep 29, 2023
@NVnavkumar NVnavkumar linked a pull request Oct 18, 2023 that will close this issue
@gerashegalov gerashegalov added the Scala 2.13 Issues related to Scala 2.13 binaries label Oct 31, 2023
@sameerz sameerz added build Related to CI / CD or cleanly building and removed feature request New feature or request labels Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Related to CI / CD or cleanly building Scala 2.13 Issues related to Scala 2.13 binaries
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants