Issue while building Build/job/KnowledgePlatform/job/InquiryFlinkJob/

[Arunkumarpodili]

Hey @pramodkvarma,
We are trying to build and deploy sunbird on our environment.
while building the service InquiryFlinkJob service (Build/job/KnowledgePlatform/job/InquiryFlinkJob/
) we are facing some depenancy issues. we are looking for your support to proceed furter.

here is the error of build job:
Inquiryflinkjob_build_log.txt

Thanks
Arunkumar.P

[krgauraw]

Hi @Arunkumarpodili , we have some changes in jenkins build script. we will provide the updated script in next 3 hr.
cc: @PoojaJManjooran @sajeshkayyath

[Arunkumarpodili]

Thank you so much for your response, we await your next update.

[krgauraw]

Hi @Arunkumarpodili
Please use release-5.7.0_RC4 (https://github.com/Sunbird-inQuiry/data-pipeline/tree/release-5.7.0_RC4) for installation.

For updated jenkins script, Please re-import all 3 jobs (Build, ArtifactUpload, Deploy) from https://github.com/Sunbird-inQuiry/data-pipeline/tree/release-5.7.0_RC4/scripts/jenkins-jobs

Hope this helps.
cc: @PoojaJManjooran @sajeshkayyath

[Arunkumarpodili]

Hello @krgauraw,
we tried to build with release-5.7.0_RC4 for installation but again we got same errors, For your reference.

Started by user Arunkumar P
12:31:08 Rebuilds build #7
12:31:08 Checking out git https://github.com/Sunbird-inQuiry/data-pipeline.git into /var/lib/jenkins/workspace/Build/KnowledgePlatform/InquiryFlinkJob@script/976a4c6e5ef673f7424491ad4bb1e25376147d371f242064514d33e9585a4d47 to read kubernetes/pipelines/build/Jenkinsfile
12:31:08 The recommended git tool is: NONE
12:31:08 No credentials specified
12:31:08 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Build/KnowledgePlatform/InquiryFlinkJob@script/976a4c6e5ef673f7424491ad4bb1e25376147d371f242064514d33e9585a4d47/.git # timeout=10
12:31:08 Fetching changes from the remote Git repository
12:31:08 > git config remote.origin.url https://github.com/Sunbird-inQuiry/data-pipeline.git # timeout=10
12:31:08 Fetching upstream changes from https://github.com/Sunbird-inQuiry/data-pipeline.git
12:31:08 > git --version # timeout=10
12:31:08 > git --version # 'git version 2.17.1'
12:31:08 > git fetch --tags --progress -- https://github.com/Sunbird-inQuiry/data-pipeline.git +refs/heads/:refs/remotes/origin/ # timeout=10
12:31:09 > git rev-parse origin/release-5.7.0_RC4^{commit} # timeout=10
12:31:09 > git rev-parse release-5.7.0_RC4^{commit} # timeout=10
12:31:09 Checking out Revision 4c9c84b8086dd262684043ffe1e646c2ec9f8d99 (release-5.7.0_RC4)
12:31:09 > git config core.sparsecheckout # timeout=10
12:31:09 > git checkout -f 4c9c84b8086dd262684043ffe1e646c2ec9f8d99 # timeout=10
12:31:09 Commit message: "Merge pull request #59 from krgauraw/release-5.7.0"
12:31:09 First time build. Skipping changelog.
12:31:10 [Pipeline] Start of Pipeline
12:31:10 [Pipeline] node
12:31:10 Running on Jenkins in /var/lib/jenkins/workspace/Build/KnowledgePlatform/InquiryFlinkJob
12:31:10 [Pipeline] {
12:31:10 [Pipeline] ansiColor
12:31:10 [Pipeline] {
12:31:10
12:31:10 [Pipeline] withEnv
12:31:10 [Pipeline] {
12:31:10 [Pipeline] stage
12:31:10 [Pipeline] { (Checkout)
12:31:10 [Pipeline] echo
12:31:10 Found environment variable named hub_org with value as: registry.sunbird-optitcloud.in
12:31:10 [Pipeline] }
12:31:10 [Pipeline] // stage
12:31:10 [Pipeline] cleanWs
12:31:10 [WS-CLEANUP] Deleting project workspace...
12:31:10 [WS-CLEANUP] Deferred wipeout is used...
12:31:10 [WS-CLEANUP] done
12:31:10 [Pipeline] dir
12:31:10 Running in /var/lib/jenkins/workspace/Build/KnowledgePlatform/InquiryFlinkJob/data-pipeline
12:31:10 [Pipeline] {
12:31:10 [Pipeline] checkout
12:31:10 The recommended git tool is: NONE
12:31:10 No credentials specified
12:31:10 Cloning the remote Git repository
12:31:10 Cloning repository https://github.com/Sunbird-inQuiry/data-pipeline.git
12:31:10 > git init /var/lib/jenkins/workspace/Build/KnowledgePlatform/InquiryFlinkJob/data-pipeline # timeout=10
12:31:10 Fetching upstream changes from https://github.com/Sunbird-inQuiry/data-pipeline.git
12:31:10 > git --version # timeout=10
12:31:10 > git --version # 'git version 2.17.1'
12:31:10 > git fetch --tags --progress -- https://github.com/Sunbird-inQuiry/data-pipeline.git +refs/heads/:refs/remotes/origin/ # timeout=10
12:31:12 > git config remote.origin.url https://github.com/Sunbird-inQuiry/data-pipeline.git # timeout=10
12:31:12 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10
12:31:12 Avoid second fetch
12:31:12 > git rev-parse origin/release-5.7.0_RC4^{commit} # timeout=10
12:31:12 > git rev-parse release-5.7.0_RC4^{commit} # timeout=10
12:31:12 Checking out Revision 4c9c84b8086dd262684043ffe1e646c2ec9f8d99 (release-5.7.0_RC4)
12:31:12 > git config core.sparsecheckout # timeout=10
12:31:12 > git checkout -f 4c9c84b8086dd262684043ffe1e646c2ec9f8d99 # timeout=10
12:31:13 Commit message: "Merge pull request #59 from krgauraw/release-5.7.0"
12:31:13 [Pipeline] sh
12:31:13 + git rev-parse --short HEAD
12:31:13 [Pipeline] sh
12:31:13 + echo release-5.7.0_RC4_4c9c84b8_8
12:31:13 [Pipeline] echo
12:31:13 build_tag: release-5.7.0_RC4_4c9c84b8_8
12:31:13 [Pipeline] stage
12:31:13 [Pipeline] { (Build)
12:31:13 [Pipeline] echo
12:31:13 Environment will be : build
12:31:13 [Pipeline] sh
12:31:14 + /opt/apache-maven-3.6.3/bin/mvn3.6 -v
12:31:14 Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
12:31:14 Maven home: /opt/apache-maven-3.6.3
12:31:14 Java version: 11, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-11-openjdk-amd64
12:31:14 Default locale: en_US, platform encoding: UTF-8
12:31:14 OS name: "linux", version: "4.15.0-213-generic", arch: "amd64", family: "unix"
12:31:14 [Pipeline] sh
12:31:14 + /opt/apache-maven-3.6.3/bin/mvn3.6 clean install -DskipTests -DCLOUD_STORE_GROUP_ID=null -DCLOUD_STORE_ARTIFACT_ID=null -DCLOUD_STORE_VERSION=null
12:31:16 [INFO] Scanning for projects...
12:31:16 [WARNING] The project org.sunbird:data-pipeline:pom:1.0 uses prerequisites which is only intended for maven-plugin projects but not for non maven-plugin projects. For such purposes you should use the maven-enforcer-plugin. See https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html
12:31:16 [INFO] ------------------------------------------------------------------------
12:31:16 [INFO] Reactor Build Order:
12:31:16 [INFO]
12:31:16 [INFO] inQuiry Flink Jobs [pom]
12:31:16 [INFO] jobs-core [jar]
12:31:16 [INFO] publish-pipeline [pom]
12:31:16 [INFO] publish-core [jar]
12:31:16 [INFO] async-questionset-publish [jar]
12:31:16 [INFO] questionset-republish [jar]
12:31:16 [INFO]
12:31:16 [INFO] ---------------------< org.sunbird:data-pipeline >----------------------
12:31:16 [INFO] Building inQuiry Flink Jobs 1.0 [1/6]
12:31:16 [INFO] --------------------------------[ pom ]---------------------------------
12:31:16 [INFO]
12:31:16 [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ data-pipeline ---
12:31:16 [INFO]
12:31:16 [INFO] --- maven-install-plugin:2.4:install (default-install) @ data-pipeline ---
12:31:16 [INFO] Installing /var/lib/jenkins/workspace/Build/KnowledgePlatform/InquiryFlinkJob/data-pipeline/pom.xml to /var/lib/jenkins/.m2/repository/org/sunbird/data-pipeline/1.0/data-pipeline-1.0.pom
12:31:16 [INFO]
12:31:16 [INFO] -----------------------< org.sunbird:jobs-core >------------------------
12:31:16 [INFO] Building jobs-core 1.0.0 [2/6]
12:31:16 [INFO] --------------------------------[ jar ]---------------------------------
12:31:17 Downloading from my-local-repo: file:///var/lib/jenkins/.m2/repository/null/null/null/null-null.pom
12:31:17 Downloading from apache-releases: https://repository.apache.org/content/groups/public/null/null/null/null-null.pom
12:31:18 Downloading from scala-tools.org: https://oss.sonatype.org/content/groups/scala-tools/null/null/null/null-null.pom
12:31:19 Downloading from central: https://repo.maven.apache.org/maven2/null/null/null/null-null.pom
12:31:20 [INFO] ------------------------------------------------------------------------
12:31:20 [INFO] Reactor Summary:
12:31:20 [INFO]
12:31:20 [INFO] inQuiry Flink Jobs 1.0 ............................. SUCCESS [ 0.327 s]
12:31:20 [INFO] jobs-core 1.0.0 .................................... FAILURE [ 3.774 s]
12:31:20 [INFO] publish-pipeline 1.0 ............................... SKIPPED
12:31:20 [INFO] publish-core 1.0.0 ................................. SKIPPED
12:31:20 [INFO] async-questionset-publish 1.0.0 .................... SKIPPED
12:31:20 [INFO] questionset-republish 1.0.0 ........................ SKIPPED
12:31:20 [INFO] ------------------------------------------------------------------------
12:31:20 [INFO] BUILD FAILURE
12:31:20 [INFO] ------------------------------------------------------------------------
12:31:20 [INFO] Total time: 4.262 s
12:31:20 [INFO] Finished at: 2023-12-19T12:31:20+05:30
12:31:20 [INFO] ------------------------------------------------------------------------
12:31:20 [ERROR] Failed to execute goal on project jobs-core: Could not resolve dependencies for project org.sunbird:jobs-core:jar:1.0.0: Failed to collect dependencies at null:null:jar:null: Failed to read artifact descriptor for null:null:jar:null: Could not transfer artifact null:null:pom:null from/to scala-tools.org (https://oss.sonatype.org/content/groups/scala-tools): Authorization failed for https://oss.sonatype.org/content/groups/scala-tools/null/null/null/null-null.pom 403 Forbidden -> [Help 1]
12:31:20 [ERROR]
12:31:20 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
12:31:20 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
12:31:20 [ERROR]
12:31:20 [ERROR] For more information about the errors and possible solutions, please read the following articles:
12:31:20 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
12:31:20 [ERROR]
12:31:20 [ERROR] After correcting the problems, you can resume the build with the command
12:31:20 [ERROR] mvn -rf :jobs-core
12:31:20 [Pipeline] }
12:31:20 [Pipeline] // stage
12:31:20 [Pipeline] }
12:31:20 [Pipeline] // dir
12:31:20 [Pipeline] }
12:31:20 [Pipeline] // withEnv
12:31:20 [Pipeline] }
12:31:20
12:31:21 [Pipeline] // ansiColor
12:31:21 [Pipeline] }
12:31:21 [Pipeline] // node
12:31:21 [Pipeline] End of Pipeline
12:31:21 ERROR: script returned exit code 1
12:31:21 Finished: FAILURE

[krgauraw]

Hi @Arunkumarpodili,
Did you re-import the job using https://github.com/Sunbird-inQuiry/data-pipeline/tree/release-5.7.0_RC4/scripts/jenkins-jobs
?
From above logs, it is clear that cloud-storage-sdk parameters are not passed from jenkins build job and same is causing build failure.

Please try deleting the existing job and re-import all 3 jobs. Please let me know, if you still face any issues

cc: @sajeshkayyath @PoojaJManjooran