Add in the ability to fingerprint JSON columns [databricks]

[revans2]

This is the same as #11002 but it was reverted because it broke databricks

[revans2]

@gerashegalov and @tgravescs can you help me out here with the build and let me know if I am seeing thing incorrectly?

I am getting an error trying to use classes in the jackson-core dependency. I have verified that the classes I need are in jackson-core-.jar, but even though it should be on the classpath it appears not to be. But the pom.xml files are complicated and I am not 100% sure that it is on the classpath, as that is not in the logs for the build.

When I look at the logs of databricks installing things I see jackson-core is being pulled in as jackson-databind.

[2024-06-13T10:37:08.109Z] Generating an execution for com.fasterxml.jackson.core jackson-core ----ws_3_3--mvn--hadoop3--com.fasterxml.jackson.core--jackson-databind--com.fasterxml.jackson.core__jackson-databind__*.jar
...
[2024-06-13T10:37:11.329Z] [INFO] --- maven-install-plugin:2.4:install-file (install-db-jar-27) @ rapids-4-spark-databricks-deps-installer ---
[2024-06-13T10:37:11.329Z] [INFO] Installing /databricks/jars/----ws_3_3--mvn--hadoop3--com.fasterxml.jackson.core--jackson-databind--com.fasterxml.jackson.core__jackson-databind__2.13.4.jar to /home/ubuntu/.m2/repository/com/fasterxml/jackson/core/jackson-core/3.3.0-databricks/jackson-core-3.3.0-databricks.jar
[2024-06-13T10:37:11.329Z] [INFO] Installing /tmp/mvninstall8760060792013558718.pom to /home/ubuntu/.m2/repository/com/fasterxml/jackson/core/jackson-core/3.3.0-databricks/jackson-core-3.3.0-databricks.pom

but jackson-databind is not jackson-core. If you run mvn dependency:tree on the datagen pom.xml you get

...
[INFO] |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.0:provided
[INFO] |  |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.0:provided
[INFO] |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.10.0:provided
...

and the contents of the jars are different. Is this a bug in our script or has databricks actually combined the jars together?

[revans2]

spark-rapids/jenkins/databricks/install_deps.py

Lines 118 to 119 in 900ae6f

	Artifact('com.fasterxml.jackson.core', 'jackson-core',
	f'{prefix_ws_sp_mvn_hadoop}--com.fasterxml.jackson.core--jackson-databind--com.fasterxml.jackson.core__jackson-databind__*.jar'),

is the line that I think is a problem.

[revans2]

build

[gerashegalov]

LGTM

[revans2]

build

[revans2]

It got past the build on databricks so that was the problem :)

[revans2]

build

[sameerz]

Can this be marked as closing issue #11053 ?

[revans2]

Can this be marked as closing issue #11053 ?

That is up to you on how you want to track it. I reverted the original patch so that might have fixed it. This puts the functionality back in, but "fixed" so that is fine too.