Add notebook for using qualification user tools in Databricks

[parthosa]

Fixes NVIDIA/spark-rapids-tools#571. This PR adds a new example notebook for using qualification user tools in Databricks

Changes:

1. Added a new dropdown field csp to choose between aws and azure (defaults to aws).
2. Create a virtual environment and install spark-rapids-user-tools
3. Run qualification on user tools.
4. Parse the logs to store the output folder as a variable.

Output:

Signed-off-by: Partho Sarthi [email protected]

[NvTimLiu]

LGTM

[nvliyuan]

Hi @parthosa , thanks for the contributing, could you help re-target the pr to branch-23.12?

The base branch was changed.

[nvliyuan]

Hi @parthosa , seems the branch is still main.
And is the notebook specific to some Databricks runtime version?
I hit below error while running on 10.4ML:

[nvliyuan]

I hit below error while running on 10.4ML:

The error will not show up if I run the cell for the second time. Another issue is FileNotFoundError, I checked the folder and confirmed there are output files in the dbfs, any idea?

[parthosa]

Thanks @nvliyuan. I have updated the base branch to branch-23.12 and fixed the issues related to output folder: The notebook now creates a separate virtual environment and installs spark-rapids-user-tools package in it. See description for updated UI.

[nvliyuan]

LGTM