Fix cluster recommendation when CPU cluster cannot be determined

[parthosa]

This PR fixes the issue where we do not generate a cluster recommendation when CPU cluster cannot be created (e.g., no matching executor instance found for the required number of cores).

Approach

Scala tool now generates a recommended GPU cluster per app basis NVIDIA/spark-rapids-tools#1188. For the case when CPU cluster is not provided, we should use the values from Scala tool output for our GPU cluster recommendation instead of python's cpu<->gpu core matching.

Output

Case 1: CPU cluster is not passed and we infer CPU cluster for each app

Logs (for each app):

INFO rapids.tools.cluster_inference: For App ID: app-20200423033538-0000, Unable to infer CPU cluster. Reason - No matching executor instance found for num cores = 80
INFO rapids.tools.cluster_recommender: For App ID: app-20200423033538-0000, CPU cluster: N/A; Recommended GPU cluster: <Driver: m6gd.xlarge, Executor: 16 X g5.4xlarge>
INFO rapids.tools.cluster_recommender: For App ID: app-20210509200722-0001, Inferred CPU cluster: <Driver: m6gd.xlarge, Executor: 1 X m6gd.4xlarge>; Recommended GPU cluster: <Driver: m6gd.xlarge, Executor: 1 X g5.4xlarge>

Final Result:

+----+---------------------+-------------------------+-----------------+----------------------------+------------------------------+-----------------------------+
|    | App Name            | App ID                  | Estimated GPU   | Qualified Node             | Full Cluster                 | GPU Config                  |
|    |                     |                         | Speedup         | Recommendation             | Config                       | Recommendation              |
|    |                     |                         | Category**      |                            | Recommendations*             | Breakdown*                  |
|----+---------------------+-------------------------+-----------------+----------------------------+------------------------------+-----------------------------|
|  1 | spark_test_apps.py  | app-20200423033538-0000 | Large           | g5.4xlarge                 | app-20200423033538-0000.conf | app-20200423033538-0000.log |
|  2 | Spark shell         | app-20210509200722-0001 | Small           | m6gd.4xlarge to g5.4xlarge | app-20210509200722-0001.conf | app-20210509200722-0001.log |
+----+---------------------+-------------------------+-----------------+----------------------------+------------------------------+-----------------------------+

Case 2: CPU cluster is passed as input (--cluster <cluster>)

Logs (for all apps):

INFO rapids.tools.cluster_recommender: CPU cluster: <Driver: m6gd.xlarge, Executor: 2 X m6gd.xlarge>; Recommended GPU cluster: <Driver: m6gd.xlarge, Executor: 2 X g5.xlarge>

Final Result:

+----+---------------------+-------------------------+-----------------+--------------------------+------------------------------+-----------------------------+
|    | App Name            | App ID                  | Estimated GPU   | Qualified Node           | Full Cluster                 | GPU Config                  |
|    |                     |                         | Speedup         | Recommendation           | Config                       | Recommendation              |
|    |                     |                         | Category**      |                          | Recommendations*             | Breakdown*                  |
|----+---------------------+-------------------------+-----------------+--------------------------+------------------------------+-----------------------------|
|  1 | spark_test_apps.py  | app-20200423033538-0000 | Large           | m6gd.xlarge to g5.xlarge | app-20200423033538-0000.conf | app-20200423033538-0000.log |
|  2 | Spark shell         | app-20210509200722-0001 | Small           | m6gd.xlarge to g5.xlarge | app-20210509200722-0001.conf | app-20210509200722-0001.log |
+----+---------------------+-------------------------+-----------------+--------------------------+------------------------------+-----------------------------+