Qualification tool: Add more information for unsupported operators

[nartal1]

This PR fixes #626 and fixes #656

In 626 - Output file had duplicate values once for Exec and another for Expressions.

Exec" "Filter" ""
"Exec" "Filter" "Filter Exec is not supported as expressions are not supported -  `AtLeastNNulls`"

This PR deduplicates the Exec in output files. i.e if an Exec is not supported due to unsupported expressions, then output will contain Exec name along with unsupported expressions. This PR also identifies UDF and dataset operations and includes that as the reason for the unsupported Exec. Also adds a new column "Ignore Operator" which indicates whether the operator is something like a DDL, Table operation which can be ignored. This helps to make sure we document all the unsupported operators in the output file but we can filter the operators to be prioritized based on "Ignore Operator" column.

Sample Output:

App ID,Unsupported Type,Details,Notes,Ignore Operator
"application_1700986078276_0001","Exec","LocalTableScan","","true"
"application_1700986078276_0001","Exec","AdaptiveSparkPlan","","true"
"application_1700986078276_0001","Exec","Scan hive cust2cc_v1.stats","","false"
"application_1700986078276_0001","Exec","SetCatalogAndNamespace","","true"
"application_1700986078276_0001","Exec","Execute CreateDataSourceTableAsSelectCommand","","true"
"application_1700986078276_0001","Exec","Filter","Filter Exec is not supported as this operation is from dataset which is not supported","false"
"application_1700992056158_0001","Exec","BatchEvalPython","BatchEvalPython Exec is not supported as it contains UDF which is not supported","false"
"application_1700992056158_0001","Exec","Filter","Filter Exec is not supported as it contains UDF which is not supported","false"
"application_1700992056158_0001","Exec","Project","Project Exec is not supported as expressions are not supported -  `normalizeIp`","false"
"application_1700992056158_0001","Exec","Filter","Filter Exec is not supported as expressions are not supported -  `validateIP;normalizeIp`","false"
"application_1700992056158_0001","Exec","SortMergeJoin","SortMergeJoin Exec is not supported as expressions are not supported -  `normalizeEmail`","false"
"application_1700992056158_0001","Exec","Sort","Sort Exec is not supported as expressions are not supported -  `normalizeEmail`","false"
"application_1700992056158_0001","Expression","normalizeName","","false"
"application_1700992056158_0001","Expression","normalizeEmail","","false"

For 656 - Generates new output file (rapids_4_spark_qualification_output_unsupportedOperatorsStageDuration.csv) based on comment - #656 (comment) . This file helps to determine wallclockStageDuration to appDuration ratio and corresponding unsupported operators in that stage. This helps in prioritizing the unsupported opeators.

Sample output:

App ID,Unsupported Type,Stage ID,Stage Duration,App Duration,Recommendation,Ignore Operator
"application_1700992056158_0001","LocalTableScan",65,4323,1691909,Recommended,true
"application_1700992056158_0001","Execute InsertIntoHiveTable",107,3940,1691909,Recommended,false
"application_1700992056158_0001","BatchEvalPython",81,141962,1691909,Recommended,false
"application_1700992056158_0001","Filter",81,141962,1691909,Recommended,false
"application_1700992056158_0001","BatchEvalPython",81,141962,1691909,Recommended,false
"application_1700992056158_0001","Project",81,141962,1691909,Recommended,false
"application_1700992056158_0001","Filter",7,77,1691909,Recommended,false
"application_1700992056158_0001","LocalTableScan",7,77,1691909,Recommended,true

[parthosa]

Thanks @nartal1.

[nartal1]

Thanks @parthosa for the review. I updated the PR based on @viadea 's offline feedback. I have updated the description. PTAL.
@viadea - Would be good to get your feedback.

[amahussein]

Thanks @nartal1