Adding GC Metrics #12
Conversation
Signed-off-by: Sayed Bilal Bari <[email protected]>
| private def getGcMetrics: (Any, Any) = { | ||
| val gcBeans = ManagementFactory.getGarbageCollectorMXBeans | ||
| val gcTime = gcBeans.map(_.getCollectionTime).sum | ||
| val gcCount = gcBeans.map(_.getCollectionCount).sum | ||
| (gcTime, gcCount) | ||
| } | ||
|
|
There was a problem hiding this comment.
If we move this to the RuntimeUtil, then we can use any where as a utility function regardless it is part of the benchmark or not.
Also, we can return a case class
case class JVMGCInfo(val gcCount: Long, val gcTime: Long, val: totalHeap: Long, val usedHeap: Long, val freeHeap: Long)
After we do that we can also define another object to manage snapshots (similar to the timer)
But we probably need another manager to handle start/end to return the diff instead of flatenning the code. We can do the manager class together after you update the PR.
There was a problem hiding this comment.
Added changes to handle the above. Added memory tracker class. Also added tracking of average used memory, average free memory and max heap allocation
Signed-off-by: Sayed Bilal Bari <[email protected]>
Signed-off-by: Sayed Bilal Bari <[email protected]>
Signed-off-by: Sayed Bilal Bari <[email protected]>
| @@ -99,18 +99,31 @@ class Benchmark( | |||
|
|
|||
| val firstBest = results.head.bestMs | |||
| // The results are going to be processor specific so it is useful to include that. | |||
| out.println(RuntimeUtil.getJVMOSInfo.mkString("\n")) | |||
| val jvmInfo = RuntimeUtil.getJVMOSInfo | |||
| out.printf(s"%-26s -> %-30s \n","JVM Name", jvmInfo("jvm.name")) | |||
There was a problem hiding this comment.
let's make it more standard : instead of ->
| out.printf(s"%-26s -> %-30s \n","Java Version",jvmInfo("jvm.version")) | ||
| out.printf(s"%-26s -> %-30s \n","OS Name",jvmInfo("os.name")) | ||
| out.printf(s"%-26s -> %-30s \n","OS Version",jvmInfo("os.version")) | ||
| out.printf(s"%-26s -> %-30s \n","MaxHeapMemory", ("%6d" format Runtime.getRuntime.maxMemory()/1024/1024)+"MB") |
There was a problem hiding this comment.
%6d will cause the string not to be aligned.
| out.printf(s"%-26s -> %-30s \n","OS Name",jvmInfo("os.name")) | ||
| out.printf(s"%-26s -> %-30s \n","OS Version",jvmInfo("os.version")) | ||
| out.printf(s"%-26s -> %-30s \n","MaxHeapMemory", ("%6d" format Runtime.getRuntime.maxMemory()/1024/1024)+"MB") | ||
| out.printf(s"%-26s -> %-30s \n","Total Warm Up Iterations","%2d" format warmUpIterations) |
| out.printf(s"%-26s -> %-30s \n","OS Version",jvmInfo("os.version")) | ||
| out.printf(s"%-26s -> %-30s \n","MaxHeapMemory", ("%6d" format Runtime.getRuntime.maxMemory()/1024/1024)+"MB") | ||
| out.printf(s"%-26s -> %-30s \n","Total Warm Up Iterations","%2d" format warmUpIterations) | ||
| out.printf(s"%-26s -> %-30s \n \n","Total Runtime Iterations","%2d" format minNumIters) |
| @@ -27,23 +27,28 @@ import com.nvidia.spark.rapids.tool.qualification.QualificationMain.mainInternal | |||
| * 2. Write the benchmark code in the runBenchmark method passing relevant arguments | |||
| * 3. Write benchmarked code inside | |||
| */ | |||
| object QualificationBenchmark extends BenchmarkBase { | |||
| object QualificationToolBenchmark extends BenchmarkBase { | |||
There was a problem hiding this comment.
we can call this SingleThreadedQualTool and set the number of threads to be 1.
In tht case, when someone is running the benchmark with multiple eventlogs, it will be forced to be single threaded.
Signed-off-by: Sayed Bilal Bari <[email protected]>
No description provided.