diff --git a/docs/EN_US/ContainerizedHPCC/ContainerizedMods/CustomConfig.xml b/docs/EN_US/ContainerizedHPCC/ContainerizedMods/CustomConfig.xml index 229e26f52e0..13a51261c2e 100644 --- a/docs/EN_US/ContainerizedHPCC/ContainerizedMods/CustomConfig.xml +++ b/docs/EN_US/ContainerizedHPCC/ContainerizedMods/CustomConfig.xml @@ -1,5 +1,5 @@ - Customizing Configurations @@ -865,14 +865,23 @@ thor: [] using cloud native HPCC Systems platform instances in a cloud setting where some planning and configuration can help reduce expenses. - Two new columns have been added to the workunits page in ECL Watch. - The columns may be sorted by either cost column, just like the other - columns in ECL Watch, by clicking at the top of the column. + New columns have been added to the workunits and the logical files + pages in ECL Watch. These columns can be sorted by any cost columns, just + like the other columns in ECL Watch, by clicking at the top of the column. + In addition, file operation costs executed by workunits are provided in + the workunit’s metrics. + + As the cost tracking matures, the cost calculations also improve + significantly providing more accurate cost tracking. For example, data + that was accessed from page cache doesn’t incur any file access cost. HPCC + Systems cost tracking now detects data that has been returned from the + page cache and adjusts cost calculations appropriately resulting in more + accurate cost calculations. Types of Costs - There are three types of costs that are tracked. + There are several types of costs that are tracked. @@ -880,12 +889,16 @@ thor: [] - Storage Cost + Compile Cost File Access Cost + + + File Cost at Rest + @@ -893,9 +906,10 @@ thor: [] NOTE: - All cost values calculated and displayed are approximate. - There are many variables that can result in inaccuracies. These - cost values are only intended to be used as a guide. + While reporting accuracy has significantly improved, all + cost values calculated and displayed remain approximate. There are + many variables that can result in inaccuracies. These cost values + are only intended to be used as a guide. @@ -962,12 +976,22 @@ thor: [] + + Compile Cost + + Compile costs are the costs of compiling the ECL code. The cost + of compiling the ECL code is included as a column in the workunit + list. In the workunit summary page there is a compile cost + field. + + - Storage Cost + Storage Costs - This is the cost of hosting the data in the storage plane. It - does not include the costs of data operations such as read or write - costs. + This is the cost of hosting the data in the storage plane or the + File Cost At Rest value and the cost of + reading/writing to the storage, is the File Access + Cost value. @@ -981,37 +1005,49 @@ thor: [] - The storage costs cannot be viewed as a separate value in ECL - Watch. They can only be viewed as part of a cost field in a logical - file’s summary page. That cost field includes other file related - costs. - - - - File Access Cost + For logical files, the costs calculated are based on two types + of file access. - File access costs are the costs of reading and writing to the - files. Many storage planes do have a separate charge for data - operations. The cost of reading and writing to the file will be - included in the file access cost value. Any other cost associated with - file operations (such as delete or copy) will not be tracked or - included as part of file access cost at this time. - - The file access cost displays as part of the cost field on the - Logical File’s summary page in ECL Watch. - - The costs incurred by a workunit for accessing logical files is - also recorded in the workunit’s statistics and attributes. The - read/write cost is recorded at the activity record and cumulated at - the graph, the subgraph and the workflow scope level. The total file - access cost for a workunit is recorded with the workunit and displayed - in the summary page. - - The new cost field is shown in the Logical File summary page. It - is the combined cost of storing and accessing the data. + + + File Access Cost + - The cost information is currently only generated for Thor and - hThor jobs. + + File Cost at Rest + + + + + File Access Cost + + File access costs are the costs of reading and writing to the + files. Many storage planes do have a separate charge for data + operations. The cost of reading and writing to the file is included + in the file access cost value. Any other cost associated with file + operations (such as delete or copy) will not be tracked or included + as part of file access cost at this time. + + The costs incurred by a workunit for accessing logical files + is also recorded in the workunit’s statistics and attributes. The + read/write cost is recorded at the activity record and cumulated at + the graph, the subgraph, and the workflow scope level. Cost Tracking + detects data that has been returned from the page cache and adjusts + cost calculations appropriately to produce more accurate cost + calculations. These file access costs for a workunit are recorded + with the workunit and displayed in the summary page and in the + workunit’s metrics. + + + File Cost at Rest + + The File Cost at Rest field is shown in the Logical File + summary page. It is the cost of storing the file without accessing + the data. Only the storage costs associated with housing the file + in the cloud. This value has been added to better differentiate + between file storage costs. + + @@ -1074,9 +1110,7 @@ thor: [] - - - + @@ -1154,9 +1188,9 @@ thor: [] - + - + @@ -1326,7 +1360,7 @@ thor: [] ldapAdminSecretKey: "additionaltmountname" # use alternate secrets creds - + @@ -1401,7 +1435,7 @@ thor: [] ldapAdminVaultId: "my-authn-vault" - +