-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add reservation metics and exportType for costs #87
Merged
Merged
Changes from all commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
131c72e
Added data in ISO format as label
jkroepke 3e6a82c
update deps
jkroepke 3b98fe5
Merge pull request #1 from PaulPowershell/date-iso
PaulPowershell e9e405a
Merge pull request #2 from PaulPowershell/deps
PaulPowershell d9e2a54
Update metrics_azurerm_costs.go
kevindelmont 1346d42
Update config_cost.go
kevindelmont e5738f0
Update example.yaml
kevindelmont 3127cd0
Merge pull request #3 from PaulPowershell/feat/ExportType
kevindelmont a84dab9
push fix to kevin PR
PaulPowershell 5d81d56
fix error L190
PaulPowershell a036bc4
Merge branch 'main' into main
PaulPowershell 70111d4
commit
PaulPowershell 66e2d1b
add reservation
PaulPowershell 5eb00b5
fix appel func reservation
PaulPowershell 4018575
delete old comment
PaulPowershell 00eb8a5
push change
PaulPowershell 10c8941
remove subscription foreach
PaulPowershell ec3dcf7
add more properties & yaml
PaulPowershell ba9245f
syntaxe
PaulPowershell 7e91573
fix utilization
PaulPowershell c27eb0f
add last change
PaulPowershell 0da8ebf
rollback
PaulPowershell 78a20a9
change value
PaulPowershell 1c4f1bb
fix date
PaulPowershell 41b93bb
add metrics
PaulPowershell e32b4cc
rollback
PaulPowershell a6e889c
decimal 3 to 2
PaulPowershell d93ce05
add customDays
PaulPowershell ebaa793
change enddate
PaulPowershell 8f293e5
commit
PaulPowershell dc36847
change authorizer
PaulPowershell 4a9f71b
fix S1025
PaulPowershell db1175a
trad comment in english
PaulPowershell 56cbd82
Merge pull request #4 from PaulPowershell/add-reservation
PaulPowershell aee531b
Merge branch 'main' into main
PaulPowershell febb17b
fix module go armcompute
PaulPowershell c7a70ee
revert example.yaml
mblaschke d226258
lowercase collector name
mblaschke dc219ce
remove custom backoff for reservations
mblaschke 7069695
refactoring
mblaschke ad4ed84
Update metrics_azurerm_resources.go
mblaschke 59d6d91
fix build
mblaschke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,4 @@ | ||
| /vendor/ | ||
| /azure-resourcemanager-exporter* | ||
| /release-assets | ||
| /.vscode |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| package config | ||
|
|
||
| type ( | ||
| CollectorReservation struct { | ||
| CollectorBase `yaml:",inline"` | ||
|
|
||
| ResourceScope string `yaml:"resourceScope"` | ||
| Granularity string `yaml:"granularity"` | ||
| FromDays int `yaml:"FromDays"` | ||
| } | ||
| ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,6 +23,8 @@ collectors: | |
|
|
||
| costs: {} | ||
|
|
||
| reservation: {} | ||
|
|
||
| portscan: | ||
| scanner: | ||
| parallel: 2 | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,216 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "time" | ||
|
|
||
| "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/consumption/armconsumption" | ||
| "github.com/prometheus/client_golang/prometheus" | ||
| "github.com/webdevops/go-common/prometheus/collector" | ||
| "github.com/webdevops/go-common/utils/to" | ||
| "go.uber.org/zap" | ||
| ) | ||
|
|
||
| // Define MetricsCollectorAzureRmReservation struct | ||
| type MetricsCollectorAzureRmReservation struct { | ||
| collector.Processor | ||
|
|
||
| prometheus struct { | ||
| reservationInfo *prometheus.GaugeVec | ||
| reservationUsage *prometheus.GaugeVec | ||
| reservationMinUsage *prometheus.GaugeVec | ||
| reservationMaxUsage *prometheus.GaugeVec | ||
| reservationUsedHours *prometheus.GaugeVec | ||
| reservationReservedHours *prometheus.GaugeVec | ||
| reservationTotalReservedQuantity *prometheus.GaugeVec | ||
| } | ||
| } | ||
|
|
||
| // Setup method to initialize Prometheus metrics | ||
| func (m *MetricsCollectorAzureRmReservation) Setup(collector *collector.Collector) { | ||
| m.Processor.Setup(collector) | ||
|
|
||
| m.prometheus.reservationInfo = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_info", | ||
| Help: "Azure ResourceManager Reservation Information", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.prometheus.reservationUsage = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_utilization", | ||
| Help: "Azure ResourceManager Reservation Utilization", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.prometheus.reservationMinUsage = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_utilization_min", | ||
| Help: "Azure ResourceManager Reservation Min Utilization", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.prometheus.reservationMaxUsage = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_utilization_max", | ||
| Help: "Azure ResourceManager Reservation Max Utilization", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.prometheus.reservationUsedHours = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_used_hours", | ||
| Help: "Azure ResourceManager Reservation Used Hours", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.prometheus.reservationReservedHours = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_reserved_hours", | ||
| Help: "Azure ResourceManager Reservation Reserved Hours", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.prometheus.reservationTotalReservedQuantity = prometheus.NewGaugeVec( | ||
| prometheus.GaugeOpts{ | ||
| Name: "azurerm_reservation_total_reserved_quantity", | ||
| Help: "Azure ResourceManager Reservation Total Reserved Quantity", | ||
| }, | ||
| []string{ | ||
| "reservationOrderID", | ||
| "reservationID", | ||
| "skuName", | ||
| "kind", | ||
| "usageDate", | ||
| }, | ||
| ) | ||
|
|
||
| m.Collector.RegisterMetricList("reservationInfo", m.prometheus.reservationInfo, true) | ||
| m.Collector.RegisterMetricList("reservationUsage", m.prometheus.reservationUsage, true) | ||
| m.Collector.RegisterMetricList("reservationMinUsage", m.prometheus.reservationMinUsage, true) | ||
| m.Collector.RegisterMetricList("reservationMaxUsage", m.prometheus.reservationMaxUsage, true) | ||
| m.Collector.RegisterMetricList("reservationUsedHours", m.prometheus.reservationUsedHours, true) | ||
| m.Collector.RegisterMetricList("reservationReservedHours", m.prometheus.reservationReservedHours, true) | ||
| m.Collector.RegisterMetricList("reservationTotalReservedQuantity", m.prometheus.reservationTotalReservedQuantity, true) | ||
| } | ||
|
|
||
| func (m *MetricsCollectorAzureRmReservation) Reset() {} | ||
|
|
||
| func (m *MetricsCollectorAzureRmReservation) Collect(callback chan<- func()) { | ||
| m.collectReservationUsage(logger, callback) | ||
| } | ||
|
|
||
| func (m *MetricsCollectorAzureRmReservation) collectReservationUsage(logger *zap.SugaredLogger, callback chan<- func()) { | ||
| reservationInfo := m.Collector.GetMetricList("reservationInfo") | ||
| reservationUsage := m.Collector.GetMetricList("reservationUsage") | ||
| reservationMinUsage := m.Collector.GetMetricList("reservationMinUsage") | ||
| reservationMaxUsage := m.Collector.GetMetricList("reservationMaxUsage") | ||
| reservationUsedHours := m.Collector.GetMetricList("reservationUsedHours") | ||
| reservationReservedHours := m.Collector.GetMetricList("reservationReservedHours") | ||
| reservationTotalReservedQuantity := m.Collector.GetMetricList("reservationTotalReservedQuantity") | ||
|
|
||
| ctx := context.Background() | ||
| days := Config.Collectors.Reservation.FromDays | ||
| resourceScope := Config.Collectors.Reservation.ResourceScope | ||
| granularity := Config.Collectors.Reservation.Granularity | ||
|
|
||
| now := time.Now() | ||
| formattedDate := now.AddDate(0, 0, -days).Format("2006-01-02") | ||
| startDate := formattedDate | ||
| endDate := time.Now().Format("2006-01-02") | ||
|
|
||
| clientFactory, err := armconsumption.NewClientFactory("<subscription-id>", AzureClient.GetCred(), AzureClient.NewArmClientOptions()) | ||
| if err != nil { | ||
| logger.Panic(err) | ||
| } | ||
|
|
||
| // "Create a pager to retrieve daily booking summaries | ||
| pager := clientFactory.NewReservationsSummariesClient().NewListPager(resourceScope, armconsumption.Datagrain(granularity), &armconsumption.ReservationsSummariesClientListOptions{ | ||
| StartDate: to.Ptr(startDate), | ||
| EndDate: to.Ptr(endDate), | ||
| Filter: nil, | ||
| ReservationID: nil, | ||
| ReservationOrderID: nil, | ||
| }) | ||
|
|
||
| // Collect and export metrics | ||
| for pager.More() { | ||
| page, err := pager.NextPage(ctx) | ||
| if err != nil { | ||
| logger.Panic(err) | ||
| } | ||
|
|
||
| for _, reservationProperties := range page.Value { | ||
| reservationOrderID := reservationProperties.Properties.ReservationOrderID | ||
| reservationID := reservationProperties.Properties.ReservationID | ||
| skuName := reservationProperties.Properties.SKUName | ||
| kind := reservationProperties.Properties.Kind | ||
| reservedHours := reservationProperties.Properties.ReservedHours | ||
| usageDate := reservationProperties.Properties.UsageDate.String() | ||
| usedHours := reservationProperties.Properties.UsedHours | ||
| minUtilizationPercentage := reservationProperties.Properties.MinUtilizationPercentage | ||
| avgUtilizationPercentage := reservationProperties.Properties.AvgUtilizationPercentage | ||
| maxUtilizationPercentage := reservationProperties.Properties.MaxUtilizationPercentage | ||
| totalReservedQuantity := reservationProperties.Properties.TotalReservedQuantity | ||
|
|
||
| labels := prometheus.Labels{ | ||
| "reservationOrderID": to.String(reservationOrderID), | ||
| "reservationID": to.String(reservationID), | ||
| "skuName": to.String(skuName), | ||
| "kind": to.String(kind), | ||
| "usageDate": usageDate, | ||
| } | ||
|
|
||
| reservationInfo.AddInfo(labels) | ||
| reservationUsage.AddIfNotNil(labels, avgUtilizationPercentage) | ||
| reservationMinUsage.AddIfNotNil(labels, minUtilizationPercentage) | ||
| reservationMaxUsage.AddIfNotNil(labels, maxUtilizationPercentage) | ||
| reservationUsedHours.AddIfNotNil(labels, usedHours) | ||
| reservationReservedHours.AddIfNotNil(labels, reservedHours) | ||
| reservationTotalReservedQuantity.AddIfNotNil(labels, totalReservedQuantity) | ||
| } | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't
<subscription-id>not set to the subscription id? and loop trough every subscription id in the collect function?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, thanx for your return,
It is surprising, but in fact the value should not be defined because this metrics_azurerm_reservation.go is intended to be used from an MCA tenant (which operates by billing profile rather than by subscription). Therefore, leaving "" works well. We could rename the field but not set it to nil.