Sane Dropwizard metrics instrumentation for InfluxDB 1.2+.
> show series
key
---
client_connections,client=influxdb-http-writer
connections,port=8080
connections,port=8081
http_server
http_server,metric=1xx-responses
http_server,metric=2xx-responses
http_server,metric=3xx-responses
http_server,metric=4xx-responses
http_server,metric=5xx-responses
http_server,metric=async-dispatches
http_server,metric=async-timeouts
http_server,metric=connect-requests
http_server,metric=delete-requests
http_server,metric=dispatches
http_server,metric=get-requests
http_server,metric=head-requests
http_server,metric=move-requests
http_server,metric=options-requests
http_server,metric=other-requests
http_server,metric=post-requests
http_server,metric=put-requests
http_server,metric=requests
http_server,metric=trace-requests
jvm
jvm_buffers
jvm_buffers,type=direct
jvm_buffers,type=mapped
jvm_classloader
jvm_gc
jvm_memory,metric=heap
jvm_memory,metric=non-heap
jvm_memory,metric=pools.Code-Cache
jvm_memory,metric=pools.Compressed-Class-Space
jvm_memory,metric=pools.Metaspace
jvm_memory,metric=pools.PS-Eden-Space
jvm_memory,metric=pools.PS-Old-Gen
jvm_memory,metric=pools.PS-Survivor-Space
jvm_memory,metric=total
jvm_threads
logging,level=all
logging,level=debug
logging,level=error
logging,level=info
logging,level=trace
logging,level=warn
resources,method=get,resource=AdminResource
thread_pools,pool=dwTo begin scheduling metric reports at a regular interval, add a reporter to your Dropwizard config. A barebones reporter will look like this:
metrics:
reporters:
- type: influxdb
sender:
type: http
database: mydbThis will send metrics to an InfluxDB instance on localhost:8086 over HTTP.
A more complex sender may look like this:
- type: influxdb
globalTags:
env: production
metricTemplates:
services:
pattern: com\.kickstarter\.services\.(?<service>[A-Za-z]+).*
tagKeys: ["service"]
groupGauges: true
groupCounters: false
sender:
type: tcp
host: localhost
port: 90210
timeout: 5000000 days # keep alive as long as possibleAdd tags to everything that gets reported.
globalTags:
env: production
service: recommendationsGroup gauges and counters together under a single measurement. Enabled by default.
<"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.jobs" value=0>
<"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.size" value=4>
<"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.utilization" value=0.455>
<"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.utilization-max" value=0.0068301848>
// => Output
name: org.eclipse.jetty.util.thread.QueuedThreadPool.dw
-------------------------------------------------------
time jobs size utilization utilization-max
2017-06-10T10:18:00Z 0 4 0.455 0.0068301848groupGauges: true
groupCounters: trueUse custom templating to convert dropwizard-style metric names into reasonable measurement names and tag sets. By default, the reporter comes with its own templates. Thread pool metrics, for example, are renamed under thread_pools:
name: thread_pools
-------------------------------------------------------
time jobs size utilization utilization-max
2017-06-10T10:18:00Z 0 4 0.455 0.0068301848
While timed Dropwizard resource metrics, for example, are grouped and tagged under resources:
resources,resource=InfoResource,method=stats
resources,resource=RecommendedProjectsResource,method=get
resources,resource=BatchRecommendedProjectsResource,method=get
metricTemplates:
services:
pattern: com\.kickstarter\.services\.(?<service>[A-Za-z]+).*
tagKeys: ["service"]The reporter is able to deserialize custom InfluxDb-style measurements passed to it via Dropwizard's instrumentation layer. This allows you to fully customize the InfluxDB output of a particular metric through Dropwizard.
final Timer restoreTimer = metricRegistry.timer(
influxName("Recommendations", ImmutableMap.of(
"action", "restore",
"model", "fun-model"
))
);
restoreTimer.time(...)
// => Output
name: Recommendations,action=restore,model=fun-model
-------------------------------------------------------
50-percentile 75-percentile 95-percentile 99-percentile ...The HTTP sender transmits InfluxDB lines directly to the database, and must provide a database for usage. It uses a Jersey client to send metrics to InfluxDB, giving us some request-level timing measurements.
The jersey client can be customized using all of the options provided by JerseyClientConfiguration.
metrics:
reporters:
type: influxdb
sender:
type: http
host: localhost
port: 8086
database: mydb
jersey:
connectionTimeout: 500 millisecondsYou may wish to send InfluxDB lines to a collector instance, like Telegraf, instead of using the direct HTTP protocol. You can use the TCP sender to transmit metrics to your collector in InfluxDB line format.
metrics:
reporters:
type: influxdb
sender:
type: tcp
host: localhost
port: 8086
timeout: 500 millisecondsA Sender sends a batch of InfluxDbMeasurements to a receiver at the Dropwizard-configured frequency. If the sender catches an exception while writing to the receiver, the exception is logged and the connection is closed. The sender will reconnect to the receiver when the next batch is scheduled to be sent.
The measurements that failed to send are stored in a queue and retried in subsequent batches. When things get real bad™️ and the queue gets backed up, we'll start dropping old metrics — this logic is all handled by Guava's EvictingQueue.
Have questions or feedback? The best way to submit feedback and report bugs is to open a GitHub issue. We'd love to see you contribute — talk to you soon!
Copyright 2017 Kickstarter, PBC.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.