Reports job metrics using Shopify's statsd-instrument library, incrementing a counter for each enqueue and dequeue per job type, and timing the full runtime of your perform method.
Add this line to your application's Gemfile:
gem 'sidekiq-instrument'
And then execute:
$ bundle
Or install it yourself as:
$ gem install sidekiq-instrument
For now, this library assumes you have already initialized StatsD
on your own;
the statsd-instrument
gem may have chosen reasonable defaults for you already. If not,
a typical Rails app would just use an initializer:
# config/initializers/statsd.rb
require 'statsd-instrument'
StatsD.prefix = 'my-app'
StatsD.backend = StatsD::Instrument::Backends::UDPBackend.new('some-server:8125')
Then add the client and server middlewares in your Sidekiq initializer:
require 'sidekiq/instrument'
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.add Sidekiq::Instrument::ServerMiddleware
end
config.client_middleware do |chain|
chain.add Sidekiq::Instrument::ClientMiddleware
end
end
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add Sidekiq::Instrument::ClientMiddleware
end
end
For each job, the following metrics will be reported:
- shared.sidekiq.queue.job.enqueue: counter incremented each time a job is pushed onto the queue.
- shared.sidekiq.queue.job.dequeue: counter incremented just before worker begins performing a job.
- shared.sidekiq.queue.job.runtime: timer of the total time spent
in
perform
, in milliseconds. - shared.sidekiq.queue.job.error: counter incremented each time a job fails.
The metric names can be changed by overriding the statsd_metric_name
method in your worker classes.
For each queue, the following metrics will be reported:
- shared.sidekiq.queue.size: gauge of how many jobs are in the queue
- shared.sidekiq.queue.latency: gauge of how long the oldest job has been in the queue
There is a worker, Sidekiq::Instrument::Worker
, that submits gauges
for various interesting statistics; namely, the bulk of the information in Sidekiq::Stats
and the sizes of each individual queue. While the worker class is a fully valid Sidekiq worker,
you should inherit from it your own job implementation instead of using it directly:
# app/jobs/sidekiq_stats_job.rb
class SidekiqStatsJob < Sidekiq::Instrument::Worker
METRIC_NAMES = %w[
processed
failed
]
sidekiq_options queue: :stats
end
In this example, we override the default stats with the ones we want reported by defining METRIC_NAMES
.
This can be either an Array or a Hash (if you also want to map a stat to a different metric name).
You can schedule this however you see fit. A simple way is to use sidekiq-scheduler to run it every N minutes.
After checking out the repo, run bin/setup
to install dependencies. Then, run rake spec
to run the tests. You can also run bin/console
for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install
. To release a new version, update the version number in version.rb
, and then run bundle exec rake release
, which will create a git tag for the version, push git commits and tags, and push the .gem
file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/enova/sidekiq-instrument.
The gem is available as open source under the terms of the MIT License.