TSDBHandler
Send metrics to a OpenTSDB server.
OpenTSDB is a distributed, scalable Time Series Database (TSDB) written on top of HBase. OpenTSDB was written to address a common need: store, index and serve metrics collected from computer systems (network gear, operating systems, applications) at a large scale, and make this data easily accessible and graphable.
Thanks to HBase's scalability, OpenTSDB allows you to collect many thousands of metrics from thousands of hosts and applications, at a high rate (every few seconds). OpenTSDB will never delete or downsample data and can easily store billions of data points. As a matter of fact, StumbleUpon uses it to keep track of hundred of thousands of time series and collects over 1 billion data points per day in their main production datacenter.
Imagine having the ability to quickly plot a graph showing the number of DELETE statements going to your MySQL database along with the number of slow queries and temporary files created, and correlate this with the 99th percentile of your service's latency. OpenTSDB makes generating such graphs on the fly a trivial operation, while manipulating millions of data point for very fine grained, real-time monitoring.
The system per default adds a tag 'hostname' with the hostname where the collection took place. You can add as many as you like. The 'tags' config element allows space separated key value pairs.
Example : tags = environment=test datacenter=north
The collectors specify the metrics with a single axis of navigation and often include aggregation counters that OpenTSDB could calculate directly. The openTSDB tagging system supports multiple axis and is better suited for dynamical values. You can [read here] (http://opentsdb.net/docs/build/html/user_guide/query/timeseries.html) how OpenTSDB suggests to sepearate tags from metrics. when doing this you also want to remove these aggregate values since they would lead to double counting when requesting OpenTSDB to do the aggregation based on the tags.
The handler currently provides the ability to extract the tags from a metric (and remove aggregate values) for following collectors : CPUCollector HaProxyCollector * MattermostCollector
==== Notes
We don't automatically make the metrics via mkmetric, so either set
set tsd.core.auto_create_metrics = true
in your [OpenTSDB configuration] (http://opentsdb.net/docs/build/html/user_guide/configuration.html) or you can run with the null handler and log the output and extract the key values to mkmetric yourself
- enable it in
diamond.conf
:
handlers = diamond.handler.tsdb.TSDBHandler
We will automatically split bigger packages of metrics. You have to set
tsd.http.request.enable_chunked = true
and if pakackes get to big you will also need to adjust
tsd.http.request.max_chunk which is 4096 by default. Learn more
here.
In your diamond.conf:
You can define how many packages you want to send in one package by setting batch<=1. We dont recommend 1 as it may have bigger impact on your CPU.
Compression can be enabled by setting compression to 1-9 while 1 is low and 9 is high.
Options
Setting | Default | Description | Type |
---|---|---|---|
batch | 1 | Amount of metrics to send at once | int |
cleanMetrics | True | Extract tag values from known collectors and make the metrics more OpenTSDB style | bool |
compression | 0 | compression level 1 (low) - 9 (high) | int |
get_default_config_help | get_default_config_help | ||
host | str | ||
password | password for Basic Authorization | str | |
port | 4242 | int | |
prefix | Is added as a prefix for every metric example: 'diamond' -> diamond.metric.name | str | |
skipAggregates | True | Only has effect when cleanMetrics is true. Then the metrics that are considered aggregates are removed | bool |
tags | Tags to be added to each metric | str | |
timeout | 5 | int | |
user | user for Basic Authorization | str |