Bithon is a word combining binocular together with python.
It targets application metrics, logging, distributed tracing, alert and application risk governance under microservice environment.
The above pic illustrates the main components of this project, including:
- Agent, which collects metrics/tracing logs automatically from client application without any code modification at the application side
- Collector, which provides various interfaces (including OpenTelemetry GRPC interface) to receive metrics/tracing logs from clients
- Pipeline, which provides a flexible and robust way to hande small data scale to a very huge data scale for incoming metrics or tracing logs
- Storage, which provides an abstraction to underlying storages like H2, MySQL or Clickhouse
- Alerting, which allows us to set up alerts by using MetricSQL style expression on existing metrics or tracing logs
- Web, which provides a simple web portal for metrics/tracing log visualization
- Around 200 built-in metrics for various JDK or various Java middlewares like Apache Http Components
- Open Telemetry Tracing standard support and integration
- Built-in debugging diagnosis commands for target application
- Flexible deployment to adapt small data scale and huge data scale use cases
- Fast queries and very low storage cost benefit from ClickHouse
- PromQL style alerting expression support
Reference:
- White Paper
- How does the agent work?
- What's the difference between Jaeger and Bithon?
- What's the difference between OpenTelemetry and Bithon?
A demo is provided by this demo repo with a docker-compose file. You can follow the README on that demo repo to start the demo within just 3 steps.
After cloning this project, remember to clone the submodules by following command
git submodule update --initSince the project is built upon SpringBoot 3.0, a JDK 17 or higher is required to build this project.
If you have multiple JDKs on your machine, use export JAVA_HOME={YOUR_JDK_HOME} command to set correct JDK.
For example
export JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-17.jdk/Contents/HomeFor the first time to build this project, use the following command to build dependencies first:
mvn clean install --activate-profiles shaded,jooqand then execute the following command to build the project.
mvn clean install -DskipTestsAfter the first build, we don't need to build the dependencies anymore unless there are changes in these dependencies.
Once the project has been built, you could run the project in a standalone mode to evaluate this project.
To launch server in evaluation mode, execute the following command:
java -Dspring.profiles.active=all-in-one -jar server/server-starter/target/server-starter.jarBy default, the application opens and listens on following ports at local
| Function | Port |
|---|---|
| tracing | 9895 |
| event | 9896 |
| metric | 9898 |
| ctrl | 9899 |
| web | 9897 |
Once the application has started, visit http://localhost:9897/web/home to view the monitor.
Note:
-Dspring.profiles.includeparameter here is just for demo.You can make changes to
server/server-starter/src/main/resources/application.ymlto reflect your own settings.You can also use enable Alibaba Nacos as your configuration storage center.
Attach agent to your java agent by adding the following VM arguments.
-javaagent:<YOUR_PROJECT_DIRECTORY>/agent/agent-distribution/target/agent-distribution/agent-main.jar -Dbithon.application.name=<YOUR_APPLICATION_NAME> -Dbithon.application.env=<YOUR_APPLICATION_ENV>| Variable | Description |
|---|---|
| YOUR_PROJECT_DIRECTORY | the directory where this project saves |
| YOUR_APPLICATION_NAME | the name of your application. It could be any string |
| YOUR_APPLICATION_ENV | the name of your environment to label your application. It could be any string. Usually it could be dev, test, prd |
By default, the agent connects collector running at local(127.0.0.1).
Collector address could be changed in file agent/agent-main/src/main/resources/agent.yml.
Make sure to re-build the project after changing the configuration file above.
Even the project is built by JDK 17 and above, the agent is compatible with JDK 1.8+. The following matrix lists the JDKs that are compatible with the agent on macOS. And in theory, this matrix works both for Windows and Linux.
| JDK | Supported |
|---|---|
| JDK 1.8.0_291 | ✓ |
| JDK 9.0.4 | ✓ |
| JDK 10.0.2 | ✓ |
| JDK 11.0.12 | ✓ |
| JDK 12.0.2 | ✓ |
| JDK 13.0.2 | ✓ |
| JDK 14.0.2 | ✓ |
| JDK 15.0.2 | ✓ |
| JDK 16.02 | ✓ |
| JDK 17 | ✓ |
| JDK 21 | ✓ |
If the target application runs under JDK 11 and above, the following arguments should be added to JVM command to allow the agent to use Java Reflection on corresponding packages.
--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED --add-exports=java.base/sun.net.www=ALL-UNNAMED
| Component | Min Version | Max Version | Metrics | Tracing |
|---|---|---|---|---|
| JVM | 1.8 | ✓ | ||
| JDK - Thread Pool | 1.8 | ✓ | ||
| JDK - HTTP Client | 1.8 | ✓ | ✓ | |
| Alibaba Druid | 1.0.28 | ✓ | ||
| Apache Druid(1) | 0.16 | 31.0 | ✓ | |
| Apache Kafka(2) | 0.10 | ✓ | ✓ | |
| Apache OZone | 1.3.0 | ✓ | ||
| Eclipse Glassfish | 2.34 | ✓ | ||
| GRPC | 1.57.0 | ✓ | ||
| Google Guice | 4.1.0 | ✓ | ||
| HTTP Client - Apache | 4.5.2 | 5.x | ✓ | ✓ |
| HTTP Client - Jetty | 9.4.6 | ✓ | ✓ | |
| HTTP Client - Netty | 3.10.6 | < 4.0 | ✓ | ✓ |
| HTTP Client - okhttp3 | 3.2 | 4.9 | ✓ | ✓ |
| HTTP Client - reactor-netty | 1.0.11 | ✓ | ✓ | |
| Jersey | 1.19.4 | ✓ | ||
| MongoDB | 3.4.2 | ✓ | ||
| MySQL | 5.x | 8.x | ✓ | |
| Open Feign | 10.8 | ✓ | ||
| Quartz | 2.x | ✓ | ✓ | |
| Redis - Jedis | 2.9 | 5.x | ✓ | ✓ |
| Redis - Lettuce(3) | 5.1.2 | 6.x | ✓ | ✓ |
| Redis - Redisson | 3.19.0 | ✓ | ✓ | |
| Spring Boot | 1.5 | 3.0+ | ✓ | |
| Spring Bean | 4.3.12 | ✓ | ||
| Spring Open Feign | 10.8 | ✓ | ||
| Spring Rest Template | 4.3.12 | ✓ | ||
| Spring Scheduling | 4.3.12 | ✓ | ||
| Spring Gateway | 3.0.0 | ✓ | ✓ | |
| HTTP Server - Jetty | 9.4.41 | ✓ | ✓ | |
| HTTP Server - Netty | 2.0.0 | ✓ | ||
| HTTP Server - Tomcat | 8.5.20 | ✓ | ✓ | |
| HTTP Server - Undertow | 1.4.12 | ✓ | ✓ | |
| xxl-job | 2.3.0 | ✓ |
- For Apache Druid, the Jersey plugin is required to be enabled to collect query information.
- For Apache Kafka clients 3.7, the consumer metrics only works when the
group.protocolis configured asclassicwhich is the default configuration of the consumer client. - For Lettuce, the tracing support is only available when it's used with Spring Data Redis API.
To develop for this project, intellij is recommended.
A code style template file(dev/bithon_intellij_code_style) must be imported into intellij for coding.
For more information, check the development doc.