Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add prometheus docker service for node performance metrics analytics #287

Merged
merged 2 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ COMPOSE_PROFILES=chat_completions_vllm
# Tracing level
TRACE_LEVEL=info

# Prometheus Configuration
PROMETHEUS_PORT=9090

# Grafana Configuration
GRAFANA_PORT=30001

# ----------------------------------------------------------------------------------
# chat completions server
CHAT_COMPLETIONS_SERVER_PORT=50000
Expand Down
18 changes: 11 additions & 7 deletions atoma-service/docs/openapi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ paths:
get:
tags:
- health
summary: Handles the health check endpoint.
summary: Health
description: |-
This function is used to verify that the server is running and responsive.
It's typically used by load balancers or monitoring systems to check the
Expand Down Expand Up @@ -41,7 +41,7 @@ paths:
get:
tags:
- metrics
summary: Handles the metrics endpoint.
summary: Metrics
description: |-
This function is used to return the metrics for the service.

Expand All @@ -56,7 +56,7 @@ paths:
post:
tags:
- chat
summary: Handles chat completion requests by forwarding them to the inference service and managing token usage.
summary: Create chat completion
description: |-
This handler performs several key operations:
1. Forwards the chat completion request to the inference service
Expand Down Expand Up @@ -102,8 +102,10 @@ paths:
post:
tags:
- embeddings
summary: Handles embedding requests by proxying them to the embeddings service.
description: |-
summary: Create embeddings
description: |2-


This handler simply forwards the request to the embeddings service and returns the response.

# Arguments
Expand Down Expand Up @@ -138,8 +140,10 @@ paths:
post:
tags:
- images
summary: Handles image generation requests by proxying them to the image generations service.
description: |-
summary: Create image generation
description: |2-


This handler simply forwards the request to the image generations service and returns the response.

# Arguments
Expand Down
25 changes: 25 additions & 0 deletions docker-compose.yaml
jorgeantonio21 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,30 @@ services:
networks:
- atoma-network

prometheus:
image: prom/prometheus:latest
restart: always
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
volumes:
- ./prometheus.yml:/etc/prometheus/prometheus.yml
env_file: .env
networks:
- atoma-network

grafana:
image: grafana/grafana:latest
restart: always
ports:
- "${GRAFANA_PORT:-30001}:30000"
depends_on:
- prometheus
volumes:
- grafana_data:/var/lib/grafana
env_file: .env
networks:
- atoma-network

atoma-node:
<<: *atoma-node
build:
Expand Down Expand Up @@ -166,3 +190,4 @@ networks:

volumes:
postgres-data:
grafana_data:
9 changes: 9 additions & 0 deletions prometheus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
global:
scrape_interval: 15s
evaluation_interval: 15s

scrape_configs:
- job_name: 'atoma-node'
metrics_path: '/metrics'
static_configs:
- targets: ['atoma-node:3000']