Spring-boot based Microservice platform with
- Distributed messaging (Apache Kafka)
- Distributed logging (Fluentd, OpenDistro Elasticsearch)
- Distributed tracing (Jaeger)
- Metrics collection and visualization (Prometheus, Loki, Grafana)
- Rest endpoint documentation using Swagger and OpenAPI
- AWS lambda function running locally in Localstack
- Jenkins CICD pipeline is included in every service that will build and deploy the service
- Ability to switch build profile at build time using Maven profiles for either
- Container platform
- AWS Lambda
- JDK 11 compiler
- Maven > 3.5.0
- Docker > 1.18 (Docker desktop on Windows and MacOS will work)
- Docker compose (latest version. Comes with Docker desktop)
- Kubernetes is turned on in Docker Desktop
- Parent pom that manages commonly used dependencies and plugins.
- All services in this project will extend from this parent pom.
- Read more...
- Template project that can be used to generate a Restful spring boot application with sample java classes, properties
- Read more...
- Admin console
- Spring boot services admin console
- Rest Service
- Exposes a rest endpoint
- Publishes a message on a Kafka topic
- Invoke a GET grpc endpoint
- Kafka client
- Consumes message from the Kafka topic published by Service A
- Protobuf Service
- Expose grpc endpoint producing protobuffer content when invoked by Service A
- Lambda Function
- AWS lambda Fn invoked using SQS and/or CloudWatch event. Creates a S3 file and sends a message to an outbound SQS queue
- Helm scripts to deploy service-rest to k8s cluster. Read more
Publish application and system logs to a centralized server.
- Logs from applications and systems are collected by Fluentd
- Review fluentd conf at logging/fluentd/conf/fluent.conf that has elasticsearch and loki outputs
- Fluentd publishes the logs to OpenDistro Elasticsearch server and Loki
- Visualize logs in Kibana
- Visualize logs in Grafana using Loki datasource
The spring boot microservices integrates with the following infrastructure components:
- Jaeger (Distributed tracing)
- Prometheus (Time-series metrics collector)
- Loki (Index free log collector just like Prometheus)
- Grafana (View Prometheus metrics and Loki logs)
- OpenDistro Elasticsearch and Kibana (Distributed logging)
- AWS Localstack
The spring boot microservices integrates with the following middleware components:
- Apache Zookeeper (Distributed coordination for Kafka)
- Apache Kafka (Distributed messaging)
- Navigate to this project
- Review docker-compose.yml and docker-compose-infra.yml files
- Run
mvn clean package
on anair-service-rest, anair-service-kafka, anair-service-proto, anair-service-admin. This has to be done anytime code is changed and has to be deployed in docker. - Run
docker-compose -f docker-compose-infra.yml up
to start infrastructure services:- Jaeger
- Elasticsearch
- Kibana
- Prometheus
- Prometheus node exporter
- cAdvisor
- Loki
- Grafana
- Localstack (AWS stack)
- Run
docker-compose -f docker-compose-middleware.yml up
to start middleware services:- Fluentd
- Zookeeper
- Kafka
- Run
docker-compose up
to start microservices:- Spring boot admin
- Rest Service
- Kafka Service
- Protobuf Service
- Verify that the below mentioned URls are accessible
- Jaeger
- Prometheus
- Grafana
- Login initially as admin/admin. If asked to enter new password, skip or change.
- cAdvisor
- Kibana
- Login as admin/admin
- Swagger - Rest Service
- Swagger - Protobuf Service
- Spring boot admin
- Login as admin/admin
- Localstack
- In Service A swagger page, try out /publish/{userId} endpoint. This will execute a kafka and grpc transaction. Check logs of all 3 services. View the trace graph in Jaeger UI.