Skip to content

Latest commit

 

History

History
73 lines (55 loc) · 3.61 KB

README.md

File metadata and controls

73 lines (55 loc) · 3.61 KB
Raw

Real-Time Analytics Service

Task Overview

Objective:
This project is designed to assess skills in high-load service design, Apache Kafka integration, API development, and object-oriented programming principles.

Scenario:
Develop a scalable, real-time analytics service that can handle millions of user events per second. The service captures, processes, and stores event data for real-time and historical analytics. This data will be consumed by other services and displayed on a dashboard for analysis.

Requirements

1. Event Processing Pipeline

  • Design a PHP service integrating Apache Kafka as the primary message broker.
  • Event Structure: Each event should contain:
    • event_id: Unique identifier of the event
    • user_id: Unique identifier for the user
    • event_type: Type of event (e.g., click, view, purchase)
    • timestamp: UNIX timestamp when the event occurred
    • metadata: JSON object with additional event data
  • Ensure the pipeline handles high loads (200,000+ events per second).

2. Event Storage and Retrieval

  • Design the system to store event data in a suitable database for real-time analytics.
  • Implement an API endpoint to fetch historical event data, filtered by user_id and event_type within a time range.
  • Apply indexing strategies and database optimizations for handling high-frequency reads and writes.

3. Real-Time Aggregation Service

  • Implement a service to calculate real-time statistics (e.g., count, average) for specific event types.
  • Push aggregated data to Kafka, enabling other services to subscribe to these statistics in real-time.

4. API Design

Design a REST API with the following endpoints:

  • POST /events: Accepts incoming events (validates input structure).
  • GET /events: Retrieves historical data based on filters (user_id, event_type, time range).
  • GET /stats: Provides real-time statistics for a specified event_type and time_interval.

Documentation should include:

  • Expected input, output, status codes, and any error handling.

5. OOP and Code Quality

  • Follow OOP best practices (e.g., SOLID principles) for maintainability and extensibility.
  • Use design patterns (e.g., Factory, Singleton, Observer) where appropriate.
  • Ensure code complies with PSR standards.

6. Testing and Performance

  • Unit Tests: Write unit tests for key components (e.g., event processing, API endpoints).
  • Load Testing: Implement basic load testing (e.g., using Apache JMeter) to assess performance under high load.
  • Document any caching strategies or optimizations for handling high requests per second (RPS).

Deliverables

  1. Code Submission:

    • A GitHub repository or a zip file with well-organized, PSR-compliant code.

  2. Documentation:

    • API documentation (e.g., using Swagger or a similar tool).
    • A short summary detailing your approach to high-load handling, Kafka integration, and design decisions.

  3. Testing Results:

    • Include unit tests and load test results, if available.

Evaluation Criteria

Your submission will be evaluated on the following:

  • Kafka Integration: Proper usage of Kafka for message brokering and real-time processing.
  • High-Load Architecture: A robust and scalable system capable of handling high loads.
  • API Design: Clear, well-documented, and functional API design.
  • Code Quality: High-quality code adhering to OOP and PSR standards.
  • Testing and Optimization: Effective testing and performance optimizations.
  • No frameworks are allowed: You can not use any framework, instead work only with composer.