AI Learning Platform

A comprehensive AI-powered learning platform composed of microservices for authentication, file streaming, real-time chat, and a RAG pipeline utilizing Kafka, Redis, and Qdrant, paired with a Next.js interactive web client.

Scope: full stackBackend Engineer / ArchitectDuration: 3 monthsYear: 2026Architecture: ai

Repository Links:

backendAI Learning Platform Backend (Microservices, gRPC, Kafka, Qdrant)↗frontendAI Learning Platform UI (Next.js 16 App Router)↗

Key Engineering Facts & Architecture Specs

Structural Invariants

microservices

Fact

proto contracts

Fact

compose services

Fact

vector store

Fact

Qdrant

Architecture View

Processing state: architecture signal graph is initializing...

Architectural Decision Log

Trade-Off Analysis

Contract-first internal communication with gRPC (protobuf)

Strong typing, generated client/server stubs, efficient binary serialization, natural fit for Java/Python interop.

Trade-off: gRPC web and browser compatibility requires the gateway to bridge to REST/HTTP; adds operational complexity for schema versioning and tooling.

Per-service Postgres persistence (one DB/schema per service)

Encapsulates schema evolution; reduces blast radius of schema changes.

Trade-off: Requires event-driven patterns for derived state and complicates multi-DB transactions (eventual consistency). Querying across services requires materialized views or read-side replication.

Kafka as asynchronous backbone with DLTs

Enables decoupled scaling, replayability and backpressure handling.

Trade-off: Operational overhead (broker management), ordering/partition design required, and increased complexity for exactly-once semantics.

Architecture Narrative

Challenge

Provides a scalable backend for interactive learning experiences that require secure identity, persistent chat, searchable user content, and low-latency AI-assisted retrieval and generation over user-owned files.

Solution

microservices + event-driven pipelines. Public edge (REST/WebSocket/SSE) -> API Gateway -> internal gRPC services; asynchronous integration via Kafka topics; vector retrieval via Qdrant for RAG.

Result

Key measurable signals: microservices (7), proto_files (6), compose_services (11).

Trade-off Matrix

Dimension	Selected Option	Impact	Compromise
Internal API protocol	gRPC / Protobuf internally, REST at edge	compact binary protocol, strong typing, fast client/server codegen	Browser compatibility requires gateway bridging; adds schema/version management overhead
Data ownership	Per-service Postgres instances (bounded contexts)	Service autonomy, safer schema evolution, easier deployments per team	Cross-service queries require asynchronous syncing or join-less designs; eventual consistency.

Failure Modes & Resilience Analysis

Degradation Protocol

Failure Path & Recovery Invariants

Failure scenario: Kafka broker unavailable Handling: Services configured with healthchecks in Compose; producers will fail to publish and should surface errors. Consumers will stop receiving events; system relies on broker uptime. Recommended mitigation: mult...

What I'd Do Differently & Evolution

Add distributed tracing (OpenTelemetry) across gateway, gRPC, and Kafka producers/consumers to track and troubleshoot AI pipeline latency end-to-end.

Introduce a schema registry (Kafka Schema Registry or Avro/JSON Schema) and automated compatibility checks for event/topic evolution; add CI enforcement for .proto changes.

Source Repositories & Access Endpoints

Verified Codebases

backend

AI Learning Platform Backend (Microservices, gRPC, Kafka, Qdrant)

Estifanos58/AI-Learning-Platform-Backend

frontendInternal / Private

AI Learning Platform UI (Next.js 16 App Router)

Private engineering codebase · Available upon request for technical interview review