> ## Documentation Index > Fetch the complete documentation index at: https://docs.callintel.io/llms.txt > Use this file to discover all available pages before exploring further. # Architecture Overview > Understanding the CallIntel system design. CallIntel is built on a modular microservices architecture designed for scalability, reliability, and ease of integration. ## System Architecture Diagram ``` ┌─────────────────────────────────────────────────────────────┐ │ External APIs │ │ (Gemini, OpenAI, Hume, Twilio, Plivo, Gmail, etc.) │ └────────────────────┬────────────────────────────────────────┘ │ ┌────────────────────▼────────────────────────────────────────┐ │ API Gateway Layer │ │ FastAPI (http://localhost:8010) │ │ │ │ ├── REST API Endpoints (/api/agents, /api/calls, etc.) │ │ ├── WebSocket Streaming (/ws/stream) │ │ └── Webhook Receivers (/webhook/twilio, etc.) │ └─────────┬──────────┬──────────┬──────────┬─────────────────-┘ │ │ │ │ ┌─────▼──┐ ┌────▼────┐ ┌──▼────┐ ┌──▼────┐ │ Agent │ │ Call │ │Org & │ │Batch │ │Service │ │ Service │ │Auth │ │Calls │ └─────┬──┘ └────┬────┘ └──┬────┘ └──┬────┘ │ │ │ │ ┌─────▼──────────▼─────────▼─────────▼──────┐ │ │ │ AI Bridges Layer │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Gemini │ │ OpenAI │ │ Hume │ │ │ └──────────┘ └──────────┘ └──────────┘ │ │ │ └─────────────┬─────────────────────────────┘ │ ┌─────────────▼──────────────────┐ │ Communication Layer │ │ ┌──────────┐ ┌──────────┐ │ │ │ Twilio │ │ Plivo │ │ │ └──────────┘ └──────────┘ │ │ ┌──────────────────────────┐ │ │ │ LiveKit (RTC Media) │ │ │ └──────────────────────────┘ │ └─────────────┬──────────────────┘ │ ┌─────────────▼──────────────────┐ │ Integration Layer (MCP) │ │ ┌──────────┐ ┌──────────┐ │ │ │ Telegram │ │ Gmail │ │ │ └──────────┘ └──────────┘ │ │ ┌──────────┐ ┌──────────┐ │ │ │WhatsApp │ │ Shopify │ │ │ └──────────┘ └──────────┘ │ └─────────────┬──────────────────┘ │ ┌─────────────▼──────────────────┐ │ Data Persistence Layer │ │ ┌──────────────────────────┐ │ │ │ Supabase (PostgreSQL) │ │ │ │ (Organizations, Agents, │ │ │ │ Calls, Schedules) │ │ │ └──────────────────────────┘ │ └────────────────────────────────┘ ``` ## Core Services ### 1. Agent Service (`Service/AgentService.py`) **Responsibilities:** * Create and manage AI agents * Configure system prompts and behavior * Handle agent lifecycle (create, update, delete) * Manage conversation context **Key Classes:** ```python theme={null} class AgentService: def create_agent(self, config: AgentConfig) -> Agent def get_agent(self, agent_id: str) -> Agent def update_agent(self, agent_id: str, config: AgentConfig) -> Agent def delete_agent(self, agent_id: str) -> None ``` **Features:** * Multi-provider AI support (Gemini, OpenAI, Hume) * Custom system prompts * Tool/function calling * Context management ### 2. Call Service (`Service/CallService.py`) **Responsibilities:** * Manage voice call lifecycle * Handle call routing and connection * Process call recordings * Track call metadata **Key Methods:** ```python theme={null} def initiate_call(self, call_request: OutboundCallRequest) -> Call def handle_incoming_call(self, twilio_data: dict) -> TwiMLResponse def process_call_stream(self, websocket: WebSocket, call_id: str) -> None def end_call(self, call_id: str) -> None ``` **Call Flow:** 1. Create call request 2. Initialize call connection 3. Stream audio to AI agent 4. Process agent responses 5. Return audio to caller 6. Log call metadata ### 3. Organization Service (`Service/OrganizationService.py`) **Responsibilities:** * Manage multi-tenant organizations * Handle credentials per organization * Manage API keys and authentication * Organization billing and usage **Features:** * Organization creation * Member management * API key generation * Usage tracking ### 4. Scheduler Service (`scheduler/scheduler.py`) **Responsibilities:** * Schedule calls at specific times * Manage recurring call patterns * Trigger batch operations * Handle call execution **Implementation:** * Uses APScheduler for scheduling * Stores schedule in Supabase * Supports cron expressions ### 5. Batch Call Service (`Service/BatchCallService.py`) **Responsibilities:** * Process multiple calls in bulk * Handle CSV upload and parsing * Manage batch execution progress * Generate batch reports **Workflow:** 1. Upload call list (CSV/JSON) 2. Validate phone numbers 3. Schedule calls 4. Monitor progress 5. Generate results report ## AI Bridges AI bridges abstract the different AI provider APIs into a common interface. ### Supported Providers | Provider | Module | Features | | ----------- | ------------------------------- | -------------------------------- | | **Gemini** | `bridges/gemini_bridge_base.py` | Real-time streaming, multi-modal | | **OpenAI** | `bridges/open_ai.py` | GPT-4, function calling, vision | | **Hume AI** | `bridges/hume_bridge.py` | Emotional intelligence, empathy | ### Bridge Interface ```python theme={null} class AIBridge: async def process_message(self, message: str, context: ConversationContext) -> str async def stream_response(self, message: str) -> AsyncIterator[str] def set_system_prompt(self, prompt: str) -> None async def execute_tool(self, tool_name: str, args: dict) -> Any ``` ## Communication Layer ### Telephony Providers **Twilio:** * Incoming call handling via webhooks * TwiML response generation * Real-time media streaming * Call recording **Plivo:** * Similar to Twilio * Alternative provider for redundancy * REST API for outbound calls ### LiveKit Integration For real-time communication: * WebRTC media streaming * Low-latency audio processing * Video support * Room-based communication ## MCP Integration Layer Model Context Protocol services enable integration with external platforms: ### Service Architecture ``` MCP Client (in AgentService) │ └─► MCP Server Process │ ├─► Tool: send_email ├─► Tool: read_email ├─► Tool: send_message └─► Tool: read_message ``` ### Supported Integrations | Service | Port | Tools | | -------- | ---- | ---------------------------- | | Telegram | 8100 | send\_message, read\_message | | Gmail | 8101 | send\_email, read\_email | | WhatsApp | 8102 | send\_message, read\_message | ## Data Layer ### Supabase Database Schema ```sql theme={null} -- Organizations CREATE TABLE organizations ( id UUID PRIMARY KEY, name TEXT NOT NULL, created_at TIMESTAMP, updated_at TIMESTAMP ); -- Agents CREATE TABLE agents ( id UUID PRIMARY KEY, organization_id UUID REFERENCES organizations, name TEXT NOT NULL, ai_provider TEXT, system_prompt TEXT, created_at TIMESTAMP ); -- Calls CREATE TABLE calls ( id UUID PRIMARY KEY, agent_id UUID REFERENCES agents, from_phone TEXT, to_phone TEXT, status TEXT, duration INTEGER, recording_url TEXT, created_at TIMESTAMP ); -- Scheduled Calls CREATE TABLE scheduled_calls ( id UUID PRIMARY KEY, agent_id UUID REFERENCES agents, phone_number TEXT, scheduled_time TIMESTAMP, status TEXT, cron_expression TEXT ); -- Batch Calls CREATE TABLE batch_calls ( id UUID PRIMARY KEY, organization_id UUID REFERENCES organizations, file_url TEXT, status TEXT, total_calls INTEGER, completed_calls INTEGER, created_at TIMESTAMP ); ``` ## Request Flow Examples ### Outbound Call Flow Outbound Call Flow Diagram ### Incoming WebSocket Stream ``` 1. Client establishes WebSocket connection 2. Audio frames received in real-time 3. Frames sent to AI provider 4. Response streamed back to client 5. Text-to-speech conversion 6. Audio sent to phone ``` ## Scaling Considerations ### Horizontal Scaling * Stateless API servers (multiple instances) * Load balancer (nginx, AWS ALB) * Shared database (Supabase) * Message queue for async tasks (optional: Redis/RabbitMQ) ### Vertical Scaling * Increase server resources (CPU, memory) * Database connection pooling * Caching layer (Redis) * CDN for media files ## Security Architecture ### Authentication ``` API Request │ ├─► Header: Authorization: Bearer │ ├─► Verify Token (Supabase JWT) │ └─► Extract User & Organization ``` ### Data Protection * Encryption at rest (Supabase) * TLS/SSL for transport * API key rotation * Audit logging ## Technology Stack | Layer | Technologies | | -------------------- | -------------------------- | | **API Framework** | FastAPI, Uvicorn, Pydantic | | **Real-time** | WebSocket, LiveKit | | **AI/LLM** | Gemini, OpenAI, Hume | | **Telephony** | Twilio, Plivo, Silero VAD | | **Integrations** | MCP, Telethon, OAuth2 | | **Database** | PostgreSQL (via Supabase) | | **Deployment** | Docker, Docker Compose | | **Task Scheduling** | APScheduler | | **Async Processing** | asyncio, aiohttp | ## Next Steps Learn how to create and configure AI agents. Understand call routing and handling. Explore available services. Integrate external platforms.