NeMo AI Ecosystem

Executive Summary

This is not just a Python server - this is a complete AI-powered life automation product.

The NeMo AI Ecosystem integrates enterprise-grade voice intelligence, AR wearables, IoT orchestration, and multi-platform connectivity into a unified system. Built for real-world use with scalable architecture, security, and enterprise patterns.

Technical Value Proposition

• Hands-Free Control: Voice commands via AR glasses control your entire digital and physical environment
• Multi-Platform Integration: Android app, Python backend, IoT hub, smart home devices - all orchestrated seamlessly
• Enterprise AI: Real-time speech recognition, speaker diarization, emotion analysis, and RAG-powered contextual memory
• Scalable Architecture: Microservice design with GPU optimization, role-based security, and Docker deployment
• Enterprise-Grade: 12,000+ lines of tested Python code, comprehensive API documentation, automated testing

Complete Ecosystem Architecture

This system connects 8+ platforms to create a unified AI-powered automation ecosystem:

Integration Flow: Voice to Action

Technology Stack

Key Features

Click any feature card to expand details

API Endpoints (30+ Total)

Full REST and WebSocket APIs with comprehensive documentation:

Core Services

• POST /transcribe - Real-time audio transcription with NeMo Parakeet
• POST /speaker/enroll - Speaker voice enrollment for identification
• POST /speaker/identify - Identify speaker from audio sample
• POST /emotion/analyze - Emotion detection from text or transcript
• POST /gemma/chat - LLM inference with RAG context injection
• WS /gemma/stream - WebSocket streaming for real-time chat

Memory & RAG

• POST /rag/search - Semantic search across conversation history
• GET /memories - Retrieve user memories with filtering
• POST /memories - Store new contextual memory
• GET /transcripts - Access historical transcriptions

IoT Control

• POST /iot/command - Execute smart home device commands
• GET /iot/status - Query device states
• POST /voicemonkey/trigger - Trigger Alexa routines

Authentication & Admin

• POST /auth/login - User authentication with session creation
• POST /auth/logout - Session invalidation
• GET /admin/users - User management (admin only)
• GET /health - System health check with GPU status

Architecture Design Decisions

Why CPU for ASR/Diarization?

Deterministic GPU allocation prevents resource contention. By reserving the GPU exclusively for Gemma LLM, we guarantee consistent inference performance. NeMo Parakeet and TitaNet run efficiently on CPU with acceptable latency for non-real-time transcription.

Why GGUF Quantization?

4-bit quantization (Q4_K_M) enables running Gemma 3 4B on 6GB VRAM with minimal quality loss (<3% perplexity increase). Reduces memory footprint from 8GB to ~3.5GB while maintaining coherent responses. Essential for consumer-grade hardware deployment.

Why Microservice Architecture?

Separate services for ASR, speaker ID, emotion, LLM, and RAG enable independent scaling and testing. Each service has clear interfaces and responsibilities. Failure in one service doesn't cascade to others. Easy to swap models or providers without refactoring.

Why Local-First AI?

Privacy-focused design runs ASR, speaker ID, and LLM locally. Only falls back to Deepgram/OpenAI APIs when local resources are unavailable. User data stays on-premises by default. Reduces cloud costs and API dependencies.

Why IoT Hub Pattern?

Centralized orchestration layer abstracts device-specific protocols. Unified interface for VoiceMonkey, curl commands, and direct HTTP APIs. Enables complex automation routines that span multiple devices. Simplifies error handling and retry logic.

Code Sample: GPU Exclusive Enforcement

This pattern ensures Gemma LLM gets deterministic GPU access while preventing other services from competing for VRAM:

# GPU-exclusive enforcement for Gemma LLM
from src.utils.gpu_utils import enforce_gpu_only, clear_gpu_cache

# Clear any residual VRAM before loading
clear_gpu_cache(force=True)

# Enforce GPU-only context (sets CUDA_VISIBLE_DEVICES)
with enforce_gpu_only(device_id=0):
    model = Llama(
        model_path=GEMMA_MODEL_PATH,
        n_gpu_layers=-1,  # All 28 layers on GPU
        n_ctx=8192,       # 8K context window
        n_batch=512,
        verbose=False
    )
    
    # Verify GPU offload succeeded
    assert model.llama_supports_gpu_offload(), "GPU offload failed!"
    
    print(f"[GEMMA] Loaded with {model.n_layers} layers on GPU")
    print(f"[GEMMA] VRAM allocated: {get_vram_usage()['allocated_gb']:.2f} GB")

# Other services run with CUDA_VISIBLE_DEVICES=""
# ensuring they never see the GPU

System Architecture & Practices

This system is built for real-world deployment with enterprise-grade practices:

Real-World Use Case

This system demonstrates full-stack integration capabilities:

• Voice commands from AR glasses control smart home devices
• Contextual AI assistant remembers conversations and preferences
• Multi-user support with isolated data and personalized experiences
• Reliable performance on consumer hardware (GTX 1660 Ti)
• Secure remote access via authenticated mobile apps

Interested in This Technology?

This ecosystem demonstrates enterprise-grade AI engineering, IoT orchestration, and product development capabilities. The architecture, code quality, and integration depth showcase scalable software engineering practices.