NVIDIA RAG Blueprint

Autonomy Principles

• Auto-detect everything: GPU, VRAM, drivers, Docker, CUDA, disk, OS, ports, existing services, NGC key, repo state.
• If it can be checked with a command, check it — don't ask the user.
• Ask only when user action is required: providing an API key, confirming data deletion, or choosing between equally valid options.
• Once analysis is done, route to the correct workflow and execute.

Intent Detection

references/deploy.md

references/troubleshoot.md

references/shutdown.md

Configure

references/deploy.md

references/configure/vlm.md

references/configure/guardrails.md

references/configure/query-and-conversation.md

references/configure/ingestion.md

references/configure/search-and-retrieval.md

references/configure/models-and-infrastructure.md

references/configure/reasoning-and-generation.md

references/configure/summarization.md

references/configure/observability.md

references/configure/multimodal-query.md

references/configure/data-catalog.md

references/configure/user-interface.md

references/configure/api-reference.md

references/configure/evaluation.md

references/configure/mcp.md

references/configure/migration.md

references/configure/notebooks.md

Configure Flow

1. Match the user's request to a reference file from the table above.
2. Detect what's running:
   bash

   echo "=== NIM ===" && docker ps --format '{{.Names}}' 2>/dev/null | grep -iE '(nim-llm|nemoretriever-embedding|nemoretriever-ranking|nemo-vlm|nemotron-vlm)' || echo "NO_LOCAL_NIMS"; echo "=== RAG ===" && docker ps --format '{{.Names}}' 2>/dev/null | grep -iE '(rag-server|ingestor-server|milvus)' || echo "NO_DOCKER_RAG"; echo "=== K8S ===" && kubectl get pods -n rag 2>/dev/null | head -5 || echo "NO_K8S"; echo "=== LIBRARY ===" && ps aux 2>/dev/null | grep -E '(nvidia_rag|uvicorn.*rag)' | grep -v grep || echo "NO_LIBRARY"

3. Use this table to determine platform, deployment type, and where config lives:

   Local NIMs running?	RAG services running?	Deployment Type	Config Location
   Yes (Docker)	Any	Self-hosted	deploy/compose/.env
   No	Yes (Docker)	NVIDIA-hosted	deploy/compose/nvdev.env
   Yes (K8s pods)	Any	Self-hosted	values.yaml (NIM sections)
   No	Yes (K8s pods)	NVIDIA-hosted	values.yaml (envVars)
   —	Library processes	Library mode	notebooks/config.yaml
   No	No	Not running	Deploy first via references/deploy.md

   Tell the user what you detected and ask to confirm. Example: "I see local NIM containers running (nim-llm-ms, nemoretriever-embedding-ms) — this is a self-hosted deployment. Config file is

   deploy/compose/.env

   . Correct?"
4. Check current feature state before changing anything — read the config location from step 3, then cross-check the live service:
   • Docker:

     docker exec rag-server env 2>/dev/null | grep -E "<VAR_NAME>"

   • Helm:

     kubectl get pod -n rag -l app=rag-server -o jsonpath='{.items[0].spec.containers[0].env}' 2>/dev/null

   If the config file and live service disagree, tell the user the service has stale config and will need a restart.
5. If the feature needs extra GPUs, check availability against hardware restrictions (see below):
   bash

   nvidia-smi --query-gpu=index,name,memory.total,memory.used --format=csv,noheader 2>/dev/null || echo "NO_GPU"

6. Read the reference file and apply changes:
   • Docker: edit the env file (uncomment to enable, re-comment to disable — the env file is the source of truth). Then restart the affected service:

     source <env-file> && docker compose -f deploy/compose/<compose-file> up -d

     Service	Compose File
     rag-server	docker-compose-rag-server.yaml
     ingestor-server	docker-compose-ingestor-server.yaml
     milvus, etcd, minio	vectordb.yaml
     NIM containers (LLM, embedding, ranking, VLM, OCR)	nims.yaml
     guardrails	docker-compose-nemo-guardrails.yaml
     observability (Grafana, Prometheus, Zipkin)	observability.yaml

   • Helm: edit

     values.yaml

     , then upgrade:

     helm upgrade rag <chart> -n rag -f values.yaml

   • Library: edit

     notebooks/config.yaml

     , then restart the Python process
7. Verify:
   • Docker:

     docker ps --format "table {{.Names}}\t{{.Status}}" | head -20; curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null | head -1

   • Helm:

     kubectl get pods -n rag; kubectl rollout status deployment/rag-server -n rag --timeout=120s

   • Library:

     curl -s http://localhost:8081/v1/health 2>/dev/null | head -1

8. If restart fails, read

   references/troubleshoot.md

   . If multiple features requested, repeat from step 1 for each.

When User Says "Configure" Without Specifics

grep -E "^(export )?(ENABLE_|APP_)" <config-file> 2>/dev/null | sort

Hardware Restrictions

docs/support-matrix.md

docs/service-port-gpu-reference.md

LLM_MS_GPU_ID