Search Results: vllm-deployment

Found 6 Skills

AI & Machine Learningg1joshi/agent-skills

mistral

Mistral AI efficient open models. Use for efficient AI.

jetson-llm-serve

Stand up vLLM or SGLang serving on Jetson, using upstream vLLM on Thor and Orin JetPack 7.2+, and NVIDIA-AI-IOT vLLM on older Orin.

🇺🇸|EnglishTranslated

AI & Machine Learningorchestra-research/ai-res...

llamaguard

Meta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons, substances, self-harm, criminal planning. 94-95% accuracy. Deploy with vLLM, HuggingFace, Sagemaker. Integrates with NeMo Guardrails.

🇺🇸|EnglishTranslated

AI & Machine Learningnvidia/skills

jetson-package

Pick Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes; maps Orin SM 8.7 vs Thor SM 11.0 and JetPack-specific package choices.

🇺🇸|EnglishTranslated

1 scripts/Checked

AI & Machine Learningvllm-project/vllm-skills

vllm-deploy-simple

Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.

🇺🇸|EnglishTranslated

1 scripts/Attention

AI & Machine Learningvllm-project/vllm-skills

vllm-deploy-docker

Deploy vLLM using Docker (pre-built images or build-from-source) with NVIDIA GPU support and run the OpenAI-compatible server.

🇺🇸|EnglishTranslated