Loading...
Loading...
Found 15 Skills
Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.
Expert skill for integrating local Large Language Models using llama.cpp and Ollama. Covers secure model loading, inference optimization, prompt handling, and protection against LLM-specific vulnerabilities including prompt injection, model theft, and denial of service attacks.
Use when integrating Foundation Models framework, implementing on-device AI with Apple Intelligence, building tool-calling AI features, working with guided generation schemas, converting models with Core ML and coremltools, or running open-source LLMs on Apple Silicon. Covers Foundation Models (LanguageModelSession, @Generable, @Guide, SystemLanguageModel, structured output, tool calling), Core ML (coremltools, model conversion, quantization, palettization, pruning, Neural Engine, MLTensor), MLX Swift (transformer inference, unified memory), and llama.cpp (GGUF, cross-platform LLM).
Run a free 35B AI coding agent on Apple Silicon Macs using local LLMs via llama.cpp or MLX with web search, shell, and file tools.
GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.
Recipes and configs for serving LLMs locally on RTX 3090 GPUs using vLLM, llama.cpp, and SGLang with OpenAI-compatible API
Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.
Diagnoses why ROCm, the HIP SDK, PyTorch, or llama.cpp is broken on an AMD GPU on Linux, Windows, or WSL2, then applies a low-risk fix with consent or hands back the exact next step. Also routes Lemonade, LM Studio, and Ollama problems to the right upstream channel. Use when the user reports that ROCm or HIP "isn't working", torch.cuda.is_available() is False, rocminfo / hipInfo can't see the GPU, or hits hipErrorNoBinaryForGpu, HSA_STATUS_ERROR_INVALID_ISA, "invalid device function", "no kernel image is available", cannot open /dev/kfd, permission denied on /dev/kfd, "ROCk module is NOT loaded", a missing libamdhip64.so / amdhip64_6.dll / hipblas.dll / vcruntime140_1.dll, an HSA_OVERRIDE_GFX_VERSION page fault, an iGPU+dGPU crash, a container that can't see the GPU, or an amdgpu-install / DKMS failure. Backed by the `rocm` CLI (`rocm examine` / `rocm diagnose` / `rocm fix`); this skill is a thin driver over those commands, not a re-implementation.
Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.
Connect to local LLM endpoints (Ollama, llama.cpp, vLLM) with automatic provider fallback. Use when: (1) you need to run LLM inference locally for privacy/cost, (2) you want to use models not available via cloud APIs, (3) you need offline capability, (4) you want automatic fallback to cloud providers when local fails.
Local-first AI coding agent powered by llama.cpp with zero tokens costs, Docker sandboxing, 20 built-in tools, LSP/Roslyn intelligence, and MCP integration
Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.