Loading...
Loading...
Found 41 Skills
PyTorch implementation of TurboQuant for LLM KV cache compression using two-stage vector quantization (random rotation + Lloyd-Max + QJL residual correction).
GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.
Vector search best practices for Azure DocumentDB using `cosmosSearch` — choosing between DiskANN / HNSW / IVF, creating indexes, tuning `lBuild` / `lSearch` / `maxDegree`, Product Quantization (up to 16,000 dims), half-precision (fp16) indexing, and normalizing embeddings for cosine similarity. Use when building RAG / semantic-search applications, creating a vector index, tuning recall/latency, or reducing vector-index memory footprint.
Use when integrating Foundation Models framework, implementing on-device AI with Apple Intelligence, building tool-calling AI features, working with guided generation schemas, converting models with Core ML and coremltools, or running open-source LLMs on Apple Silicon. Covers Foundation Models (LanguageModelSession, @Generable, @Guide, SystemLanguageModel, structured output, tool calling), Core ML (coremltools, model conversion, quantization, palettization, pruning, Neural Engine, MLTensor), MLX Swift (transformer inference, unified memory), and llama.cpp (GGUF, cross-platform LLM).
Serve a model with MAX's `max serve` command: set up the environment (pixi or uv with the max-nightly conda channel / nightly wheel index), point the server at a Hugging Face repo or local checkpoint, target a custom architecture with `--custom-architectures`, and pick the right serve flags for the model. Use this whenever the user wants to run, launch, start, or host a model on MAX, bring up an OpenAI-compatible endpoint, serve a custom/ported architecture, debug a `max serve` startup failure, or figure out which serve flags (devices, quantization-encoding, max-length, task, trust-remote-code) a given model needs, even if they don't say "max serve" by name.
Diagnoses and improves Qdrant search relevance. Use when someone reports 'search results are bad', 'wrong results', 'low precision', 'low recall', 'irrelevant matches', 'missing expected results', or asks 'how to improve search quality?', 'which embedding model?', 'should I use hybrid search?', 'should I use reranking?'. Also use when search quality degrades after quantization, model change, or data growth.
Develop, debug, and optimize SGLang LLM serving engine. Use when the user mentions SGLang, sglang, srt, sgl-kernel, LLM serving, model inference, KV cache, attention backend, FlashInfer, MLA, MoE routing, speculative decoding, disaggregated serving, TP/PP/EP, radix cache, continuous batching, chunked prefill, CUDA graph, model loading, quantization FP8/GPTQ/AWQ, JIT kernel, triton kernel SGLang, or asks about serving LLMs with SGLang.
Debug silent corruption when a MAX model loads, compiles, serves, and generates tokens but output disagrees with a reference implementation. Use whenever parity debugging stalls on scalar taps, the model returns gibberish or wrong greedy tokens, logit cosine is high but argmax differs, or generation is coherent then diverges — during an architecture port, a quantization bring-up, a multi-GPU conversion, or after a MAX upgrade. Triggers on "parity failure", "silent corruption", "logits match but tokens diverge", "top-1 mismatch", "greedy divergence", and "model serves but generates garbage". Not for crashes on load or pre-serve scaffolding (use import-model). Mandates reference-vs-MAX tensor-dump comparators first, verify fixes numerically before recompiling, and serve-vs-pipeline bisect when dumps match but text diverges.
Use the Geti application end to end through its REST API — the project → dataset → annotate → train → deploy pipeline served by the FastAPI backend in `application/backend/`. Use when a user (not a contributor) wants to create a project, upload media, add annotations, launch a training or quantization job, track job status, configure a source → model → sink inference pipeline, and enable live inference. Covers the `/api/...` endpoints and the async job model, not backend code changes.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
Expert skill for using TileKernels, a library of optimized GPU kernels for LLM operations (MoE routing, quantization, transpose, engram gating, Manifold HyperConnection) built with TileLang.
Export a promoted fine-tuned model in the right deployment format — merged safetensors, LoRA-only, GGUF with imatrix, or FP8. Use after a checkpoint passes promotion, when choosing a quantization format for a target device, or when an exported model fails its smoke test.