Skill rating

Optimize existing Triton kernels for NVIDIA TileIR backend on Blackwell GPUs (sm_100+). Adds TileIR-specific autotune configs: occupancy, num_ctas, TMA descriptors. Covers kernel classification (dot-r

Nsight Systems (nsys) CLI for system-level timeline profiling. Use when the user wants to run nsys profile, analyze .nsys-rep reports, use nsys stats/analyze/recipe commands, diagnose GPU idle time fr

Analyze ncu (NVIDIA Nsight Compute) profiling output: SOL% bottleneck classification, roofline analysis, occupancy diagnosis, memory hierarchy analysis, warp stall analysis, metric interpretation, and

Analyze host/CPU overhead in TensorRT-LLM inference from nsys traces. Detect whether host overhead is the bottleneck using GPU idle ratio, host prep exposed ratio, and per-phase evidence. For regressi

Identify and eliminate host-device synchronizations in PyTorch code. Detects sync points (.item(), .cpu(), boolean indexing, torch.tensor on CUDA), classifies false vs true dependencies, provides sync

Write and implement GPU kernels using NVIDIA CuTe DSL (CUTLASS 4.x Python API) — NOT for Triton, CUDA C++, or conceptual explanations. Trigger only when the user wants to write or implement a kernel,

Compile TensorRT-LLM on a SLURM cluster. Covers submitting a batch job with a container image, monitoring the job, and verifying the build. Use when the user wants to compile TRT-LLM remotely via SLUR

Performance optimization coordination playbook. Contains specialist routing table, TileIR two-step pipeline, kernel generation specialist selection, prioritization criteria, and safe modification work

Systematic approach to exploring the TensorRT-LLM codebase before implementing new features or optimizations. Teaches how to discover existing infrastructure, trace code paths, and avoid reimplementin

Design, implement, calibrate, and run deterministic TensorRT-LLM VisualGen L1 operation tests and L2 component-by-feature parity tests. Use for DiT, VAE, text-encoder, scheduler, parallelism, quantiza

Launch and operate this repo's perf-analyze workflow, which DIAGNOSES a TensorRT-LLM serving deployment without applying changes — benchmark at one concurrency or a Pareto curve of them (tok/s/user vs

Compile TensorRT-LLM on a compute node inside a Docker container. Use this when already on a compute node with GPUs visible.

Performance analysis coordination workflow. Guides profiling delegation, bottleneck classification (compute/memory/launch/communication/sync), and structured report generation. Use when the user asks

Code instrumentation for timing workloads. Two scenarios: (1) Training loop — inject manual timing to report per-iteration latency, throughput (samples/sec), and data load time. (2) Standalone kernel/

Apply CUDA Graphs to PyTorch workloads — API selection (torch.compile, PyTorch make_graphed_callables, TE make_graphed_callables, MCore CudaGraphManager, FullCudaGraphWrapper, manual torch.cuda.graph)

ONLY for OpenAI Triton (@triton.jit) kernel development. NEVER use for CUDA C++ kernels, CuTe DSL kernels, TileIR, or profiling tools (ncu, nsys). The user's request must involve Triton explicitly. Co

Benchmark a TensorRT-LLM VisualGen model end to end, server included — pick the workload, bring up the `trtllm-serve` server it needs (resolve the checkpoint, write the config, launch, verify it came

Review, design, and refactor TensorRT-LLM PyTorch MoE code for architecture fit, clean code, maintainability, and testability. Always use for any modification, review, refactor, or design planning tha

Execute a TensorRT-LLM workload locally in Docker. Runs a fully-resolved Docker command in background, monitors completion, reads logs, and reports results. Workflow-agnostic — does not need to know i

Submit and monitor a Slurm job on a local cluster. Supports two modes: (1) Persistent allocation (default) — allocates nodes once via nohup salloc, imports the container once, installs once, and reuse

Check the local execution environment for GPU availability, Docker support, and Slurm access. Returns the execution scenario (`satisfied, local, docker`, `satisfied, local, direct`, `satisfied, slurm,

Refresh TensorRT-LLM VisualGen LPIPS golden images or videos after an NVBug, VisualGen LPIPS CI failure, model/dependency version update, or other intentional output change, including triage, rerunnin

Build Slurm scripts or Docker commands for TensorRT-LLM workloads. Resolves all parameters (docker image, mounts, parallelism, MPI mode), generates the complete script from Category templates, and wri

Profiles and optimizes TensorRT-LLM host/CPU overhead using line_profiler (with nsys support planned). Runs iterative profile-analyze-optimize-validate rounds. Use when GPU utilization is low or optim

Casebook of past successful and classic TensorRT-LLM optimizations (runtime/execution and kernel level) recorded as reusable decision precedents. Consult when deciding which optimization to apply for

Upgrade flashinfer-python version in TensorRT-LLM. Fetches the latest releases from GitHub (stable and nightly), compares with the current pinned version, lets the user pick a target version, and upda

Remote SLURM cluster development via SSH. Use when running jobs, profiling, or developing on a remote SLURM cluster with pyxis/enroot containers. Covers SSH connection management, srun/sbatch/salloc j

Best practices for contributing code to TensorRT-LLM. Covers the official contribution process (issue tracking, fork workflow, DCO signing), coding guidelines, implementation workflow, common mistakes

Generate a source-backed starting `trtllm-serve --config` YAML for basic aggregate single-node PyTorch serving, aligned with checked-in TensorRT-LLM configs and deployment docs. Preserves explicit lat

Launch and operate this repo's perf-optimize workflow, which iteratively APPLIES TensorRT-LLM serving optimizations — baseline benchmark at one concurrency or a Pareto curve of them (tok/s/user vs tok

Cookbook of how TensorRT-LLM performance *instabilities* happen and how they were mitigated, distilled from perf-instability commits on main. Consult when a metric is unstable across runs, iterations,

Onboard a HuggingFace multimodal model (vision/audio/video + text) to the TensorRT-LLM PyTorch backend. Use when writing a new `tensorrt_llm/_torch/models/modeling_{vlm}.py` plus its input processor a

Runs model-level and module-level tests for TensorRT-LLM. First classifies the test scope (module test or model test), then dispatches to the appropriate workflow. Model tests are further classified b

Cookbook of how TensorRT-LLM performance regressions happen and how they were fixed, distilled from NVBug-fix commits on main. Consult when a perf regression is observed or suspected (throughput/TTFT/

Run TensorRT-LLM test cases, benchmarks, evaluations, or custom scripts by checking the environment (local GPU or Slurm), selecting the appropriate Docker image, and executing either locally or via Sl