cv

verl-omni — diffusion/omni RL post-training (committer, 15 PRs merged, Jul 2026 – present)

• Implemented On-Policy Distillation for diffusion RL, the core feature of the official roadmap RFC #293: teacher-anchored KL and flow-matching losses plus a frozen-teacher runtime that runs as a third forward-only worker beside actor and ref, with fit-loop hooks and fail-closed config validation (#300, #325).
• Verified it end to end on SD3.5-Medium with an OCR reward and saw student reward rise from 0.23 to 0.62–0.71, matching the FlowGRPO RL teacher about 5× faster than GRPO while distillation KL fell monotonically from 4.2e-4 to 1.8e-5; covered by 150+ mutation-tested unit tests, GPU smoke runs and a 40-step training-curve reproduction.
• Extended OPD to multi-teacher distillation that routes each sample by a configurable batch column to teachers either co-located with the actor or in standalone per-teacher resource pools scored concurrently (#427), then ported the runtime to the v1 sync trainer (#493).
• Wrote an RFC and shipped an asynchronous one-step-off teacher scheduler for the v1 separate-async trainer that overlaps teacher scoring of batch k with the actor update of batch k−1: teacher-phase wait fell from 3.6s to 0.02s, two-teacher step time from 22.1s to 19.0s, and single-teacher step time by 16.7–23.7% at full load, while training curves matched inline scheduling within 0.004 PickScore (RFC #498, #495).
• Fixed a Ray rendezvous port collision across worker groups by slicing ports per group, which unblocked standalone multi-teacher pools (#482); two teacher-config corrections, bf16 weights and the inference micro-batch, cut the teacher phase from 7.76s to 3.13s.
• Found and fixed a v1-trainer bug in which co-located reward outputs overwrote the whole batch (#513).
• Wrote a maintainer-endorsed RFC that measured rollout–training numerical consistency across three diffusion pipelines, SD3.5-Medium, Qwen-Image-20B and BAGEL-7B, over 8 experiment arms; its proposal shipped as default per-timestep monitoring (#291).
• Traced a replay crash and one-directional log-prob drift to a stale BAGEL step-count mapping and fixed it (#279), cutting mean log-prob drift 157× to bf16 noise in line with an analytical prediction.
• Wired the torch profiler into both trainer fit paths and the reward-model rollout server (#252, #256), bringing profiled-step time from 616s to 70s and trace size from 163MB to 32MB.

vllm-omni — SANA-Video acceleration stack & MAGI-2 torch.compile (5 PRs merged, Aug 2026 – present)

• Derived a sequence-parallel scheme for ReLU linear attention, where Ulysses and Ring do not apply: because the KV state sums over the sequence dimension, SP needs only one all-reduce of a single d×d state, so communication is O(d²) at any sequence length; measured 1.80× at SP2 and 3.57× at SP2+CFG2 on 4×A800 (#5940).
• Added tensor-parallel and CFG-parallel execution, measuring TP2 1.52×, CFG2 2.04× and TP2×CFG2 3.02×, with a 32-case correctness matrix and TP1 bit-exact against baseline (#5861).
• Added Cache-DiT caching with CPU offload, 1.56× faster with peak GPU memory down 7.3 GiB (#5882).
• Fixed a FLASH_ATTN cross-attention key-padding bug that silently mixed samples within a batch and affected every padded cross-attention diffusion model, raising real-weight output cosine from 0.9157 to 0.99996 (#5866).
• Applied regional torch.compile to the MAGI-2 Preview native transformer, splitting each layer into compilable regions around the eager packed-attention and MoE-routing kernels and emulating eager bf16 rounding with precision casts; denoising step fell from 1.93s to 1.48s at SP4/272p on A800 with per-GPU peak memory down 1.1–2.2 GiB, and compiled-vs-eager differences sit at kernel reduction-order level (#7174, in review).
• Fixed layerwise CPU offload that kept a nested 40-layer block fully resident and OOM’d single-GPU MAGI-2 (#7187, in review).
• Built a shadow-wrapper merge-on-load fast path for diffusion LoRA with fp32 delta fold-in and exact unload, bringing LoRA generation overhead from +122% to +4.7% while holding the PPO ratio at 1±1e-6 (#5059, in review).

sglang-omni — Qwen3-Omni inference performance (1 PR merged, Aug 2026)

• Rewrote the thinker-prefill multimodal embedding merge from a per-request Python loop into one batched GPU scatter (#1161): device-to-host syncs fell from 96 to 0 per batch for a 2.1–4.1× speedup and cleared the path to prefill CUDA graphs.
• Verified the rewrite three ways: 6000-seed differential fuzzing with element-wise identical output, byte-identical end-to-end serving logits, and an H800 performance matrix.

LLM Training Systems from Scratch (Stanford CS336 Assignments)

PyTorch, Triton, NCCL · 2026

• Wrote FSDP from scratch (backward-hook reduce-scatter, on-demand all-gather, fp32 master shards, round-robin sharded optimizer), a FlashAttention Triton kernel, and a KV cache, all inside a Transformer LM built from the ground up (RoPE, BPE tokenizer).