Hi, I'm Shrut.

A harness that profiles a cascaded ASR → LLM → TTS pipeline stage by stage with audio streamed in real time, benchmarked against Moshi, a fully duplex model. Streaming cut TTFA 4.4× over batch (11,892ms → 2,717ms p50); Moshi answered 19× faster still at 145ms p50. Inside CosyVoice2 I isolated the streaming bottleneck and recovered the LM's per-token rate from 23.6ms to 16.1ms, cutting TTFA 1,701ms → 1,173ms and total synthesis time 1.4×.

Talking to an AI feels slow because of the pause before it starts replying. I measured exactly where every millisecond of that pause goes, compared the usual build against a newer design that listens and speaks at once, then went into a popular open-source voice model and made it start speaking about 30% sooner. The fix is now with the original authors for review.

An ASR → LLM pipeline measured across five languages, two acoustic conditions, and a control run where the LLM reads the correct transcript. Word error rate turned out to understate the damage. 11.5% of words were wrong on clean audio, but 43.5% of the entities were, and entities are what the answer depends on. Swapping the ASR took response accuracy at 0 dB from 73.6% to 82.4%, level with the clean baseline.

Speech systems are usually graded on how many words they get wrong. That turns out to be the wrong thing to measure. The system got about 89% of words right while mangling almost half the names, postcodes and numbers, which are the only parts the answer actually needs. Measuring those separately showed where it really broke, and replacing one component fixed it.

Implemented per-tensor INT, block-INT and MXINT (MXINT2/4/8) and benchmarked all of them across seven vision-language models on VQAv2 and TextVQA. At 4 bits, per-tensor quantization leaves every one of the seven under 1%, emitting garbage strings rather than wrong answers. Move the same 4 bits to a scale per 32 weights and block-INT4 holds 93–99% of its BF16 VQAv2 score. The bit width was never the problem. It was where the scale came from.

Shrinking a model makes it cheaper and faster to run, but it can also make it much worse at its job. I tested seven models across several shrinking methods: the obvious method falls apart completely at 4 bits, while a newer one keeps nearly all of the original quality at the same size.

An LLM built from scratch in PyTorch: tokenizer, causal multi-head attention, grouped-query attention, RoPE, GeLU FFN, PreNorm vs PostNorm. Controlled experiments on RoPE vs absolute positional embeddings for context-length generalization, and on gradient-norm dynamics across norm placement. Dense and Mixture-of-Experts variants with and without load-balancing loss, analyzing expert routing and convergence. Then LoRA instruction tuning and DPO for alignment.

I rebuilt a language model from the ground up, with no library doing the hard parts for me, then ran experiments on the design decisions inside it. That's why I can usually tell what's actually wrong when one of these models misbehaves, instead of guessing.

VAE, VQ-VAE, VQ-VAE-2, CLIP and a DALL·E-1-style autoregressive text-to-image model, each implemented end to end from the papers with experiments, failure analysis and metrics.

The main families of image-generating models, built one at a time from the original research papers — including the runs that failed, and what the failures turned out to mean.