01
Does a network learn to undo distortion better than it tolerates it?
+21 points under extreme distortion, nothing on clean data.
Read the experiment ↗Experiments
Controlled experiments on models I built from scratch. Implementations and full write-ups are on GitHub.
02
Does a sparse MoE beat its dense counterpart at the same active parameters?
Three hypotheses, one survived.
Read the experiment ↗
03
Does RoPE hold up at context lengths the model never saw in training?
RoPE lower at every length to 4096; the expected failure of absolute embeddings did not reproduce.
Read the experiment ↗
04
Why does pix2pix keep the generator in train() mode at inference?
Why the paper keeps BatchNorm in train mode at inference, tested at batch size 1 vs 8.
Read the experiment ↗05
Does the VQ-VAE-2 hierarchy actually split global structure from local detail?
Perceptual loss cut LPIPS from 0.35 to 0.10; the faded top level was consistent with a latent-dimension capacity limit, not the loss.
Read the experiment ↗