microfake

ssh your-computing-id@login.hpc.virginia.edu

A deepfake is not magic. It is the output of a generative model — a system that studies thousands of real faces (or voices, or video frames) until it can produce brand-new ones that never existed. This lab builds that machinery from nothing, three ideas in the order they were invented: autoencoders (the original 2017 face-swap), GANs (two networks locked in a forgery game), and diffusion (the engine inside today's image generators) — then adds time, to make video. Every idea comes with a toy you can run and numbers you can trace by hand. And we finish on the other side of the fight: how to detect what we just learned to fake.

Generative-model vocabulary you may not have seen — latent space, denoiser, score function — is underlined like this: hover (or tap) for a plain-language explainer that opens right below the line.

mkdir -p lab-20 && cd lab-20
wget https://researcher111.github.io/ML-Security-Public/labs/lab-20/lab-20-code.zip
unzip lab-20-code.zip
ls    # autoencoder.py · faceswap.py · gan.py · diffusion.py · video.py · detect.py
      # + their *_faces.py HPC companions · faces_all.npz · run_hpc.slurm · microfake.py …

1 · What is a generative model?

Most models you've built in this course are discriminative: given an input $x$, they answer a question about it — is this packet malicious? is this login suspicious? is this face Alice or Bob? (In symbols, they learn $p(y \mid x)$: the probability of a label, given the input.) A generative model plays a harder game. It learns what the data itself looks like — the distribution $p(x)$ — so well that it can draw brand-new samples: faces that copy no training example, yet look like they could have come from the same pile.

Knowing $p(x)$ buys you two superpowers, and both matter for security:

• Sampling — make new $x$. That is the deepfake: a new face, a cloned voice, a forged video frame.
• Scoring — ask how likely a given $x$ is under the model. That is the seed of detection: real and fake content often sit in different parts of $p(x)$ (§6).

The trick that makes any of this possible is the latent space. Nobody models a million pixels directly. Instead, the model learns a short code — a few hundred numbers — plus a map from that code to pixels, and does its thinking in code-land. The "Try it" demo above is this idea at its smallest — a 2D space where noise becomes structure. You met a latent space in Lab 11 (the residual stream, where "refusal" was a direction); here a direction might mean smiling, pose, or age.

The rest of the lab is those three answers, in the order they were invented — each fixes a specific failure of the one before:

Then §5 adds one axis — time — to turn a photo model into a video model, and §6 flips the whole thing: how a defender tells real from synthetic.

2 · The original deepfake · autoencoders

The first answer — "make the model reconstruct its own input" — is the autoencoder, exactly how the original 2017 "deepfakes" worked. Build two networks back to back:

• an encoder $f$ that squeezes an input $x$ (a face, say 100×100×3 = 30,000 numbers) down to a small code $z = f(x)$ (a few hundred numbers — the latent);
• a decoder $g$ that expands the code back to a full image $\hat{x} = g(z)$.

Train the pair to make the output match the input. The loss has no labels in it at all — the input is the target:

$$\mathcal{L}(f,g) \;=\; \big\lVert\, x - g(f(x)) \,\big\rVert^2 \qquad\text{(reconstruct your own input)}$$

The $\lVert\cdot\rVert^2$ bars mean squared distance: subtract the reconstruction from the input pixel by pixel, square each difference, add them all up.

2.1 The swap · one encoder, two decoders

Here is the trick that launched the word "deepfake." Take two people, Alice and Bob, and a pile of face crops of each. Train one shared encoder with two decoders, routing each person's faces only through their own decoder:

$$\text{Alice's faces:}\quad x \to f(x)=z \to g_A(z)=\hat{x} \qquad\qquad \text{Bob's faces:}\quad x \to f(x)=z \to g_B(z)=\hat{x}$$

Here's the key consequence of sharing the encoder. The code is small, and every slot in it is precious. Spending a slot on "this is Alice" would be a waste — each decoder already knows whose face it paints. So the encoder learns to fill the code with the things both decoders need from it — pose, expression, gaze, lighting — and to leave identity out. The code becomes identity-agnostic: it says how the face is arranged, never whose face it is. Each decoder then paints that arrangement as its own person, and the swap is one line:

$$\text{swap}(x_{\text{Alice}}) \;=\; g_{B}\big(f(x_{\text{Alice}})\big)$$

Alice's pose and expression, rendered as Bob.

2.2 The catch that needed fixing

Autoencoders give the identity swap for free, but the faces come out blurry — and the loss function is the culprit. Squared pixel error, $\lVert x-\hat{x}\rVert^2$ (called L2 loss), rewards playing it safe. When the model can't tell which sharp answer is right (which strands of hair? which skin texture?), the output with the lowest average error is the blurry average of all the candidates. L2 pays for mush — literally, as the widget below shows:

Here is the same objective in code — hover any line for what it does and why it's there:

Now you run it · face-swap on real photos (HPC)

The idea, unchanged: one shared encoder, two decoders. Train on two real people, then feed one person's face through the other person's decoder. Everything above — now on 64×64 photographs instead of 4 hand-picked pixels, and the encoder learns the identity/expression split instead of you assigning it.

On the cluster, faceswap.py trains the shared encoder and two decoders on the shipped face set, then writes a grid of reconstructions and swaps. PyTorch comes from a prebuilt container — nothing to install.

# 0 · connect. Off campus? turn on the UVA VPN (UVA Anywhere) first —
#     without it the cluster simply won't answer.
ssh <computing-id>@login.hpc.virginia.edu        # UVA password + Duo push
# you land on a LOGIN node: fine for editing and submitting, never for training

# 1 · one-time setup, on the cluster: pull the lab zip (no scp needed)
mkdir -p lab-20 && cd lab-20
wget https://researcher111.github.io/ML-Security-Public/labs/lab-20/lab-20-code.zip
unzip lab-20-code.zip                            # faceswap.py, faces_all.npz, run_hpc.slurm, ...

# 2 · still in ~/lab-20: submit the job to a GPU compute node
sbatch run_hpc.slurm faceswap.py --a 7 --b 21    # train + swap on a GPU node
squeue --me                                      # wait until it leaves the queue

# 3 · when it finishes, from your laptop:
scp <id>@login.hpc.virginia.edu:~/lab-20/results.png .

Prefer a live shell on a GPU machine instead of a batch job? Ask Slurm for one — this drops you onto a compute node with a GPU for an hour, where you can run the apptainer exec command from run_hpc.slurm directly and iterate. Request an A100 (gpu:a100:1): the pytorch/2.11.0 container has no CUDA kernels for the older V100s and will crash with "no kernel image is available" on one.

srun -A ds6042 -p interactive --gres=gpu:a100:1 -c 4 --mem=16G -t 01:00:00 --pty bash
# prompt changes from login node to a udc-* compute node — you're on the GPU machine
module load apptainer/1.4.5 pytorch/2.11.0
apptainer exec --nv "$CONTAINERDIR/pytorch-2.11.0.sif" python faceswap.py --a 7 --b 21

What to look for: the reconstruction rows should look like each person; the swap rows should show the other identity wearing the same pose and expression. Re-run with --noise 0 and watch the swap rows turn grainy and incoherent — that latent jitter is what forces each decoder to handle the other person's codes.

A · originals A · reconstruction A→B swap B · originals B · reconstruction B→A swap

Run it yourself: ↓ faceswap.py ↓ faces_all.npz (the data) ↓ run_hpc.slurm

The next idea fixes this. Instead of scoring the output against pixels with a fixed formula, train a second network to tell real faces from fakes — and make the generator's goal "fool that critic." A critic won't fall for a blur. That's the GAN, §3.

3 · GANs · the adversarial game

The autoencoder's blur came from a fixed loss that rewarded playing it safe. So the second answer throws the fixed loss away and learns one instead: train a second network — a discriminator $D$ — to tell real images from fakes, and make the generator's goal "fool $D$." A critic isn't fooled by mush, so the generator must commit to sharp detail. This is the Generative Adversarial Network (Goodfellow et al., 2014) — what made photorealistic face synthesis ("this person does not exist") possible. Yes, click it: every page load samples a fresh $z$ and a StyleGAN renders a face of someone who has never existed. Refresh for another.

Two networks, locked in a game:

• Generator $G$: takes a noise vector $z \sim \mathcal{N}(0, I)$ — a vector whose entries are independent draws from a standard normal: mean vector $0$, identity covariance $I$, so each entry is its own $\mathcal{N}(0,1)$ sample — and outputs a fake sample $G(z)$. It never sees real data directly — it only learns from the critic's reaction.
• Discriminator $D$: takes a sample and outputs $D(x) \in [0,1]$, its estimate of "probability this is real." It must see real data — "real vs. fake" is a classification problem, and no classifier can learn a boundary from only one class — so its training set is real samples labeled $1$ and $G$'s fakes labeled $0$. And per sample it sees the whole vector: every component ($x_1$ and $x_2$ here; every pixel, for an image) feeds its input layer, and the network compresses them all into the single scalar $D(x)$.

3.1 The minimax objective

The game is a single objective the two players push in opposite directions — $D$ maximizes it, $G$ minimizes it:

In plain language: $D$ wants $\log D(x)$ big (call real real) and $\log(1 - D(G(z)))$ big (call fakes fake). $G$ touches only the second term and wants it small — $D(G(z))$ near 1, the critic fooled. At equilibrium, $G$'s samples are indistinguishable from real data and $D$ is a coin flip, $D(x)=\tfrac12$ everywhere.

3.2 Training: alternate the two players

There is no single loss that both networks want to lower, so training takes turns, forever: first $D$ practices against the current fakes (push real scores up, fake scores down), then $G$ gets a turn against that freshly sharpened critic. $G$'s turn has the one tricky part. $G$ never sees a real photo — the only way it can improve is to ask the critic: push a fake through $D$, then let backprop carry the answer to "which way should each pixel move to score more real?" backwards through $D$'s network and on into $G$'s. On that turn $D$ is only consulted, never changed — just $G$'s weights update. Here's the heart of gan.py, the complete micro-GAN you can read top to bottom and run:

3.3 Why it's sharper — and how it breaks

Sharper. The autoencoder averaged because L2 loss paid for averaging. A discriminator does the opposite: a blurry face is trivially fake (real faces have crisp edges and pore-level texture; averages don't), so the only way for $G$ to push $D(G(z))\to1$ is to produce committed high-frequency detail. The learned loss has learned that blur is a tell.

How it breaks. Adversarial training buys realism at the cost of stability — two well-known failure modes, both visible in the widget:

• Mode collapse. $G$ discovers one output that reliably fools the current $D$ and pours all its mass there, ignoring whole regions of the data (a face generator that makes one convincing face for every input). On the ring, you'll see the dots abandon several blobs.
• Instability / oscillation. There's no loss being minimized — it's a moving target. If $D$ gets too strong, $G$'s gradient dies; if $G$ races ahead, $D$ can't keep up. Training can oscillate or diverge instead of settling.

Now you run it · GAN-generate real faces (HPC)

The idea, scaled up: the same generator-vs-discriminator game you just watched on the 2D ring, now painting 64×64 faces. G turns random noise into a face; D judges real vs. fake; they co-adapt until G's faces pass. No labels, no target image — the only signal is D's verdict.

On the cluster, gan_faces.py trains both networks on the shipped faces and writes 16 generated people. It also saves gan_fakes.npz — a batch of its fakes that your §6 detector will later try to catch.

# same setup as §2: the script + faces_all.npz live in ~/lab-20 on the cluster
srun -A ds6042 -p interactive --gres=gpu:a100:1 -c 4 --mem=16G -t 01:00:00 --pty bash
# prompt changes from login node to a udc-* compute node — you're on the GPU machine
module load apptainer/1.4.5 pytorch/2.11.0
apptainer exec --nv "$CONTAINERDIR/pytorch-2.11.0.sif" python gan_faces.py --steps 4000
scp <id>@login.hpc.virginia.edu:~/lab-20/gan_samples.png .   # from your laptop

What to look for: 16 different faces — varied identities, some with glasses. If instead you see the same face 16 times, you've caught mode collapse live: G found one face that fools D and stopped exploring. Re-run with fewer steps to watch faces emerge from static.

Run it yourself: ↓ gan_faces.py ↓ faces_all.npz (the data) ↓ run_hpc.slurm

GANs powered the first wave of convincing deepfakes — StyleGAN for faces, conditional GANs for face-swap and reenactment. But mode collapse and instability made them hard to scale and hard to steer with text. The next idea keeps the sharpness, drops the adversarial game, and trains on a stable regression objective instead — by turning generation into denoising. That's diffusion, §4, where the "Try it" demo finally gets explained.

4 · Diffusion · noise to image

GANs were sharp but unstable. Diffusion keeps the sharpness and drops the instability by replacing the adversarial game with the most reliable objective in machine learning: regression. The method rests on one lopsided observation:

4.1 The forward process · destroy the image (no learning)

Take a clean image $x_0$ and add Gaussian noise scaled by a level $\sigma$. Because the noise is added in one shot, there's a closed form for "the image at noise level $\sigma$":

$$x_\sigma \;=\; x_0 \;+\; \sigma\,\varepsilon, \qquad \varepsilon \sim \mathcal{N}(0, I)$$

$\varepsilon$ is the Greek letter epsilon (not Euler's $e$): a fresh random noise image, every pixel drawn from $\mathcal{N}(0,1)$. So $\sigma\varepsilon$ is "static, scaled by the volume knob $\sigma$."

Slide $\sigma$ from $0$ up and the picture dissolves; nothing is learned here — it's a fixed recipe. Watch a face do it:

4.2 The objective · learn to undo one step (a stable regression)

Now train a network — the denoiser $D_\theta(x_\sigma, \sigma)$ — to look at a noisy image and its noise level and predict the clean image it came from. We have unlimited labeled examples (just noise a real image and remember the original), so the loss is plain mean-squared error:

$$\mathcal{L}(\theta) \;=\; \mathbb{E}_{x_0,\,\sigma,\,\varepsilon}\;\big\lVert\, D_\theta(x_0 + \sigma\varepsilon,\;\sigma) \;-\; x_0 \,\big\rVert^2$$

The $\lVert\cdot\rVert^2$ bars are a vector's squared length: subtract the two images pixel by pixel, square each difference, add them all up — ordinary squared error, written for a whole image at once. The subscripted $\mathbb{E}$ averages that error over random clean images, noise levels, and noise draws (a minibatch mean in code, same as §3).

Reminder — the best possible answer under this loss is the posterior mean $\mathbb{E}[\,x_0 \mid x_\sigma\,]$: many different clean images could have produced the noisy one you're shown, and squared error is minimized by their average, weighted by how likely each candidate is. That's the same MSE-loves-averages fact that blurred the autoencoder in §2 — diffusion tames it by asking for only a small step at a time.

4.3 Generation · reverse the process (this is the top demo)

With a trained denoiser, sampling is the loop you already watched at the top of the page. Start from pure noise $x \sim \mathcal{N}(0, \sigma_{\max}^2 I)$ — every pixel is its own random number with spread $\sigma_{\max}$. The square isn't doing anything to the noise; it's just how the notation works: $\mathcal{N}$'s second slot always reports the spread squared (statisticians call that the variance), so noise with spread $\sigma_{\max}$ gets written $\sigma_{\max}^2$. Then repeat: ask the denoiser for its best guess of the clean image, take a step toward that guess, lower the noise level, and go again. That loop — guess, step toward it, lower the noise, repeat — is the entire sampler, and it has a name you'll keep meeting in the code and the assignment: DDIM (after the paper that introduced it — short for "denoising diffusion implicit models," but the name is just a label for this exact update):

$$\hat{x}_0 = D_\theta(x, \sigma), \qquad x \;\leftarrow\; \hat{x}_0 + \frac{\sigma_{\text{next}}}{\sigma}\,(x - \hat{x}_0)$$

In these formulas, $x$ is the one thing in memory: the current image, which starts as pure static and gets a little cleaner every turn of the loop. $\hat{x}_0$ is the denoiser's guess at the clean image hiding inside the current $x$ (the hat means "estimate"). And $\sigma \to \sigma_{\text{next}}$ are two rungs on a noise ladder written down in advance — say $1.3 \to 0.9 \to 0.6 \to \cdots \to 0$ — so each turn of the loop climbs down exactly one rung.

Now the two formulas, which are one turn of the loop. Left = guess: show the denoiser the current $x$ and its rung $\sigma$; it answers $\hat{x}_0$. Right = move: the $\leftarrow$ means "replace $x$ with what's on the right" — and what's on the right is a point between where $x$ is now and what the guess says, keeping only the fraction $\sigma_{\text{next}}/\sigma$ of the gap. If that fraction is $0.6$, this turn closes $40\%$ of the distance to the guess and keeps the rest. Every turn the fraction shrinks, and on the last rung $\sigma_{\text{next}} = 0$: nothing is kept, and $x$ lands exactly on the final guess. Here it is with real numbers you can check on paper:

Which brings back the loop's one missing ingredient: a denoiser. The stepper above used a perfect one, and the top-of-page demo cheated the same way — we chose the data ourselves, so the perfect denoiser had a closed formula. Below is the honest version, and its two buttons are exactly the two days in the table: Train denoiser is training day (practice on damaged copies of the green data), Generate is generation day (restoration with no original). Try Generate before training — an untrained denoiser pushes noise nowhere. Then train, and watch the same noise resolve into the shape.

4.4 What scales this to real images

Three engineering ideas turn the toy above into Midjourney, and each is a small twist on what you just built:

• A bigger denoiser. Swap the little MLP for a U-Net (a convolutional network with skip connections) so the denoiser can handle millions of pixels. The objective is unchanged.
• Latent diffusion. Don't diffuse pixels — diffuse in the compressed code of an autoencoder (§2!). Stable Diffusion runs the whole noise/denoise loop in a small latent space, then decodes once. The two halves of this lab literally stack.
• Text conditioning. Feed the denoiser a text embedding alongside $\sigma$, so $D_\theta(x, \sigma, \text{"a photo of …"})$ denoises toward the prompt. That's how a deepfake gets directed.

A footnote for the curious: real denoisers also rescale their inputs and outputs per noise level (so the network always regresses a clean, unit-variance target) — the "preconditioning" the widget quietly uses. The idea is unchanged; it just makes training behave at every $\sigma$.

Now you run it · diffusion-generate real faces (HPC)

The same recipe, on faces: the 2D denoiser you trained above, scaled up to 64×64. diffusion_faces.py learns a small U-Net denoiser (the first item on the list above) on the shipped faces, then runs the reverse process from pure noise to a brand-new face — the same idea behind Stable Diffusion, tiny.

Compare it directly to your §3 GAN. Same data, same output size — but trained by plain regression, with no adversary. Watch which failure modes show up and which don't.

# same setup as §2: the script + faces_all.npz live in ~/lab-20 on the cluster
sbatch run_hpc.slurm diffusion_faces.py --steps 12000    # trains + samples on a GPU
squeue --me
scp <id>@login.hpc.virginia.edu:~/lab-20/diff_samples.png .   # from your laptop

What to look for: 16 distinct faces, soft but coherent. Crucially, every sample is a different person — no mode collapse, because there's no adversary to collapse toward. That stability, not raw sharpness, is why diffusion took over. It also saves diff_fakes.npz for the §6 detector.

Run it yourself: ↓ diffusion_faces.py ↓ faces_all.npz (the data) ↓ run_hpc.slurm

You now have the complete still-image toolkit: compress (§2), sharpen (§3), and the stable, scalable generator (§4). The last move is to make it move — add a time axis so the faces blink and turn without flickering into a different person each frame. That's §5.

5 · From image to video

A video is just a stack of images — say 24 frames a second. So the obvious plan is: take the diffusion model from §4 and generate each frame with it. That plan fails immediately, and why it fails points straight at how real video models are built.

5.1 The flicker problem

Your image generator samples from $p(\text{image})$ — a plausible face, drawn partly from random noise. Run it 24 times and you get 24 independent faces. They won't be the same face: hair, skin texture, and lighting all re-roll every frame, because nothing tied this frame's randomness to the last. Played back, that's flicker — the subject boils and jitters even when it should hold still.

5.2 Content vs. motion

"Share the randomness" is the intuition; the way models act on it is to split a video into two parts and treat them differently:

• Content — who/what is in the shot (identity, background, style). This should stay fixed across the whole clip. When it varies frame-to-frame, you see flicker.
• Motion — pose, expression, position, camera. This should change smoothly over time. It's the only thing that should differ between adjacent frames.

You met this split already: the §2 face-swap idea (identity vs. expression), extended down the time axis. A video model keeps one content code for the clip and lets a smoothly-varying motion code drive each frame — exactly what the "consistent" panel does (fixed identity + noise; only the sway/blink/smile change).

5.3 How a denoiser learns to be consistent · temporal attention

You don't hand-code "keep content fixed" — the model learns it, once the denoiser can look across frames while it denoises. Run the §4 image denoiser on all frames at once, then add temporal attention: layers where each frame's features attend to the same region in the other frames (the Lab 02 attention, pointing along time instead of a sentence). Now when frame 5 is unsure how to denoise the cheek, it copies from frames 4 and 6 — the frames agree, and consistency falls out of training instead of being bolted on.

5.4 What scales this to Sora

• Latent video diffusion. Don't denoise pixels across 100 frames — compress each frame with an autoencoder (§2) and diffuse the small codes. Same move that made image diffusion affordable, now paying off ~100× over because there are so many frames.
• Space-time patches. Chop the video into 3D patches (a little block of pixels across a few frames) and treat them like tokens — diffusion over a sequence of space-time tokens. This is the core of Sora's design.
• Conditioning. Same as §4: feed a text embedding (and often a starting image) so the clip is directed — "a dog surfing, cinematic." That conditioning is also what makes a video deepfake controllable.

Now you run it · flicker on a real face (HPC)

The idea, on real faces: a video is one sample from $p(\text{video})$, not $T$ independent samples from $p(\text{image})$ — and that independence is exactly what makes generated video flicker. video_faces.py proves it by rendering the same face-morph two ways: one smooth latent trajectory (consistent) versus the same trajectory with fresh noise re-drawn every frame (naive).

# same setup as §2: the script + faces_all.npz live in ~/lab-20 on the cluster
sbatch run_hpc.slurm video_faces.py --a 7 --b 21    # trains an AE, renders both clips
squeue --me
scp '<id>@login.hpc.virginia.edu:~/lab-20/clip_*.png' .   # quotes keep zsh off the *

What to look for: both filmstrips show the same morph, but the naive one jitters frame-to-frame while the consistent one glides. The script prints a flicker number for each — the naive clip's is several times larger, and that gap is the flicker, with the motion held identical.

Run it yourself: ↓ video_faces.py ↓ faces_all.npz (the data) ↓ run_hpc.slurm

Independence causes flicker; sharing structure across time — content/motion split, temporal attention, correlated noise — fixes it. That same structure is also a weakness a defender can exploit: a face that's perfect frame-by-frame but inconsistent across frames (a flicker at the jaw, a blink that doesn't match) is one of the clearest deepfake tells. That's §6.

6 · Detecting deepfakes

You've built every generator in the attacker's kit. Now the defender's turn. The principle is simple: a generator can't help leaving fingerprints. Every method you built takes shortcuts — an upsampling layer, an averaged texture, an independently-sampled frame — and each is a tell. Detection is the craft of finding them; the catch, which we'll be honest about, is that it's an arms race the generators keep winning.

6.1 Frequency fingerprints · the spectrum sees what you can't

The most reliable forensic signal for GAN- and diffusion-made images lives in the Fourier spectrum — a re-plot of the image that shows which repeating patterns it contains instead of which pixels it has. Here's why that view matters. To turn a small code into a big image, generators upsample — and standard upsampling layers (transposed convolutions, pixel-shuffles) deposit a faint, regular checkerboard. You usually can't see it. But a repeating pattern in pixels becomes a sharp spike in the spectrum, so what the eye misses, the spectrum screams. Drag the artifact up below — the image barely changes, the spectrum erupts:

6.2 Physical & physiological tells

Generators model appearance, not physics — so they get the world's bookkeeping subtly wrong, and a careful eye (or a targeted model) can catch it:

• Lighting & shadows that don't agree. A face lit from the left wearing a shadow cast to the left; specular highlights in the two eyes that imply two different light sources. The generator never solved a lighting equation.
• Reflections & geometry. Eyeglasses or pupils whose reflections don't match the scene; ears, teeth, and earrings that are subtly asymmetric because the model has no notion of a single rigid object.
• Physiology. Early GAN faces blinked too little or irregularly; pupils came out non-circular. (Each published tell gets trained away in the next model — see the arms race below.)

6.3 Temporal tells · you already built one

For video, the strongest signal is the one you manufactured in §5: inconsistency across frames. A deepfake can be flawless frame-by-frame yet betray itself in motion — a flicker at the swapped jawline, a blink that doesn't match the eyelids, lip motion that drifts out of sync with the audio, identity that subtly wobbles. The detector watches the differences between consecutive frames (exactly the flicker metric in video.py) and flags regions that change in ways real footage doesn't. The same property that makes video hard to generate makes it easier to catch.

6.4 Likelihood · is this point even on the manifold?

Recall from §1 that a generative model gives you not just sampling but scoring — how typical is this $x$? That's a detector hiding in plain sight: real images sit on the manifold (§2's thin surface in pixel space where real data lives); many fakes sit just off it. You already have the tool — the §4 denoiser. Feed it an image at a small noise level and see how much it wants to change: a real image is near the manifold, so the correction is tiny; an off-manifold fake gets yanked harder. The generator, run as its own critic, becomes a detector.

6.5 Learned detectors and the arms race

The workhorse in practice is the obvious one: collect real images and fakes, and train a classifier (a CNN) to tell them apart. It works extremely well — on the generator it was trained against. The trouble is generalization:

• It overfits the generator's fingerprint. A detector trained on StyleGAN faces leans on StyleGAN's specific upsampling artifact — and faceplants on diffusion images, which upsample differently. New generator, new fingerprint, detector retrained. The defender is always a step behind.
• The artifact is trainable away. Once a tell is published ("GANs don't blink"), the next model is trained to fix it. You can even put the detector in the training loop — which is just §3's adversarial game again, now aimed at beating forensics.

Now you run it · catch the fakes you just made (HPC)

Close the loop: detect_faces.py trains a small classifier on real faces vs. the fakes your own §3 GAN and §4 diffusion models generated (run those first — they save gan_fakes.npz and diff_fakes.npz). It's the whole arms race in one script.

It reports two numbers: accuracy on the generator it trained on, and accuracy on the generator it has never seen. The gap is the point.

# same setup as §2 — and run these IN ORDER: step 3 reads the fakes
# steps 1 and 2 write, so wait for each to finish (squeue --me) first
sbatch run_hpc.slurm gan_faces.py --steps 4000        # 1. make GAN fakes
sbatch run_hpc.slurm diffusion_faces.py --steps 12000 # 2. make diffusion fakes
sbatch run_hpc.slurm detect_faces.py                  # 3. train + test the detector
# accuracies print to microfake-<jobid>.out; scp spectra.png down for the fingerprints

What to look for: ~99% on the generator it trained on, but near 50% — pure chance — on the unseen generator. Each generator leaves its own fingerprint, so a detector tuned to one is blind to the next. That is why passive detection loses the arms race and provenance (C2PA, SynthID) is the real defense — the course's defense-in-depth spine.

Run it yourself: ↓ detect_faces.py ↓ run_hpc.slurm

7 · From text to image · how a sentence steers the denoiser

One piece of the modern story is still missing. §4 built a diffusion model that generates faces — but Stable Diffusion and DALL·E generate "an astronaut riding a horse" from a typed sentence. This section adds that piece, and it is smaller than you might expect: the sentence becomes one more input to the denoiser, plus one sampling-time trick to make the model actually listen.

7.1 Condition the denoiser · the caption is one more input

Recall the §4.2 denoiser: $D_\theta(x_\sigma, \sigma)$ — noisy image and noise level in, clean-image guess out. To make it text-aware, train on (image, caption) pairs scraped from the web, and hand the caption in as a third input:

$$\mathcal{L}(\theta) \;=\; \mathbb{E}\;\big\lVert\, D_\theta(x_0 + \sigma\varepsilon,\;\sigma,\;c) \;-\; x_0 \,\big\rVert^2$$

$c$ is the caption. Same loss as §4.2 — one extra input, nothing else changes. The denoiser now learns "what do astronaut pixels look like under this much noise?"

A caption can't enter a network as raw letters, so a text encoder turns it into embedding vectors first — the same tokens-to-vectors move you built in microgpt. In Stable Diffusion that encoder is CLIP, pre-trained on ~400 million (image, caption) pairs to place matching images and captions near each other in embedding space. Inside the denoiser, every layer reads the caption vectors with cross-attention: attention where the queries come from the image being denoised and the keys and values come from the caption's tokens — each image region asking "which words are about me?"

Trained this way, the model can follow text — but only weakly. Web captions describe their images loosely, so "cat" makes cat-ness only somewhat more likely, and samples drift off-prompt. The fix is one of the neatest tricks in the field.

7.2 Classifier-free guidance · exaggerate what the text changes

It starts with a small oddity in training: the caption is randomly dropped about 10% of the time (replaced by an empty string). So one network ends up knowing two jobs — a text-conditioned denoiser $D(x_\sigma, \sigma, c)$ and an unconditional one $D(x_\sigma, \sigma, \varnothing)$ — and at sampling time you can run both on the same noisy image and subtract. Look at what that difference is. Everything about the two calls is identical except the caption, so $D(c) - D(\varnothing)$ is exactly what the caption changes: an arrow pointing from "generic image" toward "image of $c$."

Now the trick: if the model follows the text too weakly, don't accept its step along that arrow — exaggerate it, going $s$ times as far. That is classifier-free guidance, and every modern text-to-image system uses it. At each denoising step, run the denoiser twice — once with the caption, once without — and blend:

$$\hat{x}_0^{\text{guided}} \;=\; \hat{x}_0(\varnothing) \;+\; s \cdot \big(\hat{x}_0(c) - \hat{x}_0(\varnothing)\big)$$

Read it as: start from what you'd paint with no text, find the direction the caption pulls, and go $s$ times as far. $s = 0$ ignores the prompt; $s = 1$ is the plain conditional model; real systems run $s \approx 5$–$10$. (Papers write the same update on the noise guess $\hat\varepsilon$ — at a fixed $\sigma$ it is the same extrapolation.)

Before you touch the widget, commit to one prediction: if a little exaggeration helps the model listen, what goes wrong with a lot — say $10\times$? Then test yourself:

Assignment · finish the generator, then think like a defender

You've read every model in this lab; now build the one that matters. We hand you a complete micro-diffusion generator — the MLP, the training loop, the EDM preconditioning, the data — with exactly the two functions that are diffusion left blank. Implement them, pass the autograder, then write a short defender's analysis tying your working generator to §6.

Part 1 · Build — complete microfake.py (~45 min)

Open microfake.py and implement the two functions in the YOUR JOB block. Both are 2-D vector operations (work component-wise over k = 0, 1) — the whole assignment is two short returns, but you have to understand §4 to get them right.

Run and grade locally, from the folder holding your microfake.py:

python3 microfake.py          # trains, then prints the avg distance of samples to the data
python3 test_microfake.py     # the autograder (4 checks)

The autograder runs four checks — two fast exact-value tests, one boundary case, and one end-to-end generation test:

• precond_target returns the EDM target (exact: precond_target([0.8,0], [0.2,0], 0.5, 0.4) must equal [1.75, 0.0]).
• ddim_step takes the right partial step (exact: ddim_step([0.2,0], [0.8,0], 0.5, 0.3) → [0.44, 0.0]).
• ddim_step lands exactly on x0_hat when sigma_next = 0.
• generation works end-to-end: after a short training run, the model's samples land near the data manifold (avg distance < 0.18).

Part 2 · Think like a defender — reflection.md (~20 min)

Your generator now matches a data distribution. Write a short page answering three questions, each tying your build to a specific idea from the lab:

1. Stability. §3 trained a GAN by an adversarial game; you trained this by plain regression. Point to the line in microfake.py that makes diffusion training stable, and explain in two sentences why it can't mode-collapse the way a GAN can.
2. Detection. Suppose a defender uses the §6.4 likelihood test — your own denoiser — to decide if a point is real or fake. As your model trains better, does that detector get more or less able to flag your samples? Justify it (this is the §3 / §6 equilibrium argument).
3. The real defense. Given your answer to (2), argue in three sentences why §6 concludes that provenance (signing/watermarking) is a more durable defense than any pixel-level detector.

Bonus (+10 pts) · push the generator

Pick one and document it in your reflection: (a) change DATA to a new shape (e.g., three blobs, a spiral) and show samples still converge; (b) cut the sampling steps in sample() and report the steps-vs-quality trade you saw (the §4.3 question, made quantitative); or (c) add stochastic "churn" to ddim_step and describe how it changes the samples.

Submission

Submit microfake.py (with both functions implemented) and reflection.md. The Gradescope autograder runs a superset of test_microfake.py — passing locally is necessary but not sufficient.

Rubric

FAQ

Is this lab going to teach me to make harmful deepfakes?

No. The in-browser widgets run on toy 2D points and tiny abstract images — smileys and spirals. The "Now you run it" companions do use real photographs, deliberately chosen to be ethically clean: the Olivetti faces, 40 volunteers photographed for research at AT&T Laboratories Cambridge in 1992–94, distributed for exactly this kind of use. Nothing here works on a photo of someone who didn't consent, and nothing approaches the resolution or realism of a harmful fake. The goal is to understand the machinery so you can reason about it as a security professional: what's cheap, what's hard, and how detection works (§6). Understanding the generator is the prerequisite to building the detector.

Do I need a GPU or the HPC cluster?

Only for the "Now you run it" blocks — and even then, not strictly. The in-browser widgets need nothing. The pure-stdlib scripts (gan.py, diffusion.py, …) run on any Python 3, including the cluster login node's default python3 with no modules loaded. The real-face companions are written for the class allocation (ds6042) with the cluster's prebuilt PyTorch container, but they auto-detect the device: on any machine with PyTorch installed they run on CPU in a few minutes, and a GPU just makes them fast.

Why build three different models instead of just the best one?

Because each was invented to fix the previous one's flaw, and seeing that progression is the lesson. Autoencoders gave us the face-swap but produce blurry output; GANs sharpened it but are unstable and mode-collapse; diffusion is stable and high-quality but slow. You can't appreciate why diffusion looks the way it does without having felt the problems it solves.