You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
92 lines
2.6 KiB
92 lines
2.6 KiB
#import "@preview/typslides:1.2.5": *
|
|
|
|
#show: typslides.with(
|
|
ratio: "16-9",
|
|
theme: "bluey",
|
|
)
|
|
|
|
#front-slide(
|
|
title: "Diffusion and joint laws",
|
|
subtitle: ["And how time might help learn them"],
|
|
authors: ("Ramon"),
|
|
info: [Code: #link("https://git.ramoncalvo.com/noctrog/flow-points")],
|
|
)
|
|
|
|
// #table-of-contents()
|
|
|
|
//#title-slide[
|
|
//Section 1: Introduction
|
|
//]
|
|
|
|
//#slide[
|
|
// == Overview
|
|
// - Background and motivation
|
|
//- Research question
|
|
// - Objectives
|
|
//]
|
|
|
|
#slide(title: "Data generation process")[
|
|
- Visualization of $p_"init"$. We want the diffusion te learn the 2 constraints (annulus and wedge).
|
|
#image("./results/p_data.png")
|
|
]
|
|
|
|
#slide(title: "Flow matching after 1.000.000 steps")[
|
|
#image("./results/mlp_l4_h64/scatter_1000000.png")
|
|
]
|
|
|
|
#slide(title: "Flow matching after 2.000.000 steps")[
|
|
#image("./results/mlp_l4_h64/scatter_2000000.png")
|
|
]
|
|
|
|
#slide(title: "Flow matching after 3.000.000 steps")[
|
|
#image("./results/mlp_l4_h64/scatter_3000000.png")
|
|
]
|
|
|
|
#slide(title: "Flow matching after 4.000.000 steps")[
|
|
#image("./results/mlp_l4_h64/scatter_4000000.png")
|
|
]
|
|
|
|
#slide(title: "Flow matching after 5.000.000 steps")[
|
|
#image("./results/mlp_l4_h64/scatter_4900000.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion p_data")[
|
|
- Generation process: sample form $z_0 ~ p_"data"$, do 1 brownian step and $z_1 ~ z_0 + cal(N)(0, sigma^2)$
|
|
#image("./results/brownian.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion after 1.000.000 steps ")[
|
|
#image("./results/mlp_b_l4_h64/scatter_1000000.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion after 2.000.000 steps ")[
|
|
#image("./results/mlp_b_l4_h64/scatter_2000000.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion after 3.000.000 steps ")[
|
|
#image("./results/mlp_b_l4_h64/scatter_3000000.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion after 4.000.000 steps ")[
|
|
#image("./results/mlp_b_l4_h64/scatter_4000000.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion after 5.000.000 steps ")[
|
|
#image("./results/mlp_b_l4_h64/scatter_4900000.png")
|
|
]
|
|
|
|
#slide(title: "Brownian motion training")[
|
|
- The vanilla Flow maching loss is:
|
|
$
|
|
cal(L)_"CFM" = EE_(t ~ cal(U)(0, 1) \ z ~ p_"data" \ x ~ p_t (dot | z))[ || u_t^theta (x) - u_t^"target" (x | z) ||^2 ]
|
|
$
|
|
- The new Flow Matching loss becomes:
|
|
$
|
|
cal(L)_"CFM" = EE_(t ~ cal(U)(0, 1) \ z_0 ~ p_"data" \ z_1 ~ z_0 + cal(N)(0, sigma^2) \ x ~ p_t (dot | z_1))[ ||u_t^theta (x | z_0) - u_t^"target" (x | z_1) ||^2 ]
|
|
$
|
|
]
|
|
|
|
#slide(title: "Vanilla Flow Matching vs. Brownian Mov. Flow Matching (BFM)")[
|
|
- BFM: worse annulus constraint compliance, better wedge constraint compliance.
|
|
#image("./results/grouped_experiments_constraint_stats.png")
|
|
]
|
|
|