Add some basic tutorials

[Red-Portal]

This PR does the following:

• Update the basic example (also in the README) to be more realistic.
• Add a Stan model example, which is now possible thanks to the MixedAD PR (Yay!)
• Add a normalizing flow example.
• Add a subsampling example with a (relatively) large dataset.

[github-actions]

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

JuliaFormatter

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Lines 118 to 119 in f9ab239

	import DifferentiationInterface
	import LogDensityProblemsAD

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 126 in f9ab239

```@example basic

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 133 in f9ab239

The bijector can now be applied to `q` to match the support of the target problem.

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 134 in f9ab239

```@example basic

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 140 in f9ab239

We can now run VI:

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 141 in f9ab239

```@example basic

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Lines 144 to 148 in f9ab239

	alg,
	max_iter,
	model_ad,
	q_transformed;
	show_progress=false,

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 160 in f9ab239

plot([i.iteration for i in info], [i.elbo for i in info]; xlabel="Iteration", ylabel="ELBO", label=nothing)

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 179 in f9ab239

import StatsFuns

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 188 in f9ab239

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Lines 190 to 192 in f9ab239

	samples = rand(q_avg, n_samples)
	β = samples[1:end-1,:]

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Lines 194 to 196 in f9ab239

	logit = model.X*β
	pred_prob = mean(StatsFuns.logistic.(logit), dims=2)
	y_pred = pred_prob .> 0.5

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 200 in f9ab239

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 205 in f9ab239

(elbo_callback = elbo_callback, accuracy = acc,)

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 212 in f9ab239

Note that the interface for the callback function will depend on the VI algorithm being used.

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 216 in f9ab239

```@example basic

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Lines 219 to 224 in f9ab239

	alg,
	max_iter,
	model_ad,
	q_transformed;
	show_progress=false,
	callback=callback,

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 230 in f9ab239

```@example basic

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 243 in f9ab239



---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 249 in f9ab239

```@example basic

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 251 in f9ab239

plot(t_callback, acc_callback; xlabel="Iteration", ylabel="Prediction Accuracy", label=nothing)

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 255 in f9ab239



---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/basic.md

Line 257 in f9ab239

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 1 in f9ab239

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 7 in f9ab239

## Problem Setup

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 13 in f9ab239

model_src = """

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 34 in f9ab239

```@example stan

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Lines 35 to 36 in f9ab239

	import OpenML
	import DataFrames

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 43 in f9ab239

stan_data = (X=transpose(X), y=y, N=size(X,1), D=size(X,2))

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 48 in f9ab239

```@example stan

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 49 in f9ab239

import JSON

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 63 in f9ab239

```@example stan

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 64 in f9ab239

import StanLogDensityProblems

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 71 in f9ab239

```@example stan

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 78 in f9ab239

alg = KLMinRepGradProxDescent(ADTypes.AutoReverseDiff(), optimizer=DoG())

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Lines 84 to 90 in f9ab239

	q_out, info, _ = AdvancedVI.optimize(
	alg,
	max_iter,
	model,
	q;
	show_progress=false,
	)

---

[JuliaFormatter] _{reported by reviewdog 🐶}

AdvancedVI.jl/docs/src/tutorials/stan.md

Line 98 in f9ab239

[github-actions]

AdvancedVI.jl documentation for PR #195 is available at:
https://TuringLang.github.io/AdvancedVI.jl/previews/PR195/

[github-actions]

Benchmark Results

Benchmark suite	Current: a6bee5c	Previous: 3494495	Ratio
normal/RepGradELBO + STL/meanfield/Zygote	3926634559 ns	3996757467 ns	0.98
normal/RepGradELBO + STL/meanfield/ReverseDiff	1095369782 ns	1140047768 ns	0.96
normal/RepGradELBO + STL/meanfield/Mooncake	1227522224 ns	1230760781 ns	1.00
normal/RepGradELBO + STL/fullrank/Zygote	3922841597 ns	3936331638.5 ns	1.00
normal/RepGradELBO + STL/fullrank/ReverseDiff	1559768677 ns	1641473569.5 ns	0.95
normal/RepGradELBO + STL/fullrank/Mooncake	1267644104 ns	1252509818 ns	1.01
normal/RepGradELBO/meanfield/Zygote	2792369270 ns	2793834189 ns	1.00
normal/RepGradELBO/meanfield/ReverseDiff	757805271 ns	787744069 ns	0.96
normal/RepGradELBO/meanfield/Mooncake	1093440132 ns	1093725423 ns	1.00
normal/RepGradELBO/fullrank/Zygote	2802181226 ns	2807135945 ns	1.00
normal/RepGradELBO/fullrank/ReverseDiff	933451124 ns	971111099 ns	0.96
normal/RepGradELBO/fullrank/Mooncake	1136995207 ns	1146732313 ns	0.99
normal + bijector/RepGradELBO + STL/meanfield/Zygote	5521524597 ns	5608462304 ns	0.98
normal + bijector/RepGradELBO + STL/meanfield/ReverseDiff	2367096923 ns	2417067769 ns	0.98
normal + bijector/RepGradELBO + STL/meanfield/Mooncake	4094493188.5 ns	4104543374 ns	1.00
normal + bijector/RepGradELBO + STL/fullrank/Zygote	5547829229 ns	5672628843 ns	0.98
normal + bijector/RepGradELBO + STL/fullrank/ReverseDiff	3008094862.5 ns	3036403182 ns	0.99
normal + bijector/RepGradELBO + STL/fullrank/Mooncake	4268212649.5 ns	4247049117 ns	1.00
normal + bijector/RepGradELBO/meanfield/Zygote	4356303801 ns	4346947957.5 ns	1.00
normal + bijector/RepGradELBO/meanfield/ReverseDiff	2069840696 ns	2050795942 ns	1.01
normal + bijector/RepGradELBO/meanfield/Mooncake	3958629341.5 ns	3967316792 ns	1.00
normal + bijector/RepGradELBO/fullrank/Zygote	4434786693 ns	4416104778.5 ns	1.00
normal + bijector/RepGradELBO/fullrank/ReverseDiff	2256610991 ns	2295255958 ns	0.98
normal + bijector/RepGradELBO/fullrank/Mooncake	4093637525 ns	4063691177.5 ns	1.01

This comment was automatically generated by workflow using github-action-benchmark.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.38%. Comparing base (c3ddbac) to head (5899b9d).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #195      +/-   ##
==========================================
- Coverage   88.98%   87.38%   -1.61%     
==========================================
  Files          18       18              
  Lines         445      444       -1     
==========================================
- Hits          396      388       -8     
- Misses         49       56       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[zuhengxu]

Looks good to me aside from a few minor comments!

[zuhengxu]

Great work and great tutorial (very nice interface for the subsampling part)! This PR looks good to me!

[Red-Portal]

@zuhengxu Thanks! @yebai how should we go from here? Should I wait for the others to take a look?

[Red-Portal]

@yebai @penelopeysm @sunxd3 Gentle ping

[Red-Portal]

@yebai @penelopeysm @sunxd3 Gentle ping

[yebai]

yes, feel free to merge