You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
- `rng`: Random number generator internally used by the algorithm.
31
+
- `iteration`: The index of the current iteration.
32
+
- `restructure`: Function that restructures the variational approximation from the variational parameters. Calling `restructure(params)` reconstructs the current variational approximation.
33
+
- `params`: Current variational parameters.
34
+
- `averaged_params`: Variational parameters averaged according to the averaging strategy.
35
+
- `gradient`: The estimated (possibly stochastic) gradient.
36
+
21
37
# Requirements
22
38
- The trainable parameters in the variational approximation are expected to be extractable through `Optimisers.destructure`. This requires the variational approximation to be marked as a functor through `Functors.@functor`.
23
39
- The variational approximation ``q_{\\lambda}`` implements `rand`.
24
40
- The target distribution and the variational approximation have the same support.
25
41
- The target `LogDensityProblems.logdensity(prob, x)` must be differentiable with respect to `x` by the selected AD backend.
26
42
- Additonal requirements on `q` may apply depending on the choice of `entropy`.
- `rng`: Random number generator internally used by the algorithm.
120
+
- `iteration`: The index of the current iteration.
121
+
- `restructure`: Function that restructures the variational approximation from the variational parameters. Calling `restructure(params)` reconstructs the current variational approximation.
122
+
- `params`: Current variational parameters.
123
+
- `averaged_params`: Variational parameters averaged according to the averaging strategy.
124
+
- `gradient`: The estimated (possibly stochastic) gradient.
125
+
66
126
# Requirements
67
127
- The variational family is `MvLocationScale`.
68
128
- The target distribution and the variational approximation have the same support.
69
129
- The target `LogDensityProblems.logdensity(prob, x)` must be differentiable with respect to `x` by the selected AD backend.
70
130
- Additonal requirements on `q` may apply depending on the choice of `entropy_zerograd`.
- `rng`: Random number generator internally used by the algorithm.
207
+
- `iteration`: The index of the current iteration.
208
+
- `restructure`: Function that restructures the variational approximation from the variational parameters. Calling `restructure(params)` reconstructs the current variational approximation.
209
+
- `params`: Current variational parameters.
210
+
- `averaged_params`: Variational parameters averaged according to the averaging strategy.
211
+
- `gradient`: The estimated (possibly stochastic) gradient.
212
+
109
213
# Requirements
110
214
- The trainable parameters in the variational approximation are expected to be extractable through `Optimisers.destructure`. This requires the variational approximation to be marked as a functor through `Functors.@functor`.
111
215
- The variational approximation ``q_{\\lambda}`` implements `rand`.
112
216
- The variational approximation ``q_{\\lambda}`` implements `logpdf(q, x)`, which should also be differentiable with respect to `x`.
113
217
- The target distribution and the variational approximation have the same support.
Copy file name to clipboardExpand all lines: src/algorithms/paramspacesgd/paramspacesgd.jl
+29-68Lines changed: 29 additions & 68 deletions
Original file line number
Diff line number
Diff line change
@@ -1,68 +1,9 @@
1
1
2
-
"""
3
-
ParamSpaceSGD(
4
-
objective::AbstractVariationalObjective,
5
-
adtype::ADTypes.AbstractADType,
6
-
optimizer::Optimisers.AbstractRule,
7
-
averager::AbstractAverager,
8
-
operator::AbstractOperator,
9
-
)
10
-
11
-
This algorithm applies stochastic gradient descent (SGD) to the variational `objective` over the (Euclidean) space of variational parameters.
12
-
13
-
The trainable parameters in the variational approximation are expected to be extractable through `Optimisers.destructure`.
14
-
This requires the variational approximation to be marked as a functor through `Functors.@functor`.
15
-
16
-
!!! note
17
-
Different objective may impose different requirements on `adtype`, variational family, `optimizer`, and `operator`. It is therefore important to check the documentation corresponding to each specific objective. Essentially, each objective should be thought as forming its own unique algorithm.
18
-
19
-
# Arguments
20
-
- `objective`: Variational Objective.
21
-
- `adtype`: Automatic differentiation backend.
22
-
- `optimizer`: Optimizer used for inference.
23
-
- `averager` : Parameter averaging strategy.
24
-
- `operator` : Operator applied to the parameters after each optimization step.
25
-
26
-
# Output
27
-
- `q_averaged`: The variational approximation formed from the averaged SGD iterates.
28
-
29
-
# Callback
30
-
The callback function `callback` has a signature of
- `rng`: Random number generator internally used by the algorithm.
36
-
- `iteration`: The index of the current iteration.
37
-
- `restructure`: Function that restructures the variational approximation from the variational parameters. Calling `restructure(params)` reconstructs the current variational approximation.
38
-
- `params`: Current variational parameters.
39
-
- `averaged_params`: Variational parameters averaged according to the averaging strategy.
40
-
- `gradient`: The estimated (possibly stochastic) gradient.
0 commit comments