* Implementation: Google AutoML's Lion * Tunable hyperparameters: learning_rate, weight_decay, one_minus_beta1, beta2 * Fixed hyperparameters: warmup_factor