You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/index.rst
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,8 +4,9 @@ Unit Scaling
4
4
Welcome to the :code:`unit-scaling` library. This library is designed to facilitate
5
5
the use of the *unit scaling* and *u-µP* methods, as outlined in the papers
6
6
`Unit Scaling: Out-of-the-Box Low-Precision Training (ICML, 2023)
7
-
<https://arxiv.org/abs/2303.11257>`_ and `u-μP: The Unit-Scaled Maximal Update Parametrization
8
-
<https://arxiv.org/abs/2407.17465>`_.
7
+
<https://arxiv.org/abs/2303.11257>`_ and
8
+
`u-μP: The Unit-Scaled Maximal Update Parametrization
9
+
<https://arxiv.org/abs/2407.17465>`_
9
10
10
11
For a demonstration of the library, see `u-μP using the unit_scaling library
11
12
<https://github.com/graphcore-research/unit-scaling/blob/main/examples/demo.ipynb>`_ — a notebook showing the definition and training of a u-µP language model, comparing against Standard Parametrization (SP).
0 commit comments