complete addition of update methods + other tweaks

Author: ablaom

docs/make.jl

@@ -15,11 +15,11 @@ makedocs(
         "Anatomy of an Implementation" => "anatomy_of_an_implementation.md",
         "Reference" => [
             "Overview" => "reference.md",
-            "fit" => "fit.md",
+            "fit/update" => "fit.md",
             "predict/transform" => "predict_transform.md",
             "Kinds of Target Proxy" => "kinds_of_target_proxy.md",
             "minimize" => "minimize.md",
-            "target/weights/input" => "target_weights_input.md",
+            "target/weights/features" => "target_weights_features.md",
             "obs" => "obs.md",
             "Accessor Functions" => "accessor_functions.md",
             "Algorithm Traits" => "traits.md",

docs/src/accessor_functions.md

@@ -1,6 +1,7 @@
 # [Accessor Functions](@id accessor_functions)
 
-The sole argument of an accessor function is the output, `model`, of [`fit`](@ref).
+The sole argument of an accessor function is the output, `model`, of
+[`fit`](@ref). Algorithms are free to implement any number of these, or none of them.
 
 - [`LearnAPI.algorithm(model)`](@ref)
 - [`LearnAPI.extras(model)`](@ref)
@@ -15,6 +16,9 @@ The sole argument of an accessor function is the output, `model`, of [`fit`](@re
 - [`LearnAPI.training_scores(model)`](@ref)
 - [`LearnAPI.components(model)`](@ref)
 
+Algorithm-specific accessor functions may also be implemented. The names of all accessor
+functions are included in the list returned by [`LearnAPI.functions(algorithm)`](@ref).
+
 ## Implementation guide
 
 All new implementations must implement [`LearnAPI.algorithm`](@ref). While, all others are

docs/src/anatomy_of_an_implementation.md

@@ -5,26 +5,34 @@ regression](https://en.wikipedia.org/wiki/Ridge_regression) with no intercept. T
 workflow we want to enable has been previewed in [Sample workflow](@ref). Readers can also
 refer to the [demonstration](@ref workflow) of the implementation given later.
 
-For a transformer, implementations ordinarily implement `transform` instead of
+A transformer ordinarily implements `transform` instead of
 `predict`. For more on `predict` versus `transform`, see [Predict or transform?](@ref)
 
 !!! note
 
     New implementations of `fit`, `predict`, etc,
     always have a *single* `data` argument, as in
         `LearnAPI.fit(algorithm, data; verbosity=1) = ...`.
-    For convenience, user calls like `fit(algorithm, X, y)` automatically fallback
+    For convenience, user-calls, such as `fit(algorithm, X, y)`, automatically fallback
         to `fit(algorithm, (X, y))`.
 
 !!! note
 
+    By default, it is assumed that `data` supports the [`LearnAPI.RandomAccess`](@ref)
+    interface; this includes all matrices, with observations-as-columns, most tables, and
+    tuples thereof). See [`LearnAPI.RandomAccess`](@ref) for details. If this is not the
+    case then an implementation must either: 
+
     If the `data` object consumed by `fit`, `predict`, or `transform` is not
     not a suitable table¹, array³, tuple of tables and arrays, or some
     other object implementing
     the MLUtils.jl `getobs`/`numobs` interface,
-    then an implementation must: (i) suitably overload the trait
-    [`LearnAPI.data_interface`](@ref); and/or (ii) overload [`obs`](@ref), as
-     illustrated below under [Providing an advanced data interface](@ref).
+    then an implementation must: (i) overload [`obs`](@ref) to articulate how
+    provided data can be transformed into a form that does support
+    it, as illustrated below under 
+	[Providing an advanced data interface](@ref); or (ii) overload the trait
+    [`LearnAPI.data_interface`](@ref) to specify a more relaxed data
+    API. 
 
 The first line below imports the lightweight package LearnAPI.jl whose methods we will be
 extending. The second imports libraries needed for the core algorithm.
@@ -152,9 +160,9 @@ from training data, by implementing [`LearnAPI.target`](@ref):
 LearnAPI.target(algorithm, data) = last(data)
 ```
 
-There is a similar method, [`LearnAPI.input`](@ref) for declaring how input data can be
-extracted (for passing to `predict`, for example) but this method has a fallback which
-typically suffices: return `first(data)` if `data` is a tuple, and otherwise return
+There is a similar method, [`LearnAPI.features`](@ref) for declaring how training features
+can be extracted (for passing to `predict`, for example) but this method has a fallback
+which typically suffices: return `first(data)` if `data` is a tuple, and otherwise return
 `data`.
 
 
@@ -218,7 +226,7 @@ A macro provides a shortcut, convenient when multiple traits are to be defined:
         :(LearnAPI.algorithm),
         :(LearnAPI.minimize),
         :(LearnAPI.obs),
-        :(LearnAPI.input),
+        :(LearnAPI.features),
         :(LearnAPI.target),
         :(LearnAPI.predict),
         :(LearnAPI.coefficients),
@@ -325,7 +333,7 @@ LearnAPI.minimize(model::RidgeFitted) =
         :(LearnAPI.algorithm),
         :(LearnAPI.minimize),
         :(LearnAPI.obs),
-        :(LearnAPI.input),
+        :(LearnAPI.features),
         :(LearnAPI.target),
         :(LearnAPI.predict),
         :(LearnAPI.coefficients),
@@ -423,7 +431,7 @@ LearnAPI.predict(model::RidgeFitted, ::LiteralTarget, Xnew) =
     predict(model, LiteralTarget(), obs(model, Xnew))
 ```
 
-### `target` and `input` methods
+### `target` and `features` methods
 
 We provide an additional overloading of [`LearnAPI.target`](@ref) to handle the additional
 supported data argument of `fit`:
@@ -432,11 +440,11 @@ supported data argument of `fit`:
 LearnAPI.target(::Ridge, observations::RidgeFitObs) = observations.y
 ```
 
-Similarly, we must overload [`LearnAPI.input`](@ref), which extracts inputs from training
-data (objects that can be passed to `predict`) like this
+Similarly, we must overload [`LearnAPI.features`](@ref), which extracts features from
+training data (objects that can be passed to `predict`) like this
 
 ```@example anatomy2
-LearnAPI.input(::Ridge, observations::RidgeFitObs) = observations.A
+LearnAPI.features(::Ridge, observations::RidgeFitObs) = observations.A
 ```
 as the fallback mentioned above is no longer adequate.
 
@@ -482,6 +490,9 @@ ẑ = predict(model, MLUtils.getobs(observations_for_predict, test))
 @assert ẑ == ŷ
 ```
 
+For an application of [`obs`](@ref) to efficient cross-validation, see [here](@ref
+obs_workflows).
+
 ---
 
 ¹ In LearnAPI.jl a *table* is any object `X` implementing the

docs/src/fit.md

@@ -1,22 +1,28 @@
-# [`fit`](@ref fit)
+# [`fit`, `update`, `update_observations`, and `update_features`](@id fit)
 
-Training for the first time:
+### Training
 
 ```julia
 fit(algorithm, data; verbosity=1) -> model
 fit(algorithm; verbosity=1) -> static_model 
 ```
 
-Updating:
+A "static" algorithm is one that does not generalize to new observations (e.g., some
+clustering algorithms); there is no trainiing data and the algorithm is executed by
+`predict` or `transform` which receive the data. See example below.
+
+When `fit` expects a tuple form of argument, `data = (X1, ..., Xn)`, then the signature
+`fit(algorithm, X1, ..., Xn)` is also provided.
+
+### Updating
 
 ```
-fit(model, data; verbosity=1, param1=new_value1, param2=new_value2, ...) -> updated_model
-fit(model, NewObservations(), new_data; verbosity=1, param1=new_value1, ...) -> updated_model
-fit(model, NewFeatures(), new_data; verbosity=1, param1=new_value1, ...) -> updated_model
+update(model, data; verbosity=1, param1=new_value1, param2=new_value2, ...) -> updated_model
+update_observations(model, new_data; verbosity=1, param1=new_value1, ...) -> updated_model
+update_features(model, new_data; verbosity=1, param1=new_value1, ...) -> updated_model
 ```
 
-When `fit` expects a tuple form of argument, `data = (X1, ..., Xn)`, then the signature
-`fit(algorithm, X1, ..., Xn)` is also provided. 
+Data slurping forms are similarly provided for updating methods.
 
 ## Typical workflows
 
@@ -27,46 +33,55 @@ algorithm = Algorithm(n=100)
 model = fit(algorithm, (X, y)) # or `fit(algorithm, X, y)`
 
 # Predict probability distributions:
-ŷ = predict(model, Distribution(), Xnew)
+ŷ = predict(model, Distribution(), Xnew) 
 
 # Inspect some byproducts of training:
 LearnAPI.feature_importances(model)
 
 # Add 50 iterations and predict again:
-model = fit(model; n=150)
+model = update(model; n=150)
 predict(model, Distribution(), X)
 ```
 
 ### A static algorithm (no "learning")
 
 ```julia
 # Apply some clustering algorithm which cannot be generalized to new data:
-model = fit(algorithm)
-labels = predict(model, LabelAmbiguous(), X) # mutates `model`
+model = fit(algorithm) # no training data
+labels = predict(model, LabelAmbiguous(), X) # may mutate `model`
+
+# Or, in one line:
+labels = predict(algorithm, LabelAmbiguous(), X)
 
-# inspect byproducts of the clustering algorithm (e.g., outliers):
+# But two-line version exposes byproducts of the clustering algorithm (e.g., outliers):
 LearnAPI.extras(model)
 ```
 
 ## Implementation guide
 
-Initial training: 
+### Training
 
 | method                                                                         | fallback                                                         | compulsory?        |
 |:-------------------------------------------------------------------------------|:-----------------------------------------------------------------|--------------------|
 | [`fit`](@ref)`(algorithm, data; verbosity=1)`                                  | ignores `data` and applies signature below                       | yes, unless static |
 | [`fit`](@ref)`(algorithm; verbosity=1)`                                        | none                                                             | no, unless static  |
 
-Updating:
+### Updating
+
+| method                                                                               | fallback | compulsory? |
+|:-------------------------------------------------------------------------------------|:---------|-------------|
+| [`update`](@ref)`(model, data; verbosity=1, hyperparameter_updates...)`              | none     | no          |
+| [`update_observations`](@ref)`(model, data; verbosity=1, hyperparameter_updates...)` | none     | no          |
+| [`update_features`](@ref)`(model, data; verbosity=1, hyperparameter_updates...)`     | none     | no          |
 
-| method                                                                         | fallback                                                                   | compulsory? |
-|:-------------------------------------------------------------------------------|:---------------------------------------------------------------------------|-------------|
-| [`fit`](@ref)`(model, data; verbosity=1, param_updates...)`                    | retrains from scratch on `data` with specified hyperparameter replacements | no          |
-| [`fit`](@ref)`(model, ::NewObservations, data; verbosity=1, param_updates...)` | none                                                                       | no          |
-| [`fit`](@ref)`(model, ::NewFeatures, data; verbosity=1, param_updates...)`     | none                                                                       | no          |
+There are some contracts regarding the behaviour of the update methods, as they relate to
+a previous `fit` call. Consult the document strings for details.
 
 ## Reference
 
 ```@docs
-LearnAPI.fit
+fit
+update
+update_observations
+update_features
 ```

docs/src/index.md

@@ -9,12 +9,14 @@ A base Julia interface for machine learning and statistics </span>
 <br>
 ```
 
-LearnAPI.jl is a lightweight, functional-style interface, providing a collection of
-[methods](@ref Methods), such as `fit` and `predict`, to be implemented by algorithms from
-machine learning and statistics. Through such implementations, these algorithms buy into
-functionality, such as hyperparameter optimization and model composition, as provided by
-ML/statistics toolboxes and other packages. LearnAPI.jl also provides a number of Julia
-[traits](@ref traits) for promising specific behavior.
+LearnAPI.jl is a lightweight, functional-style interface, providing a
+collection of [methods](@ref Methods), such as `fit` and `predict`, to be implemented by
+algorithms from machine learning and statistics. Through such implementations, these
+algorithms buy into functionality, such as hyperparameter optimization and model
+composition, as provided by ML/statistics toolboxes and other packages. LearnAPI.jl also
+provides a number of Julia [traits](@ref traits) for promising specific behavior.
+
+LearnAPI.jl has no package dependencies.
 
 ```@raw html
 &#128679;
@@ -41,15 +43,18 @@ X = <some training features>
 y = <some training target>
 Xnew = <some test or production features>
 
+# List LearnaAPI functions implemented for `forest`:
+LearnAPI.functions(forest)
+
 # Train:
 model = fit(forest, X, y)
 
+# Generate point predictions:
+ŷ = predict(model, Xnew) # or `predict(model, LiteralTarget(), Xnew)`
+
 # Predict probability distributions:
 predict(model, Distribution(), Xnew)
 
-# Generate point predictions:
-ŷ = predict(model, LiteralTarget(), Xnew) # or `predict(model, Xnew)`
-
 # Apply an "accessor function" to inspect byproducts of training:
 LearnAPI.feature_importances(model)
 
@@ -77,13 +82,14 @@ data_interface) (read as "observations") gives users and meta-algorithms access
 algorithm-specific representation of input data, which is also guaranteed to implement a
 standard interface for accessing individual observations, unless the algorithm explicitly
 opts out. Moreover, the `fit` and `predict` methods will also be able to consume these
-alternative data representations.
+alternative data representations, for performance benefits in some situations.
 
 The fallback data interface is the [MLUtils.jl](https://github.com/JuliaML/MLUtils.jl)
-`getobs/numobs` interface, and if the input consumed by the algorithm already implements
-that interface (tables, arrays, etc.) then overloading `obs` is completely optional. Plain
-iteration interfaces, with or without knowledge of the number of observations, can also be
-specified (to support, e.g., data loaders reading images from disk).
+`getobs/numobs` interface (here tagged as [`LearnAPI.RandomAccess()`](@ref)) and if the
+input consumed by the algorithm already implements that interface (tables, arrays, etc.)
+then overloading `obs` is completely optional. Plain iteration interfaces, with or without
+knowledge of the number of observations, can also be specified (to support, e.g., data
+loaders reading images from disk).
 
 ## Learning more
 

docs/src/kinds_of_target_proxy.md

@@ -47,7 +47,7 @@ expectiles at 50% will provide `LiteralTarget` instead.
 > Table of concrete subtypes of `LearnAPI.IID <: LearnAPI.KindOfProxy`.
 
 
-## Proxies for distribution-fitting algorithms
+## Proxies for density estimation lgorithms
 
 ```@docs
 LearnAPI.Single

docs/src/obs.md

@@ -11,7 +11,7 @@ obs(algorithm, data) # can be passed to `fit` instead of `data`
 obs(model, data)     # can be passed to `predict` or `transform` instead of `data`
 ```
 
-## Typical workflows
+## [Typical workflows](@id obs_workflows)
 
 LearnAPI.jl makes no universal assumptions about the form of `data` in a call
 like `fit(algorithm, data)`. However, if we define
@@ -46,7 +46,7 @@ import MLUtils
 algorithm = <some supervised learner>
 
 data = <some data that `fit` can consume, with 30 observations>
-X = LearnAPI.input(algorithm, data)
+X = LearnAPI.features(algorithm, data)
 y = LearnAPI.target(algorithm, data)
 
 train_test_folds = map([1:10, 11:20, 21:30]) do test