added snippets for Connection, Model Registry#160
added snippets for Connection, Model Registry#160rktraz wants to merge 3 commits intologicalclocks:mainfrom
Conversation
jimdowling
left a comment
There was a problem hiding this comment.
I think Javier should review this.
docs/index.md
Outdated
|
|
||
| HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models. | ||
|
|
||
| The library automatically configures itself based on the environment it is run. |
There was a problem hiding this comment.
This sentence doesn't mean anything.
configures itself how?
Technically, it should be
"The library automatically configures itself based on the environment it is run in."
Do you mean?
The library automatically configures itself based on the environment it is run in, either Python or Spark.
If you mean - it configures itself based on whether it is run inside Hopsworks or outside Hopsworks - in this case, just remove the sentence. Users will expect it should work inside/outside Hopsworks. We shouldn't need to tell them.
docs/index.md
Outdated
| HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models. | ||
|
|
||
| The library automatically configures itself based on the environment it is run. | ||
| However, to connect from an external Python environment additional connection information, such as host and port, is required. For more information about the setup from external environments, see the setup section. |
There was a problem hiding this comment.
If you connect to Hopsworks from an external Python environment to your own managed Hopsworks cluster, you need to provide additional connection information - the Hopsworks hostname (or IP address) and port (if it is not port 443).
| import hsml | ||
|
|
||
| # Create a connection | ||
| connection = hsml.connection() |
There was a problem hiding this comment.
put the hostname and project in the example here:
connection = hsml.connection(host="my.cluster.com", project="fraud")
docs/index.md
Outdated
| version=1, | ||
| metrics={"accuracy": 0.94}, | ||
| description="mnist model description") | ||
| model.save("/tmp/model_directory") # or /tmp/model_file |
There was a problem hiding this comment.
Add a line of code before this where you export the tensorflow model to the local directory:
"/tmp/model_directory"
Otherwise this code snippet doesn't actually upload a tensorflow model
|
|
||
| !!! example | ||
| ```python | ||
| mr = conn.get_model_registry() # Get the project's default model registry |
There was a problem hiding this comment.
| mr = conn.get_model_registry() # Get the project's default model registry | |
| mr = conn.get_model_registry() |
|
|
||
| !!! example | ||
| ```python | ||
| model.save('model.pkl') |
There was a problem hiding this comment.
| model.save('model.pkl') | |
| model.save('model.pkl') # save a single model file, or | |
| model.save('path/to/model_directory') # save a model with multiple files |
python/hsml/model.py
Outdated
| # get model object | ||
| model = mr.get_model("citibike_mlp_model", version=1) | ||
|
|
||
| # delete this model version | ||
| model.delete() |
There was a problem hiding this comment.
| # get model object | |
| model = mr.get_model("citibike_mlp_model", version=1) | |
| # delete this model version | |
| model.delete() | |
| # get model object of a specific version | |
| model = mr.get_model("mnist", version=1) | |
| # delete the model version | |
| model.delete() |
python/hsml/model_registry.py
Outdated
|
|
||
| !!! example | ||
| ```python | ||
| models = mr.get_models("churnmodel") |
There was a problem hiding this comment.
| models = mr.get_models("churnmodel") | |
| models = mr.get_models("mnist") |
| SORT_METRICS_BY = "max" # your sorting criteria | ||
|
|
||
| # get best model based on custom metrics | ||
| best_model = mr.get_best_model("citibike_mlp_model", |
There was a problem hiding this comment.
| best_model = mr.get_best_model("citibike_mlp_model", | |
| best_model = mr.get_best_model("mnist", |
| model = mr.tensorflow.create_model( | ||
| name="bitcoin_price_model", | ||
| metrics=metrics, | ||
| description="bitcoin daily price detection model.", | ||
| input_example=[1613512800000] | ||
| ) |
There was a problem hiding this comment.
| model = mr.tensorflow.create_model( | |
| name="bitcoin_price_model", | |
| metrics=metrics, | |
| description="bitcoin daily price detection model.", | |
| input_example=[1613512800000] | |
| ) | |
| model = mr.tensorflow.create_model( | |
| name="mnist_classifier", | |
| metrics=metrics, | |
| model_schema=model_schema, | |
| input_example=input_example, | |
| description="MNIST Classifier" | |
| ) |
| model = mr.torch.create_model( | ||
| name="electricity_price_prediction_model", | ||
| metrics=metrics, | ||
| description="Daily electricity price prediction model.", | ||
| input_example=n_step_window.example[0].numpy() | ||
| ) |
There was a problem hiding this comment.
| model = mr.torch.create_model( | |
| name="electricity_price_prediction_model", | |
| metrics=metrics, | |
| description="Daily electricity price prediction model.", | |
| input_example=n_step_window.example[0].numpy() | |
| ) | |
| model = mr.torch.create_model( | |
| name="mnist_classifier", | |
| metrics=metrics, | |
| model_schema=model_schema, | |
| input_example=input_example, | |
| description="MNIST Classifier" | |
| ) |
|
|
||
| HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models. | ||
|
|
||
| However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section. |
There was a problem hiding this comment.
I think we can keep the first sentence as
| However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section. | |
| The library automatically configures itself based on the environment it is run. However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section. |
docs/index.md
Outdated
| import hsml | ||
|
|
||
| # Create a connection | ||
| connection = hsml.connection(host="my.cluster.com", project="fraud") |
There was a problem hiding this comment.
| connection = hsml.connection(host="my.cluster.com", project="fraud") | |
| connection = hsml.connection() | |
| # or connection = hsml.connection(host="my.hopsworks.cluster", project="my_project") |
docs/index.md
Outdated
| import tensorflow as tf | ||
|
|
||
|
|
||
| model = mr.tensorflow.create_model(name="mnist", | ||
| version=1, | ||
| metrics={"accuracy": 0.94}, | ||
| description="mnist model description") | ||
|
|
||
| export_path = "/tmp/model_directory" # or "/tmp/model_file" | ||
| tf.saved_model.save(model, export_path) | ||
| model.save(export_path) |
There was a problem hiding this comment.
| import tensorflow as tf | |
| model = mr.tensorflow.create_model(name="mnist", | |
| version=1, | |
| metrics={"accuracy": 0.94}, | |
| description="mnist model description") | |
| export_path = "/tmp/model_directory" # or "/tmp/model_file" | |
| tf.saved_model.save(model, export_path) | |
| model.save(export_path) | |
| import tensorflow as tf | |
| export_path = "/tmp/model_directory" | |
| tf.saved_model.save(model, export_path) | |
| model = mr.tensorflow.create_model(name="mnist", | |
| version=1, | |
| metrics={"accuracy": 0.94}, | |
| description="mnist model description") | |
| model.save(export_path) # "/tmp/model_directory" or "/tmp/model_file" |
No description provided.