diff --git a/modules/develop/pages/connect/configuration/resource-management.adoc b/modules/develop/pages/connect/configuration/resource-management.adoc index caf66b2b7..96c902fb5 100644 --- a/modules/develop/pages/connect/configuration/resource-management.adoc +++ b/modules/develop/pages/connect/configuration/resource-management.adoc @@ -6,7 +6,7 @@ == Prerequisites -- A running xref:get-started:cluster-types/byoc/index.adoc[BYOC] (not BYOVPC/BYOVNet) or xref:get-started:cluster-types/dedicated/create-dedicated-cloud-cluster.adoc[Dedicated] cluster. +- A running xref:get-started:cluster-types/byoc/index.adoc[BYOC] or xref:get-started:cluster-types/dedicated/create-dedicated-cloud-cluster.adoc[Dedicated] cluster. - An estimate of the throughput of your data pipeline. You can get some basic statistics by running your data pipeline locally using the xref:redpanda-connect:components:processors/benchmark.adoc[`benchmark` processor]. === Understanding compute units @@ -143,7 +143,7 @@ Data Plane API:: . Make a request to xref:api:ROOT:cloud-dataplane-api.adoc#get-/v1/redpanda-connect/pipelines[`GET /v1/redpanda-connect/pipelines`], which lists details of all pipelines on your cluster by ID. + * Memory (`memory_shares`) is displayed in megabytes. For example, `1` compute unit is `400M` or 400 MB. -* CPU resources (`cpu_shares`) are displayed milliCPU. For example, `1` compute unit is `100m` or 0.1 CPU. +* CPU resources (`cpu_shares`) are displayed in milliCPU. For example, `1` compute unit is `100m` or 0.1 CPU. -- ===== @@ -182,14 +182,15 @@ curl -X PUT "https:///v1/redpanda-connect/pipelines/xxx..." -H 'accept: application/json'\ -H 'authorization: Bearer xxx...' \ -H "content-type: application/json" \ - -d '{"config_yaml":"input:\n generate:\n interval: 1s\n mapping: |\n root.id = uuid_v4()\n root. user.name = fake(\"name\")\n root.user.email = fake(\"email\")\n root.content = fake(\"paragraph\")\n\npipeline:\n processors:\n - mutation: |\n root.title = \"PRIVATE AND CONFIDENTIAL\"\n\noutput:\n kafka_franz:\n seed_brokers:\n - seed-j888.byoc.prd.cloud.redpanda.com:9092\n sasl:\n - mechanism: SCRAM-SHA-256\n password: password\n username: connect\n topic: processed-emails\n tls:\n enabled: true\n", \ - "description":"Email processor", \ - "display_name":"emailprocessor-pipeline", \ - "resources":{ \ - "memory_shares":"800M" \ - "cpu_shares":"200m", \ - } \ - }' + -d '{ + "config_yaml": "input:\n generate:\n interval: 1s\n mapping: |\n root.id = uuid_v4()\n root.user.name = fake(\"name\")\n root.user.email = fake(\"email\")\n root.content = fake(\"paragraph\")\n\npipeline:\n processors:\n - mutation: |\n root.title = \"PRIVATE AND CONFIDENTIAL\"\n\noutput:\n kafka_franz:\n seed_brokers:\n - seed-j888.byoc.prd.cloud.redpanda.com:9092\n sasl:\n mechanism: SCRAM-SHA-256\n password: password\n username: connect\n topic: processed-emails\n tls:\n enabled: true\n", + "description": "Email processor", + "display_name": "emailprocessor-pipeline", + "resources": { + "memory_shares": "800M", + "cpu_shares": "200m" + } + }' ---- + A successful response shows the updated resource allocations with the `cpu_shares` value returned in milliCPU. diff --git a/modules/develop/pages/connect/configuration/secret-management.adoc b/modules/develop/pages/connect/configuration/secret-management.adoc index fbca8e892..38bd9ee5e 100644 --- a/modules/develop/pages/connect/configuration/secret-management.adoc +++ b/modules/develop/pages/connect/configuration/secret-management.adoc @@ -3,23 +3,11 @@ Learn how to manage secrets in Redpanda Connect, and how to add them to your data pipelines without exposing them. -Secrets are stored in the secret management solution of your Cloud provider and are retrieved when you run a pipeline configuration that references them. - -== Prerequisites - -* A running Serverless, Dedicated, or BYOC (not BYOVPC/BYOVNet) cluster -* On BYOC clusters only, check that secrets management is enabled: - -.. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -.. Go to the **Connect** page. -.. Select the **Redpanda Connect** tab and make sure you can see a **Secrets** tab. - -+ -If you cannot see a **Secrets** tab, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda Support^] +Secrets are stored in the secret management solution of your cloud provider and are retrieved when you run a pipeline configuration that references them. == Manage secrets -You can manage secrets from the Cloud UI or Data Plane API. +You can manage secrets from the Cloud UI or the Data Plane API. === Create a secret @@ -27,32 +15,19 @@ You can create a secret and reference it in multiple data pipelines on the same [tabs] ===== -Cloud UI (BYOC and Dedicated):: +Cloud UI:: + -- . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to the **Connect** page. -. Select the **Redpanda Connect** tab and then the **Secrets** tab. -. Click **Create secret**. -. In **Secret name**, enter a name for the secret. You cannot rename the secret once it is created. -. In **Secret value**, enter the secret you need to add. +. Go to the **Secrets Store** page. . Click **Create secret**. -+ -The secret details are listed in the **Secrets** tab ready to <>. - --- +. For **ID**, enter a name for the secret. You cannot rename the secret once it is created. +. For **Value**, enter the secret you need to add. +. For **Scopes**, select Redpanda Connect. +. Optionally, add labels to help organize your secrets. +. Click **Create**. -Cloud UI (Serverless):: -+ --- -. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to the **Connect** page. -. Select the **Secrets** tab and click **Create secret**. -. In **Secret name**, enter a name for the secret. You cannot rename the secret once it is created. -. In **Secret value**, enter the secret you need to add. -. Click **Create secret**. -+ -The secret details are listed in the **Secrets** tab ready to <>. +You can now <>. -- @@ -63,7 +38,7 @@ You must use a Base64-encoded secret. . xref:manage:api/cloud-api-quickstart.adoc#try-the-cloud-api[Authenticate and get the base URL] for the Data Plane API. . Make a request to xref:api:ROOT:cloud-dataplane-api.adoc#post-/v1/secrets[`POST /v1/secrets`]. -+ + [,bash] ---- curl -X POST "https:///v1/secrets" \ @@ -72,7 +47,7 @@ curl -X POST "https:///v1/secrets" \ -H 'content-type: application/json' \ -d '{"id":"","scopes":["SCOPE_REDPANDA_CONNECT"],"secret_data":""}' ---- -+ + You must include the following values: - ``: The base URL for the Data Plane API. @@ -80,7 +55,6 @@ You must include the following values: - ``: The ID or name of the secret you want to add. Use only the following characters: `^[A-Z][A-Z0-9_]*$`. - ``: The Base64-encoded secret. - This scope: `"SCOPE_REDPANDA_CONNECT"`. - + The response returns the name of the secret and the scope `"SCOPE_REDPANDA_CONNECT"`. @@ -97,26 +71,13 @@ NOTE: Changes to secret values do not take effect until a pipeline is restarted. [tabs] ===== -Cloud UI (BYOC and Dedicated):: +Cloud UI:: + -- . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to the **Connect** page. -. Select the **Redpanda Connect** tab and then the **Secrets** tab. -. Find the secret you want to update and click the edit icon. -. Enter the new secret value and click **Update Secret**. -. Start and stop any pipelines that reference the secret. - --- - -Cloud UI (Serverless):: -+ --- -. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to the **Connect** page. -. Select the **Secrets** tab. -. Find the secret you want to update and click the edit icon. -. Enter the new secret value and click **Update Secret**. +. Go to the **Secrets Store** page. +. Find the secret you want to update, and click the edit icon. +. Enter the new secret value or labels, and click **Update**. . Start and stop any pipelines that reference the secret. -- @@ -161,24 +122,12 @@ NOTE: Changes do not affect pipelines that are already running. [tabs] ===== -Cloud UI (BYOC and Dedicated):: -+ --- -. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to the **Connect** page. -. Select the **Redpanda Connect** tab and then the **Secrets** tab. -. Find the secret you want to remove and click the delete icon. -. Confirm your deletion. - --- - -Cloud UI (Serverless):: +Cloud UI:: + -- . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to the **Connect** page. -. Select the **Secrets** tab. -. Find the secret you want to remove and click the delete icon. +. Go to the **Secrets Store** page. +. Find the secret you want to remove, and click the delete icon. . Confirm your deletion. -- @@ -207,13 +156,28 @@ You must include the following values: == Add a secret to a data pipeline -You can add a secret to any pipeline in your cluster using the notation `${secrets.SECRET_NAME}`. In the Cloud UI, you can copy the notation from the **Secrets** tab. +[tabs] +===== +Cloud UI:: ++ +-- +. Go to the **Connect** page, and create a pipeline (or open an existing pipeline to edit). +. Click the **Secret** button to add a new or existing secret to the pipeline. +-- +Data Plane API:: ++ +-- +You can add a secret to any pipeline in your cluster using the notation `${secrets.SECRET_NAME}`. For example: -```yml - sasl: - - mechanism: SCRAM-SHA-256 - username: "user" - password: "${secrets.PASSWORD}" -``` +[,yml] +---- +sasl: + - mechanism: SCRAM-SHA-256 + username: "user" + password: "${secrets.PASSWORD}" +---- +-- +===== +