Ux 457/sj/impersonation role sync script #1

sago2k8 · 2025-09-18T23:38:58Z

Implement Cli for reconciling redpanda roles based on controlplane roles

Copilot

Pull Request Overview

Implements a new CLI/tooling layer to migrate/sync Redpanda Cloud roles (and their principals/ACLs) to dataplane (Core) clusters, plus supporting pagination, interceptors, and typed ACL/role utilities.

Adds generic pagination utilities and comprehensive tests.
Introduces Cloud and Dataplane API clients, migration logic (IdentityMigrator), and CLI with ACL configuration.
Provides retry/auth interceptors, CI workflow, and documentation (README, env example, ACL TOML).

Reviewed Changes

Copilot reviewed 20 out of 27 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/utils/pagination.ts	Generic pagination helpers with accumulation strategies.
src/utils/pagination.test.ts	Unit tests covering pagination behaviors.
src/types/index.ts	Exposes core domain types and config interfaces.
src/types/acl-config.ts	Strongly typed ACL configuration schema.
src/sync-role.ts	Migration/sync orchestration of roles, principals, and ACL application.
src/sync-role.test.ts	Tests for migrator cluster selection and basic migration paths.
src/interceptors.ts	Auth and retry interceptors for ConnectRPC.
src/interceptors.test.ts	Unit tests for interceptor behavior and combinations.
src/clients/dataplane-client.ts	Dataplane client for roles, ACL creation, and membership sync.
src/clients/cloud-client.ts	Cloud client for clusters, roles, bindings, users, SAs, with pagination usage.
src/cli.ts	CLI command definitions, flag parsing, interactive prompts, ACL confirmation.
src/index.ts	Library entrypoint exports.
package.json	Project metadata, scripts, dependencies, build targets.
biome.json	Linting/formatting configuration.
acls.toml	ACL definition (currently only admin role).
README.md	Comprehensive usage and operations documentation.
.github/workflows/ci.yml	CI tasks for lint, test, build integrity.
.env.example	Environment variable template.
.bunfig.toml / .npmrc	Tooling and registry configuration.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-18T23:41:45Z

src/clients/cloud-client.ts

+  async getRoleBindings(): Promise<RoleBinding[]> {
+    try {
+      return paginateToArray(
+        { filter: { roleName: 'Admin' } },
+        (request) => this.roleBindingClient.listRoleBindings(request),
+        (response) => response.roleBindings,
+        { requestName: 'Role bindings' }
+      );


The method hard‑codes a filter for roleName 'Admin', so downstream logic (e.g., getRolesWithPrincipals expecting Admin, Writer, Reader) will never see bindings for Writer or Reader. Remove or parameterize this filter (or fetch unfiltered) so all target roles are discoverable.

Copilot · 2025-09-18T23:41:46Z

src/sync-role.ts

+      console.log('📋 Target roles for migration:');
+      const targetRoles = ['Admin', 'Writer', 'Reader'];
+      targetRoles.forEach((role) => {
+        console.log(`  - ${role}`);
+      });


[nitpick] The targetRoles array is only used for logging and not for filtering or validation; this can cause misleading output if upstream discovery fails to include some roles. Either enforce filtering based on this list or remove it to avoid implying those roles were considered in the actual migration loop.

Copilot · 2025-09-18T23:41:46Z

acls.toml

+[roles.admin]
+description = "Full administrative access to all cluster resources"


Only the admin role is defined here, but the codebase (e.g., migration and discovery logic) references Writer and Reader roles; absence will trigger warnings and skip ACL creation for those roles. Either add writer/reader sections or adjust docs and logic to reflect that only admin ACLs are currently supported.

Copilot · 2025-09-18T23:41:46Z

acls.toml

+resource_name = "*"
+resource_pattern_type = "LITERAL"
+operation = "ALL"
+permission_type = "ALLOW"


Only the admin role is defined here, but the codebase (e.g., migration and discovery logic) references Writer and Reader roles; absence will trigger warnings and skip ACL creation for those roles. Either add writer/reader sections or adjust docs and logic to reflect that only admin ACLs are currently supported.

Suggested change

permission_type = "ALLOW"

permission_type = "ALLOW"

host = "*"

[roles.writer]

description = "Write access to topics and groups"

[[roles.writer.acls]]

resource_type = "TOPIC"

resource_name = "*"

resource_pattern_type = "LITERAL"

operation = "WRITE"

permission_type = "ALLOW"

host = "*"

[[roles.writer.acls]]

resource_type = "GROUP"

resource_name = "*"

resource_pattern_type = "LITERAL"

operation = "WRITE"

permission_type = "ALLOW"

host = "*"

[roles.reader]

description = "Read access to topics and groups"

[[roles.reader.acls]]

resource_type = "TOPIC"

resource_name = "*"

resource_pattern_type = "LITERAL"

operation = "READ"

permission_type = "ALLOW"

host = "*"

[[roles.reader.acls]]

resource_type = "GROUP"

resource_name = "*"

resource_pattern_type = "LITERAL"

operation = "READ"

permission_type = "ALLOW"

Copilot · 2025-09-18T23:41:46Z

src/cli.ts

+  messages.push(`  👤 Principal: RedpandaRole:__redpanda_cloud_role_${roleType}`);
+  messages.push(`  📊 Total ACL entries: ${roleConfig.acls.length}`);


[nitpick] The principal displayed assumes the internal prefixed role name; ensure README and user guidance clarify this generated principal format, especially since the user supplies unprefixed role names (Admin/Writer/Reader). If mismatch is intentional, add a short clarification in the CLI help output.

Copilot · 2025-09-18T23:41:47Z

src/clients/dataplane-client.ts

+      console.log(`📤 Syncing membership for role: ${roleName}`);
+      console.log('   Target principals:', principals);


Duplicate logging of 'Target principals:' (lines 133 and 146) adds noise; remove one occurrence.

Copilot · 2025-09-18T23:41:47Z

src/clients/dataplane-client.ts

+      );
+      const normalizedTargetMembers = new Set(principals);
+
+      console.log('   Target principals:', principals);


Duplicate logging of 'Target principals:' (lines 133 and 146) adds noise; remove one occurrence.

Suggested change

console.log(' Target principals:', principals);

Copilot · 2025-09-18T23:41:48Z

package.json

+  "main": "src/index.ts",
+  "bin": {
+    "redpanda-migration": "./redpanda-migration"
+  },


The declared bin path ./redpanda-migration does not correspond to any generated artifact (build scripts output unified-identity-toolkit* in out/*) and main points to TypeScript sources without a build pipeline for consumers. Provide a compiled JS entry (e.g., dist/index.js), align bin to an existing file (or add a wrapper script), or adjust build scripts to produce the declared binary name at project root.

Copilot · 2025-09-18T23:41:48Z

package.json

+    "build": "npm run build:linux",
+    "build:js": "bun build --target node --sourcemap --bundle --outdir dist src/cli.ts",
+    "build:linux": "mkdir -p out/linux && bun build --compile --minify --sourcemap --target bun-linux-x64 src/cli.ts --outfile out/linux/unified-identity-toolkit",
+    "build:macos": "mkdir -p out/macos && bun build --compile --minify --sourcemap --target bun-darwin-x64 src/cli.ts --outfile out/macos/unified-identity-toolkit",
+    "build:windows": "mkdir -p out/windows && bun build --compile --minify --sourcemap --target bun-windows-x64 src/cli.ts --outfile out/windows/unified-identity-toolkit.exe",
+    "build:all": "npm run build:linux && npm run build:macos && npm run build:windows",


The declared bin path ./redpanda-migration does not correspond to any generated artifact (build scripts output unified-identity-toolkit* in out/*) and main points to TypeScript sources without a build pipeline for consumers. Provide a compiled JS entry (e.g., dist/index.js), align bin to an existing file (or add a wrapper script), or adjust build scripts to produce the declared binary name at project root.

Suggested change

"build": "npm run build:linux",

"build:js": "bun build --target node --sourcemap --bundle --outdir dist src/cli.ts",

"build:linux": "mkdir -p out/linux && bun build --compile --minify --sourcemap --target bun-linux-x64 src/cli.ts --outfile out/linux/unified-identity-toolkit",

"build:macos": "mkdir -p out/macos && bun build --compile --minify --sourcemap --target bun-darwin-x64 src/cli.ts --outfile out/macos/unified-identity-toolkit",

"build:windows": "mkdir -p out/windows && bun build --compile --minify --sourcemap --target bun-windows-x64 src/cli.ts --outfile out/windows/unified-identity-toolkit.exe",

"build:all": "npm run build:linux && npm run build:macos && npm run build:windows",

"build": "npm run build:linux && npm run build:bin",

"build:js": "bun build --target node --sourcemap --bundle --outdir dist src/cli.ts",

"build:linux": "mkdir -p out/linux && bun build --compile --minify --sourcemap --target bun-linux-x64 src/cli.ts --outfile out/linux/unified-identity-toolkit",

"build:macos": "mkdir -p out/macos && bun build --compile --minify --sourcemap --target bun-darwin-x64 src/cli.ts --outfile out/macos/unified-identity-toolkit",

"build:windows": "mkdir -p out/windows && bun build --compile --minify --sourcemap --target bun-windows-x64 src/cli.ts --outfile out/windows/unified-identity-toolkit.exe",

"build:all": "npm run build:linux && npm run build:macos && npm run build:windows",

"build:bin": "cp out/linux/unified-identity-toolkit ./redpanda-migration",

Copilot · 2025-09-18T23:41:48Z

README.md

+- **Linux**: `redpanda-toolkit-linux`
+- **macOS**: `redpanda-toolkit-macos`
+- **Windows**: `redpanda-toolkit-windows.exe`
+
+Make the binary executable (Linux/macOS):
+```bash
+chmod +x redpanda-toolkit-linux


Binary names here do not match build outputs (unified-identity-toolkit* in package.json scripts) or the declared npm bin (redpanda-migration). Update the README or rename build artifacts for consistency to prevent user confusion.

Suggested change

- **Linux**: `redpanda-toolkit-linux`

- **macOS**: `redpanda-toolkit-macos`

- **Windows**: `redpanda-toolkit-windows.exe`

Make the binary executable (Linux/macOS):

```bash

chmod +x redpanda-toolkit-linux

- **Linux**: `unified-identity-toolkit-linux`

- **macOS**: `unified-identity-toolkit-macos`

- **Windows**: `unified-identity-toolkit-windows.exe`

Make the binary executable (Linux/macOS):

```bash

chmod +x unified-identity-toolkit-linux

malinskibeniamin · 2025-09-19T09:41:46Z

.github/workflows/ci.yml

+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4


We can also use @v5. it was released 2 months ago

.github/workflows/ci.yml

malinskibeniamin · 2025-09-19T11:25:15Z

.npmrc

+# Enable package-lock generation
+package-lock=true


Do we need this? We use yarn.lock already it seems, I think this setting is for package-lock.json specifically.

malinskibeniamin · 2025-09-19T11:40:34Z

package.json

+    "check": "biome check .",
+    "check:fix": "biome check --write .",
+    "type-check": "tsc --noEmit",
+    "clean": "rm -rf dist out"


If we need to support different arch, we can consider using rimraf

malinskibeniamin · 2025-09-19T11:40:49Z

package.json

+    "node": ">=18.0.0"
+  },
+  "scripts": {
+    "build": "npm run build:linux",


Maybe use bun everywhere instead of npm?

malinskibeniamin · 2025-09-19T11:41:46Z

src/clients/cloud-client.ts

I think in subsequent PRs we could consider moving this to inkjs (React for CLIs)

Yeah I thought about it, but today, it's very simple.

malinskibeniamin · 2025-09-19T11:42:27Z

src/interceptors.test.ts

+ * Unit tests for ConnectRPC interceptors
+ */
+
+import { beforeEach, describe, expect, jest, test } from 'bun:test';


Really cool you could manage to get bun:test working, it improved a lot recently.

Mateoc · 2025-09-19T17:23:32Z

.env.example

@@ -0,0 +1,13 @@
+# Redpanda Cloud API Token (required)
+# Get this bearer token from your Redpanda Cloud organization settings
+REDPANDA_CLOUD_API_TOKEN=your-cloud-api-bearer-token-here


It would be useful if you could set up a service account credentials, since tokens expire

I tought about it bt wanted to keep it small and simple initially

Mateoc · 2025-09-19T17:24:42Z

dist/cli.js

are we versioning this?

yes, it will be semver with tags, for next PR

Mateoc · 2025-09-19T17:27:19Z

src/utils/pagination.ts

+/**
+ * Convenience function for paginating into an array
+ */
+export async function paginateToArray<


Weird name, ListAllPages maybe?

you can also paginate to Map hehe

Mateoc · 2025-09-19T17:28:34Z

src/utils/pagination.ts

+  let pageCount = 0;
+  let totalResults = 0;
+
+  do {


This got to be the frist time I see a do implemented in ts

andresaristizabal · 2025-09-19T17:54:41Z

package.json

+    "build:macos": "mkdir -p out/macos && bun build --compile --minify --sourcemap --target bun-darwin-x64 src/cli.ts --outfile out/macos/unified-identity-toolkit",
+    "build:windows": "mkdir -p out/windows && bun build --compile --minify --sourcemap --target bun-windows-x64 src/cli.ts --outfile out/windows/unified-identity-toolkit.exe",


do you need this?

Maybe windows is not important, but I want to have completeness, what's the concern?

andresaristizabal · 2025-09-19T18:45:46Z

src/utils/pagination.ts

+  let pageCount = 0;
+  let totalResults = 0;
+
+  do {


andresaristizabal · 2025-09-19T18:50:11Z

src/cli.ts

+            {
+              name: 'Admin - Full administrative access to cluster',
+              value: 'Admin',
+            },


don't you need the reader and writer options here?

Initially I want to focus only on the Admin Role, but eventually yes .

andresaristizabal

LGTM

Basic bun setup with ci and bun well known configurations Signed-off-by: santiago <[email protected]>

Install bun dependencies, make sure all necessary deps are installed Signed-off-by: santiago <[email protected]>

implement simple sync cli for redpada roles Signed-off-by: santiago <[email protected]>

Basic testing for the cli Signed-off-by: santiago <[email protected]>

Signed-off-by: santiago <[email protected]>

Implement JS bundle so the execution of the cli is very simple Signed-off-by: santiago <[email protected]>

Copilot AI review requested due to automatic review settings September 18, 2025 23:38

Copilot AI reviewed Sep 18, 2025

View reviewed changes

sago2k8 force-pushed the UX-457/sj/impersonation-role-sync-script branch 8 times, most recently from 24d1607 to 8552ca2 Compare September 19, 2025 09:39

malinskibeniamin reviewed Sep 19, 2025

View reviewed changes

sago2k8 force-pushed the UX-457/sj/impersonation-role-sync-script branch 2 times, most recently from f0b1267 to 59a5175 Compare September 19, 2025 09:50

sago2k8 requested a review from malinskibeniamin September 19, 2025 10:49

malinskibeniamin reviewed Sep 19, 2025

View reviewed changes

Mateoc reviewed Sep 19, 2025

View reviewed changes

andresaristizabal reviewed Sep 19, 2025

View reviewed changes

chore: intial set up for bun

115949b

Basic bun setup with ci and bun well known configurations Signed-off-by: santiago <[email protected]>

sago2k8 force-pushed the UX-457/sj/impersonation-role-sync-script branch from 59a5175 to 617d200 Compare September 22, 2025 09:24

malinskibeniamin approved these changes Sep 22, 2025

View reviewed changes

deps: install dependencies

5244eaf

Install bun dependencies, make sure all necessary deps are installed Signed-off-by: santiago <[email protected]>

sago2k8 force-pushed the UX-457/sj/impersonation-role-sync-script branch from 617d200 to e3ec89c Compare September 22, 2025 11:41

andresaristizabal approved these changes Sep 22, 2025

View reviewed changes

sago2k8 added 4 commits September 22, 2025 15:12

feat: implement sync cli for roles

9a01813

implement simple sync cli for redpada roles Signed-off-by: santiago <[email protected]>

test: implement basic testing for the cli

9063e64

Basic testing for the cli Signed-off-by: santiago <[email protected]>

docs: update README

17013ee

Signed-off-by: santiago <[email protected]>

gen: generate js bundle

bafeccd

Implement JS bundle so the execution of the cli is very simple Signed-off-by: santiago <[email protected]>

sago2k8 force-pushed the UX-457/sj/impersonation-role-sync-script branch from e3ec89c to bafeccd Compare September 22, 2025 13:13

sago2k8 merged commit 10cec64 into main Sep 22, 2025
3 checks passed

		[roles.admin]
		description = "Full administrative access to all cluster resources"

-permission_type = "ALLOW"
+permission_type = "ALLOW"
+host = "*"
+[roles.writer]
+description = "Write access to topics and groups"
+[[roles.writer.acls]]
+resource_type = "TOPIC"
+resource_name = "*"
+resource_pattern_type = "LITERAL"
+operation = "WRITE"
+permission_type = "ALLOW"
+host = "*"
+[[roles.writer.acls]]
+resource_type = "GROUP"
+resource_name = "*"
+resource_pattern_type = "LITERAL"
+operation = "WRITE"
+permission_type = "ALLOW"
+host = "*"
+[roles.reader]
+description = "Read access to topics and groups"
+[[roles.reader.acls]]
+resource_type = "TOPIC"
+resource_name = "*"
+resource_pattern_type = "LITERAL"
+operation = "READ"
+permission_type = "ALLOW"
+host = "*"
+[[roles.reader.acls]]
+resource_type = "GROUP"
+resource_name = "*"
+resource_pattern_type = "LITERAL"
+operation = "READ"
+permission_type = "ALLOW"

		messages.push(` 👤 Principal: RedpandaRole:__redpanda_cloud_role_${roleType}`);
		messages.push(` 📊 Total ACL entries: ${roleConfig.acls.length}`);

		console.log(`📤 Syncing membership for role: ${roleName}`);
		console.log(' Target principals:', principals);

		# Enable package-lock generation
		package-lock=true No newline at end of file

		"build:macos": "mkdir -p out/macos && bun build --compile --minify --sourcemap --target bun-darwin-x64 src/cli.ts --outfile out/macos/unified-identity-toolkit",
		"build:windows": "mkdir -p out/windows && bun build --compile --minify --sourcemap --target bun-windows-x64 src/cli.ts --outfile out/windows/unified-identity-toolkit.exe",

Ux 457/sj/impersonation role sync script #1

Ux 457/sj/impersonation role sync script #1

Uh oh!

Conversation

sago2k8 commented Sep 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!