Skip to content

Conversation

@fabianfett
Copy link
Collaborator

@fabianfett fabianfett commented Apr 9, 2025

This adds a structured response to the CLUSTER SHARDS command.

@fabianfett fabianfett changed the title [Draft] Add ValkeyClusterDescription [Do not merge | Draft] Add ValkeyClusterDescription Apr 9, 2025
Base automatically changed from ff-add-HashSlots to main April 9, 2025 16:06
@fabianfett fabianfett force-pushed the ff-add-ValkeyClusterDescription branch from 801c7e6 to 72d52b4 Compare April 22, 2025 18:39
@github-actions
Copy link

✅ Pull request no significant performance differences ✅

Summary

New baseline 'pull_request' is WITHIN the 'main' baseline thresholds.

Full Benchmark Comparison

Comparing results between 'main' and 'pull_request'

Host '1260e492e933' with 2 'x86_64' processors with 7 GB memory, running:
#25-Ubuntu SMP Wed Jan 15 20:45:09 UTC 2025

ValkeyBenchmarks

GET benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 4 4 4 4 12 24 24 11
pull_request 4 4 4 4 12 24 24 11
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

HashSlot – {user}.whatever metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 20
pull_request 0 0 0 0 0 0 0 21
Δ 0 0 0 0 0 0 0 1
Improvement % 0 0 0 0 0 0 0 1

RESPCommandEncoder – Command with 7 words metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 744
pull_request 0 0 0 0 0 0 0 711
Δ 0 0 0 0 0 0 0 -33
Improvement % 0 0 0 0 0 0 0 -33

RESPCommandEncoder – Simple GET metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 1923
pull_request 0 0 0 0 0 0 0 1930
Δ 0 0 0 0 0 0 0 7
Improvement % 0 0 0 0 0 0 0 7

RESPCommandEncoder – Simple MGET 15 keys metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 360
pull_request 0 0 0 0 0 0 0 361
Δ 0 0 0 0 0 0 0 1
Improvement % 0 0 0 0 0 0 0 1

@fabianfett fabianfett force-pushed the ff-add-ValkeyClusterDescription branch from 72d52b4 to ffbfc9e Compare April 24, 2025 13:41
@fabianfett fabianfett changed the title [Do not merge | Draft] Add ValkeyClusterDescription Add ValkeyClusterDescription Apr 24, 2025
Copy link
Collaborator

@adam-fowler adam-fowler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks ok. Cluster description is fairly complex. I would prefer if we try to use the constructs that are already in place.

package var token: RESPToken
}

package struct ValkeyClusterDescription: Hashable, Sendable {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we conform this to RESPTokenRepesentable instead of rolling our own RESPToken conversion. This would allow us to set CLUSTER.SHARDS.Response to ValkeyClusterDescription. Although that requires a change in the command renderer to ensure we aren't attempting to set the Response type for that function.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change to the command renderer is in, so you can override the response now

extension CLUSTER.SHARDS {
    public typealias Response = ValkeyClusterDescription
}

}
}

extension String {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use String(from: RESPToken) and extend it if we need the numbers as strings

}
}

extension Int64 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we conform this to RESPTokenRepresentable and move this conversion to RESPTokenRepresentable.swift

import NIOCore
import Valkey

enum RESP3Value: Hashable {
Copy link
Collaborator

@adam-fowler adam-fowler Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worthwhile just extending RESPToken.Value instead of creating another type

EDIT: Actually ignore that you wouldn't be able to create an array.

func testClusterDescription() throws {
let val = RESP3Value.array([
.array([
.simpleString(.init(string: "slots")),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valkey probably uses bulkString for these, simpleString is used for things like the ok returned from functions that have no return value.

enum RESP3Value: Hashable {
case simpleString(ByteBuffer)
case simpleError(ByteBuffer)
case bulkString(ByteBuffer)
Copy link
Collaborator

@adam-fowler adam-fowler Apr 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know bulkString etc can be used to represent non-UTF8 data, but do we need this for what is essentially a debug tool. It'll make using it a little easier to use if we just use String and will match the output of #50

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does that function fix your concerns?

   static func bulkString(_ value: String) -> RESP3Value {
        return .bulkString(ByteBuffer(string: value))
    }```

@fabianfett fabianfett force-pushed the ff-add-ValkeyClusterDescription branch from 9575927 to 92f0b41 Compare April 28, 2025 14:56
@fabianfett fabianfett requested a review from adam-fowler April 28, 2025 14:56
Copy link
Collaborator

@adam-fowler adam-fowler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just some minor comments, which can be ignored if you so feel

.push,
.set,
.null,
.map:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you want to include all possible cases explicitly here. But given that if a new type was added it would be require a new version of the protocol this seems overkill. I'm happy to let this through but don't think it should be a requirement for all the decode functions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. really not a fan of all the defaults everywhere. There was never a time, where default hasn't bit me

.bigNumber(let buffer),
.simpleError(let buffer),
.blobError(let buffer):
self = buffer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should errors be allowed to convert to a String or ByteBuffer? They are never presented to the user as a returned value, they have already been converted to a thrown error.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what you are talking about is semantics build on top of RESP, whereas a conversion in the RESP world is absolutely valid.

while let nodeKey = nodeIterator.next(), let key = try? String(from: nodeKey), let nodeVal = nodeIterator.next() {
switch key {
case "id":
id = try? String(from: nodeVal)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason you don't convert straight to an Int?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

id is a String?

@fabianfett fabianfett merged commit 8ebebcd into main Apr 28, 2025
@fabianfett fabianfett deleted the ff-add-ValkeyClusterDescription branch April 28, 2025 15:54
adam-fowler pushed a commit that referenced this pull request Jul 11, 2025
adam-fowler pushed a commit that referenced this pull request Jul 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants