Failed to create initial connections

[jfk9w]

Hi!

I'm trying to use github.com/valkey-io/valkey-glide/go/v2 to connect to a Valkey Cluster with TLS.
I'm using the following snippet to initiate connection:

func TestBare(t *testing.T) {
	conf := config.NewClusterClientConfiguration().
		WithAddress(&config.NodeAddress{Host: "host", Port: 6379}).
		WithUseTLS(true).
		WithCredentials(config.NewServerCredentials("username", "password")).
		WithAdvancedConfiguration(config.NewAdvancedClusterClientConfiguration().
			WithConnectionTimeout(time.Minute)).
		WithRequestTimeout(time.Minute)

	client, err := glide.NewClusterClient(conf)
	require.NoError(t, err)

	println("connected")

	defer client.Close()

	ctx := context.Background()
	result, err := client.Get(ctx, "x")
	require.NoError(t, err)

	println("Result:", result.Value())
}

I'm getting a connection attempt timed out error on client creation.
What's weird is that the code fails exactly after 10 seconds despite both request and connection timeouts set to one minute.

Using redis-cli is fine:

❯ redis-cli -h host -c --tls --cacert $SSL_CERT_PATH --user username -a password
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
host:6379> GET x
-> Redirected to slot [16287] located at host2:6379
"2"
(1.16s)

At the same time using glide.NewClient instead of glide.NewClusterClient is OK. In this case when I try to do result, err := client.Get(ctx, "x"), I get a An error was signalled by the server: - Moved: 16287. This is to be expected an corresponds to redis-cli:

❯ redis-cli -h host --tls --cacert $SSL_CERT_PATH --user username -a password
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
host:6379> GET x
(error) MOVED 16287 host2:6379
host:6379>

So how do I go about creating cluster client from Go?

[jfk9w]

Ok, I think I found the culprit.

When connecting from Go, I use WithAddress(&config.NodeAddress{Host: "host", Port: 6379}) while cluster nodes returns the following:

host> cluster nodes
94f11767837cd57a068380a26f1eb72acbd2befd 10.0.0.1:6379@16379,host.k8s master - 0 1758095823616 3 connected 5460-8189
e61554e4e2e0b12dc07b35aacb38bb1ab9595f43 10.0.0.2:6379@16379,host2.k8s slave 2e46fbd8e9c5c1e089b50ecc6a4c85986f3b383a 0 1758095823617 25 connected

So hostname in connection parameters does not match the one returned by cluster nodes.
If I use the matching hostname it seems to connect fine.
I guess I can simply connect with glide.NewClient to a node, execute cluster nodes to discover hostnames used by the cluster and then use glide.NewClusterClient, but it would be nice if the cluster client would perform this automatically (or at least would not fail).

[jamesx-improving]

Hi @jfk9w , thank you for reaching out!

Not an expert of Kubernetes myself, but may I ask: what is the nature of the host names "host" and "host2" you mentioned here? Is it supposed to be resolve by a DNS server, presumably an internal one in the k8s network/system? Or is it any kind of non-DNS name like NetBIOS or anything else?

Also, anywhere in the system did you configured the "host.k8s" and "host2.k8s" name yourself, or they just appear in the response of the CLUSTER NODES command?

Valkey-GLIDE do have some TBD improvement in the scenarios when host name used for client creation doesn't align with what is returned from CLUSTER NODES command, like when nodes only have internal addresses/names, and some type of proxy is needed for connection. We need to learn more about all these scenarios to dish out a comprehensive solution in upcoming releases. It'll be great if you could share a bit more on your configuration, including Valkey cluster creation method, cluster-announce-hostname used, if any, and host naming mechanism.

Much appreciated!