Multi-Region Without Sacrificing Availability

[ryansun96]

I'm trying to deploy NATS into three regions, Americas, Europe, and Asia, (Region A, B, C), each region having different numbers of nodes. From the doc if three clusters are connected via gateways, the availability of any region depends on that of (or connectivity to) other regions, because stream and consumer creation relies on a global quorum (which also introduces significant latency). This dependency is more pronounced when one region has significantly more server than the sum of the other two, such that the loss of one region will render two other regions partially unavailable.

Is there an architecture that keeps each region independent from each other, but still offers the convenience of "one virtual cluster"? For example:

• Ease of mirroring stream from one region into another
• Ease or consuming from / publish to streams from different regions

The closest to that I can think of is to deploy leaf nodes of Asia cluster in Americas, aside from the "main" Americas cluster. Does this offer any benefits over clients connecting to other regions directly?

[jnmoyne]

In the case of imbalance in the cluster sizes or when connectivity between regions can be bad, you are better off using one cluster (using your largest one) and the others connecting as Leaf Nodes, each one with their own JS domain name.

Sourcing and consuming from streams in other regions remains the same but (for JS operations) you have to specify the JS API domain name (by default the domain name of the specific server the client is connected to is used). Connecting through a leaf node is functionally transparent from the client application's point of view (according to the permissions the LN is using to connect to the hub): Core NATS messages (and requests) can be published to or subscribed to from anywhere, so as long as you can publish to the subject that a stream in another region is capturing on, then the messages will be stored in the stream.

I recommend this video to learn about running nats-server as a leaf node: https://youtu.be/WH55czo1BNk

[ryansun96]

Hi @jnmoyne Thanks for your reply! Does making an entire region a leaf node cluster impose any child-parent / edge-central related limitations? Semantically, each region should be independent and not a subordinate of another. I'm ok if the subordination relationship exists only in names, but would like to know if that's the case. Specifically, is there a way to "convert" a leaf cluster into a non-leaf cluster in the future, if necessary? What happens if the now-largest cluster is overtaken in size by a leaf cluster in the future?

In other words, does the leaf node architecture have any down side compared to, say, a super cluster? This is the information I find missing from the docs / videos, i.e. the cons of each architecture.

[ryansun96]

Similarly, if we already have a regular cluster, is there a way to convert it into a leaf cluster without losing data?