Creating and using a Redis Cluster

All: https://redis.io/topics/cluster-tutorial

Install Ruby 2.4: https://wp.huangshiyang.com/how-to-install-ruby-2-4-on-centos-rhel-7-6

Installing Redis

The suggested way of installing Redis is compiling it from sources as Redis has no dependencies other than a working GCC compiler and libc. Installing it using the package manager of your Linux distribution is somewhat discouraged as usually the available version is not the latest.

You can either download the latest Redis tar ball from the redis.io web site, or you can alternatively use this special URL that always points to the latest stable Redis version, that is, http://download.redis.io/redis-stable.tar.gz.

In order to compile Redis follow this simple steps:

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make

Note: to deploy a Redis Cluster manually it is very important to learn certain operational aspects of it. However if you want to get a cluster up and running ASAP (As Soon As Possible) skip this section and the next one and go directly to Creating a Redis Cluster using the create-cluster script.

To create a cluster, the first thing we need is to have a few empty Redis instances running in cluster mode. This basically means that clusters are not created using normal Redis instances as a special mode needs to be configured so that the Redis instance will enable the Cluster specific features and commands.

The following is a minimal Redis cluster configuration file:

port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

As you can see what enables the cluster mode is simply the cluster-enabled directive. Every instance also contains the path of a file where the configuration for this node is stored, which by default is nodes.conf. This file is never touched by humans; it is simply generated at startup by the Redis Cluster instances, and updated every time it is needed.

Note that the minimal cluster that works as expected requires to contain at least three master nodes. For your first tests it is strongly suggested to start a six nodes cluster with three masters and three slaves.

To do so, enter a new directory, and create the following directories named after the port number of the instance we’ll run inside any given directory.

Something like:

mkdir cluster-test
cd cluster-test
mkdir 7000 7001 7002 7003 7004 7005

Create a redis.conf file inside each of the directories, from 7000 to 7005. As a template for your configuration file just use the small example above, but make sure to replace the port number 7000 with the right port number according to the directory name.

Now copy your redis-server executable, compiled from the latest sources in the unstable branch at GitHub, into the cluster-test directory, and finally open 6 terminal tabs in your favorite terminal application.

Start every instance like that, one every tab:

cd 7000
../redis-server ./redis.conf

As you can see from the logs of every instance, since no nodes.conf file existed, every node assigns itself a new ID.

[82462] 26 Nov 11:56:55.329 * No cluster configuration found, I'm 97a3a64667477371c4479320d683e4c8db5858b1

This ID will be used forever by this specific instance in order for the instance to have a unique name in the context of the cluster. Every node remembers every other node using this IDs, and not by IP or port. IP addresses and ports may change, but the unique node identifier will never change for all the life of the node. We call this identifier simply Node ID.

Creating the cluster

Now that we have a number of instances running, we need to create our cluster by writing some meaningful configuration to the nodes.

This is very easy to accomplish as we are helped by the Redis Cluster command line utility called redis-trib, a Ruby program executing special commands on instances in order to create new clusters, check or reshard an existing cluster, and so forth.

The redis-trib utility is in the src directory of the Redis source code distribution. You need to install redis gem to be able to run redis-trib.

gem install redis

To create your cluster simply type:

redis-trib.rb: http://download.redis.io/redis-stable/src/redis-trib.rb

./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

The command used here is create, since we want to create a new cluster. The option --replicas 1 means that we want a slave for every master created. The other arguments are the list of addresses of the instances I want to use to create the new cluster.

Obviously the only setup with our requirements is to create a cluster with 3 masters and 3 slaves.

Redis-trib will propose you a configuration. Accept the proposed configuration by typing yes. The cluster will be configured and joined, which means, instances will be bootstrapped into talking with each other. Finally, if everything went well, you’ll see a message like that:

[OK] All 16384 slots covered

This means that there is at least a master instance serving each of the 16384 slots available.

Manually create cluster:

Assume you have 3 master nodes:

for i in {0..5459}; do redis-cli -p 7000 CLUSTER ADDSLOTS $i > /dev/null; done
for i in {5460..10919}; do redis-cli -p 7001 CLUSTER ADDSLOTS $i > /dev/null; done
for i in {10920..16383}; do redis-cli -p 7002 CLUSTER ADDSLOTS $i > /dev/null; done

redis-cli -p 7000 CLUSTER MEET NODE_IP 7001
redis-cli -p 7000 CLUSTER MEET NODE_IP 7002

If you need slave nodes:
redis-cli -p 7000 CLUSTER MEET NODE_IP 7003
redis-cli -p 7000 CLUSTER MEET NODE_IP 7004
redis-cli -p 7000 CLUSTER MEET NODE_IP 7005

Get node name with:
redis-cli -c -p 7000 cluster nodes

c8f41a1c5639442b7fa511a047f0516ca1e8508a 10.105.118.11:7002@17002 master - 0 1568332762315 0 connected 10920-16383
b862f5b8b0436e1ca6f9d8a2bc795e0a483f7016 10.105.118.11:7001@17001 master - 0 1568332761312 2 connected 5460-10919
036ef862dd089343b7c98174e359de38392f8204 10.105.118.11:7000@17000 myself,master - 0 1568332761000 1 connected 0-5459

And do a replicate with:

redis-cli -h NODE_IP -p 7003 cluster replicate NODE_1_NAME
redis-cli -h NODE_IP -p 7004 cluster replicate NODE_2_NAME
redis-cli -h NODE_IP -p 7005 cluster replicate NODE_3_NAME

LEAVE A COMMENT