C# redis叢集批量操作之slot計算出16384個字串

引入一個大家都用的到的需求來說吧。

需求：要在三主三從的redis叢集，存入資料，會對資料進行批量刪除操作，資料要求要在redis叢集負載均衡。

思路：

1.存入資料好辦

var connect = ConnectionMultiplexer.Connect(redisConn);
var redisDb = connect.GetDatabase();
var res1 = redisDb.StringSet("1", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
var res2 = redisDb.StringSet("1111", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));

2.批量刪除直接異常

redisDb.KeyDelete(new RedisKey[] { "1", "1111" });

Ex:"Multi-key operations must involve a single slot; keys can use 'hash tags' to help this, i.e. '{/users/12345}/account' and '{/users/12345}/contacts' will always be in the same slot"

 

3.查到異常是因為redis hash slot 機制導致的，什麼是 hash slot？
hash slot 介紹：https://redis.io/topics/cluster-tutorial

4.加上hash slot 字串，讓key進入同一個slot
var res1 = redisDb.StringSet("{myslot}key1", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
var res2 = redisDb.StringSet("{myslot}key2", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
redisDb.KeyDelete(new RedisKey[] { "{myslot}key1", "{myslot}key2" });
能進行批量操作，但是都被分配到了同一臺伺服器上的同一個槽點，不負載均衡。

5.如何負載均衡， 讓Key分佈到各個伺服器，並且可以批量操作？
如果知道每個槽點對應的字串，key可以按照演算法計算出自己對應的字串，加上後，就可以進行分組批量增刪改操作。

6.hash slot 計算方法
HASH_SLOT = CRC16(key) mod 16384 （crc16-XMODEM）
介紹 ：https://redis.io/topics/cluster-spec

7.net core 計算出16384個slot 字串 演算法例子和 結果模板

        static void Main(string[] args)
        {
            var data = new Dictionary<int, string>();
            var i = 0;
            while (data.Keys.Count < 16384)
            {
                var temp = i.ToString("X");
                var value = Crc16(Encoding.UTF8.GetBytes(temp)) % 16384;
                data[int.Parse(value.ToString())] = temp;
                i++;

            }
            var sb = new StringBuilder();
            foreach (var item in data.OrderBy(s => s.Key))
            {
                var temp = $"slot num:{item.Key}   str:{item.Value} \r\n";
                Console.WriteLine(temp);
                sb.Append(temp);
            }
            File.WriteAllText("data.txt", sb.ToString());
            Console.ReadLine();
        }
        private static ushort Crc16(byte[] bytes)
        {
            ushort poly = 0x1021;
            ushort[] table = new ushort[256];
            ushort initialValue = 0x0;
            ushort temp, a;
            ushort crc = initialValue;
            for (int i = 0; i < table.Length; ++i)
            {
                temp = 0;
                a = (ushort)(i << 8);
                for (int j = 0; j < 8; ++j)
                {
                    if (((temp ^ a) & 0x8000) != 0)
                        temp = (ushort)((temp << 1) ^ poly);
                    else
                        temp <<= 1;
                    a <<= 1;
                }
                table[i] = temp;
            }
            for (int i = 0; i < bytes.Length; ++i)
            {
                crc = (ushort)((crc << 8) ^ table[((crc >> 8) ^ (0xff & bytes[i]))]);
            }
            return crc;
        }

　　data.txt 檔案下載

8.校驗算出來的字串 對應 的slot位置 是否正確

 

9.批量設定和批量刪除方法
假定三主三從，那麼三臺伺服器，取九個slot字串，這九個是均分的位置(均分利於叢集擴充套件)。即16384/10=1638 1638是第一位，1638*2是第二位，以此類推取字串
共九個["1A73F","18B13","1AAD3","184FF","143BF","18413","17B8D","18BFF","1B1C4"]
先分組，再批量插入，再批量刪除

try
            {
                var redisConn = "{叢集地址}";
                var connect = ConnectionMultiplexer.Connect(redisConn);
                var redisDb = connect.GetDatabase();
                var res1 = redisDb.StringSet("1", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
                var res2 = redisDb.StringSet("1111", DateTime.Now.ToString(), TimeSpan.FromSeconds(600));
                redisDb.KeyDelete(new RedisKey[] { "1", "1111" });


                var redisSlotKeyList = new string[] { "1A73F", "18B13", "1AAD3", "184FF", "143BF", "18413", "17B8D", "18BFF", "1B1C4" };
                var userIdArray = new int[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 };
                //group
                var dic = new Dictionary<int, Dictionary<RedisKey,RedisValue>>();
                foreach (var userId in userIdArray)
                {
                    var index = userId % redisSlotKeyList.Length;
                    var slotKey = redisSlotKeyList[index];
                    var redisKey = $"{{{slotKey}}}test_{userId}";
                    Console.WriteLine($"{ redisKey}  {userId}");
                    if (dic.ContainsKey(index))
                    {
                        dic[index].Add(redisKey, DateTime.Now.ToLongTimeString());
                    }
                    else 
                    {
                        dic[index] = new Dictionary<RedisKey, RedisValue> { { new RedisKey(redisKey), new RedisValue("values") } };
                    }
                }

                //set
                foreach (var item in dic)
                {
                    var addRes = redisDb.StringSet(item.Value.ToArray());
                    Console.WriteLine(addRes);
                }


                //delete
                foreach (var item in dic)
                {
                    var deleteRes = redisDb.KeyDelete(item.Value.Keys.ToArray());
                    Console.WriteLine(deleteRes);
                }



            }
            catch (Exception ex)
            {
                throw ex;
            }

 

10.注意點
叢集的分割slot配置不一定的均分的，提前先檢視，命令：cluster nodes。 檢視之後再根據實際情況取slot string