Redis 物件系統
Redis原始碼剖析和註釋(八)--- 物件系統(redisObject)
Redis 物件系統
1. 介紹
redis中基於雙端連結串列、簡單動態字串(sds)、字典、跳躍表、整數集合、壓縮列表、快速列表等等資料結構實現了一個物件系統,並且實現了5種不同的物件,每種物件都使用了至少一種前面的資料結構,優化物件在不同場合下的使用效率。
2. 物件的系統的實現
redis 3.2版本。所有註釋在github中:物件系統的註釋
2.1 物件的結構
物件結構robj功能:
- 為5種不同的物件型別提供同一的表示形式。
- 為不同的物件適用於不同的場景,支援同一種物件型別採用多種的資料結構方式。
- 支援引用計數,實現物件共享機制。
- 記錄物件的訪問時間,便於刪除物件。
物件結構定義在redis 3.2版本的server.h
#define LRU_BITS 24
#define LRU_CLOCK_MAX ((1<<LRU_BITS)-1) /* Max value of obj->lru */
#define LRU_CLOCK_RESOLUTION 1000 /* LRU clock resolution in ms */
typedef struct redisObject {
//物件的資料型別,佔4bits,共5種型別
unsigned type:4;
//物件的編碼型別,佔4bits,共10種型別
unsigned encoding:4;
//least recently used
//實用LRU演算法計算相對server.lruclock的LRU時間
unsigned lru:LRU_BITS; /* lru time (relative to server.lruclock) */
//引用計數
int refcount;
//指向底層資料實現的指標
void *ptr;
} robj;
//type的佔5種型別:
/* Object types */
#define OBJ_STRING 0 //字串物件
#define OBJ_LIST 1 //列表物件
#define OBJ_SET 2 //集合物件
#define OBJ_ZSET 3 //有序集合物件
#define OBJ_HASH 4 //雜湊物件
/* Objects encoding. Some kind of objects like Strings and Hashes can be
* internally represented in multiple ways. The 'encoding' field of the object
* is set to one of this fields for this object. */
// encoding 的10種型別
#define OBJ_ENCODING_RAW 0 /* Raw representation */ //原始表示方式,字串物件是簡單動態字串
#define OBJ_ENCODING_INT 1 /* Encoded as integer */ //long型別的整數
#define OBJ_ENCODING_HT 2 /* Encoded as hash table */ //字典
#define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */ //不在使用
#define OBJ_ENCODING_LINKEDLIST 4 /* Encoded as regular linked list */ //雙端連結串列,不在使用
#define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */ //壓縮列表
#define OBJ_ENCODING_INTSET 6 /* Encoded as intset */ //整數集合
#define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */ //跳躍表和字典
#define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */ //embstr編碼的簡單動態字串
#define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */ //由壓縮列表組成的雙向列表-->快速列表
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
2.2 字串物件的底層實現型別
編碼—encoding | 物件—ptr |
---|---|
OBJ_ENCODING_RAW | 簡單動態字串實現的字串物件 |
OBJ_ENCODING_INT | 整數值實現的字串物件 |
OBJ_ENCODING_EMBSTR | embstr編碼的簡單動態字串實現的字串物件 |
2.3 列表物件的底層實現型別
編碼—encoding | 物件—ptr |
---|---|
OBJ_ENCODING_QUICKLIST | 快速列表實現的列表物件 |
OBJ_ENCODING_ZIPLIST | 壓縮列表實現的列表物件 |
2.4 集合物件的底層實現型別
編碼—encoding | 物件—ptr |
---|---|
OBJ_ENCODING_HT | 字典實現的集合物件 |
OBJ_ENCODING_INTSET | 整數集合實現的集合物件 |
2.5 雜湊物件的底層實現型別
編碼—encoding | 物件—ptr |
---|---|
OBJ_ENCODING_ZIPLIST | 壓縮列表實現的雜湊物件 |
OBJ_ENCODING_HT | 字典實現的雜湊物件 |
2.6 有序集合物件的底層實現型別
編碼—encoding | 物件—ptr |
---|---|
OBJ_ENCODING_SKIPLIST | 跳躍表和字典實現的有序集合物件 |
OBJ_ENCODING_ZIPLIST | 壓縮列表實現的有序集合物件 |
3. 物件系統的重要操作
3.1建立一個字串物件
- 編碼為OBJ_ENCODING_RAW
robj *createObject(int type, void *ptr) { //建立一個物件
robj *o = zmalloc(sizeof(*o)); //分配空間
o->type = type; //設定物件型別
o->encoding = OBJ_ENCODING_RAW; //設定編碼方式為OBJ_ENCODING_RAW
o->ptr = ptr; //設定
o->refcount = 1; //引用計數為1
/* Set the LRU to the current lruclock (minutes resolution). */
o->lru = LRU_CLOCK(); //計算設定當前LRU時間
return o;
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 編碼為OBJ_ENCODING_EMBSTR
/* Create a string object with encoding OBJ_ENCODING_EMBSTR, that is
* an object where the sds string is actually an unmodifiable string
* allocated in the same chunk as the object itself. */
//建立一個embstr編碼的字串物件
robj *createEmbeddedStringObject(const char *ptr, size_t len) {
robj *o = zmalloc(sizeof(robj)+sizeof(struct sdshdr8)+len+1); //分配空間
struct sdshdr8 *sh = (void*)(o+1); //o+1剛好就是struct sdshdr8的地址
o->type = OBJ_STRING; //型別為字串物件
o->encoding = OBJ_ENCODING_EMBSTR; //設定編碼型別OBJ_ENCODING_EMBSTR
o->ptr = sh+1; //指向分配的sds物件,分配的len+1的空間首地址
o->refcount = 1; //設定引用計數
o->lru = LRU_CLOCK(); //計算設定當前LRU時間
sh->len = len; //設定字串長度
sh->alloc = len; //設定最大容量
sh->flags = SDS_TYPE_8; //設定sds的型別
if (ptr) { //如果傳了字串引數
memcpy(sh->buf,ptr,len); //將傳進來的ptr儲存到物件中
sh->buf[len] = '\0'; //結束符標誌
} else {
memset(sh->buf,0,len+1); //否則將物件的空間初始化為0
}
return o;
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 兩種字串物件編碼方式的區別
/* Create a string object with EMBSTR encoding if it is smaller than
* REIDS_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
* used.
*
* The current limit of 39 is chosen so that the biggest string object
* we allocate as EMBSTR will still fit into the 64 byte arena of jemalloc. */
//sdshdr8的大小為3個位元組,加上1個結束符共4個位元組
//redisObject的大小為16個位元組
//redis使用jemalloc記憶體分配器,且jemalloc會分配8,16,32,64等位元組的記憶體
//一個embstr固定的大小為16+3+1 = 20個位元組,因此一個最大的embstr字串為64-20 = 44位元組
#define OBJ_ENCODING_EMBSTR_SIZE_LIMIT 44
// 建立字串物件,根據長度使用不同的編碼型別
// createRawStringObject和createEmbeddedStringObject的區別是:
// createRawStringObject是當字串長度大於44位元組時,robj結構和sdshdr結構在記憶體上是分開的
// createEmbeddedStringObject是當字串長度小於等於44位元組時,robj結構和sdshdr結構在記憶體上是連續的
robj *createStringObject(const char *ptr, size_t len) {
if (len <= OBJ_ENCODING_EMBSTR_SIZE_LIMIT)
return createEmbeddedStringObject(ptr,len);
else
return createRawStringObject(ptr,len);
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
3.2 字串物件編碼的優化
/* Try to encode a string object in order to save space */
//嘗試優化字串物件的編碼方式以節約空間
robj *tryObjectEncoding(robj *o) {
long value;
sds s = o->ptr;
size_t len;
/* Make sure this is a string object, the only type we encode
* in this function. Other types use encoded memory efficient
* representations but are handled by the commands implementing
* the type. */
serverAssertWithInfo(NULL,o,o->type == OBJ_STRING);
/* We try some specialized encoding only for objects that are
* RAW or EMBSTR encoded, in other words objects that are still
* in represented by an actually array of chars. */
//如果字串物件的編碼型別為RAW或EMBSTR時,才對其重新編碼
if (!sdsEncodedObject(o)) return o;
/* It's not safe to encode shared objects: shared objects can be shared
* everywhere in the "object space" of Redis and may end in places where
* they are not handled. We handle them only as values in the keyspace. */
//如果refcount大於1,則說明物件的ptr指向的值是共享的,不對共享物件進行編碼
if (o->refcount > 1) return o;
/* Check if we can represent this string as a long integer.
* Note that we are sure that a string larger than 20 chars is not
* representable as a 32 nor 64 bit integer. */
len = sdslen(s); //獲得字串s的長度
//如果len小於等於20,表示符合long long可以表示的範圍,且可以轉換為long型別的字串進行編碼
if (len <= 20 && string2l(s,len,&value)) {
/* This object is encodable as a long. Try to use a shared object.
* Note that we avoid using shared integers when maxmemory is used
* because every object needs to have a private LRU field for the LRU
* algorithm to work well. */
if ((server.maxmemory == 0 ||
(server.maxmemory_policy != MAXMEMORY_VOLATILE_LRU &&
server.maxmemory_policy != MAXMEMORY_ALLKEYS_LRU)) &&
value >= 0 &&
value < OBJ_SHARED_INTEGERS) //如果value處於共享整數的範圍內
{
decrRefCount(o); //原物件的引用計數減1,釋放物件
incrRefCount(shared.integers[value]); //增加共享物件的引用計數
return shared.integers[value]; //返回一個編碼為整數的字串物件
} else { //如果不處於共享整數的範圍
if (o->encoding == OBJ_ENCODING_RAW) sdsfree(o->ptr); //釋放編碼為OBJ_ENCODING_RAW的物件
o->encoding = OBJ_ENCODING_INT; //轉換為OBJ_ENCODING_INT編碼
o->ptr = (void*) value; //指標ptr指向value物件
return o;
}
}
/* If the string is small and is still RAW encoded,
* try the EMBSTR encoding which is more efficient.
* In this representation the object and the SDS string are allocated
* in the same chunk of memory to save space and cache misses. */
//如果len小於44,44是最大的編碼為EMBSTR型別的字串物件長度
if (len <= OBJ_ENCODING_EMBSTR_SIZE_LIMIT) {
robj *emb;
if (o->encoding == OBJ_ENCODING_EMBSTR) return o; //將RAW物件轉換為OBJ_ENCODING_EMBSTR編碼型別
emb = createEmbeddedStringObject(s,sdslen(s)); //建立一個編碼型別為OBJ_ENCODING_EMBSTR的字串物件
decrRefCount(o); //釋放之前的物件
return emb;
}
/* We can't encode the object...
*
* Do the last try, and at least optimize the SDS string inside
* the string object to require little space, in case there
* is more than 10% of free space at the end of the SDS string.
*
* We do that only for relatively large strings as this branch
* is only entered if the length of the string is greater than
* OBJ_ENCODING_EMBSTR_SIZE_LIMIT. */
//無法進行編碼,但是如果s的未使用的空間大於使用空間的10分之1
if (o->encoding == OBJ_ENCODING_RAW &&
sdsavail(s) > len/10)
{
o->ptr = sdsRemoveFreeSpace(o->ptr); //釋放所有的未使用空間
}
/* Return the original object. */
return o;
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
3.3 引用計數管理物件
//引用計數加1
void incrRefCount(robj *o) {
o->refcount++;
}
//引用計數減1
void decrRefCount(robj *o) {
if (o->refcount <= 0) serverPanic("decrRefCount against refcount <= 0");
//當引用物件等於1時,在操作引用計數減1,直接釋放物件的ptr和物件空間
if (o->refcount == 1) {
switch(o->type) {
case OBJ_STRING: freeStringObject(o); break;
case OBJ_LIST: freeListObject(o); break;
case OBJ_SET: freeSetObject(o); break;
case OBJ_ZSET: freeZsetObject(o); break;
case OBJ_HASH: freeHashObject(o); break;
default: serverPanic("Unknown object type"); break;
}
zfree(o);
} else {
o->refcount--; //否則減1
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
3.4 物件的複製,建立的物件非共享
//返回 複製的o物件的副本的地址,且建立的物件非共享
robj *dupStringObject(robj *o) {
robj *d;
serverAssert(o->type == OBJ_STRING); //一定是OBJ_STRING型別
switch(o->encoding) { //根據不同的編碼型別
case OBJ_ENCODING_RAW:
return createRawStringObject(o->ptr,sdslen(o->ptr)); //建立的物件非共享
case OBJ_ENCODING_EMBSTR:
return createEmbeddedStringObject(o->ptr,sdslen(o->ptr)); //建立的物件非共享
case OBJ_ENCODING_INT: //整數編碼型別
d = createObject(OBJ_STRING, NULL); //即使是共享整數範圍內的整數,建立的物件也是非共享的
d->encoding = OBJ_ENCODING_INT;
d->ptr = o->ptr;
return d;
default:
serverPanic("Wrong encoding.");
break;
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
3.5 物件的解碼操作
將儲存的整數值解碼成字串物件返回回來。
/* Get a decoded version of an encoded object (returned as a new object).
* If the object is already raw-encoded just increment the ref count. */
//將物件是整型的解碼為字串並返回,如果是字串編碼則直接返回輸入物件,只需增加引用計數
robj *getDecodedObject(robj *o) {
robj *dec;
if (sdsEncodedObject(o)) { //如果是OBJ_ENCODING_RAW或OBJ_ENCODING_EMBSTR型別的物件
incrRefCount(o); //增加引用計數,返回一個共享的物件
return o;
}
if (o->type == OBJ_STRING && o->encoding == OBJ_ENCODING_INT) { //如果是整數物件
char buf[32];
ll2string(buf,32,(long)o->ptr); //將整數轉換為字串
dec = createStringObject(buf,strlen(buf)); //建立一個字串物件
return dec;
} else {
serverPanic("Unknown encoding type");
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
3.6 其他操作
所有註釋在github中:物件系統的註釋
相關文章
- Redis物件系統Redis物件
- Common Lisp物件系統是現存最好的物件系統? - mendhekarLisp物件
- Redis 資料結構和物件系統,記住這 12 張圖就夠啦!Redis資料結構物件
- 物件界作業系統物件作業系統
- Redis有序集合物件Redis物件
- Redis - 物件結構Redis物件
- 將redis作為windows系統的系統服務RedisWindows
- 若依管理系統新增redisRedis
- Redis OM .NET Redis物件對映框架Redis物件框架
- juicefs:一個基於Redis和雲物件儲存的分散式POSIX檔案系統UIRedis物件分散式
- 物件導向系統設計物件
- Redis物件——有序集合(ZSet)Redis物件
- Redis的String物件Redis物件
- Linux系統安裝redis教程。LinuxRedis
- 使用Redis記錄系統日誌Redis
- Redis-資料結構與物件-物件Redis資料結構物件
- Redis效能篇(三)Redis關鍵系統配置:如何應對Redis變慢Redis
- FreeBSD系統核心物件(轉)物件
- Redis的字串物件筆記Redis字串物件筆記
- Linux系統安裝Redis服務LinuxRedis
- Redis輕鬆實現秒殺系統Redis
- Redis 快速提高系統效能的銀彈Redis
- 電商系統架構總結2(Redis)架構Redis
- windows系統上安裝Redis,並且設定Redis密碼WindowsRedis密碼
- oss-server 小型物件儲存系統Server物件
- 系統,物件,角色許可權簡析物件
- 內控系統之控制點物件研究物件
- Redis資料結構與物件Redis資料結構物件
- 用Redis輕鬆實現秒殺系統Redis
- 用 Redis 輕鬆實現秒殺系統Redis
- 在Mac系統上安裝redis服務MacRedis
- 《閒扯Redis十一》Redis 有序集合物件底層實現Redis物件
- ZKEYS管理系統服務於哪些物件?物件
- Qt原始碼解析——元物件系統熱身QT原始碼物件
- 系統、角色、物件相關許可權字典物件
- 物件導向設計-圖書管理系統物件
- oracle物件與系統許可權小測Oracle物件
- Asp.NET系統狀態與物件管理ASP.NET物件