redis rebloom 報錯 Maximum expansions reached

使用 rebloom 遇到了報錯：redis.exceptions.ResponseError: Maximum expansions reached

具體內容如下：

  File "/usr/local/lib/python3.10/site-packages/redisbloom/client.py", line 411, in cfAddNX
    return self.execute_command(self.CF_ADDNX, *params)
           │    │               │    │          └ ['crawler', 'https://power.in-en.com/html/power-2402142.shtml']
           │    │               │    └ 'CF.ADDNX'
           │    │               └ Client<ConnectionPool<Connection<host=172.16.36.108,port=6379,db=0>>>
           │    └ <function Redis.execute_command at 0x7f49110785e0>
           └ Client<ConnectionPool<Connection<host=172.16.36.108,port=6379,db=0>>>
  File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 901, in execute_command
    return self.parse_response(conn, command_name, **options)
           │    │              │     │               └ {}
           │    │              │     └ 'CF.ADDNX'
           │    │              └ Connection<host=172.16.36.108,port=6379,db=0>
           │    └ <function Redis.parse_response at 0x7f4911078670>
           └ Client<ConnectionPool<Connection<host=172.16.36.108,port=6379,db=0>>>
  File "/usr/local/lib/python3.10/site-packages/redis/client.py", line 915, in parse_response
    response = connection.read_response()
               │          └ <function Connection.read_response at 0x7f49111f3c70>
               └ Connection<host=172.16.36.108,port=6379,db=0>
  File "/usr/local/lib/python3.10/site-packages/redis/connection.py", line 756, in read_response
    raise response
          └ ResponseError('Maximum expansions reached')

redis.exceptions.ResponseError: Maximum expansions reached

問題原因：布隆過濾器的容量滿了

解決方案：

如果你是用 redis cli，就先刪除 key，再重新建立 key。用這個命令建立 CF.RESERVE，具體參考：CF.RESERVE

比如 CF.RESERVE crawler 100000000 就是建立一個容量為一億的布隆過濾器

當然，這個容量到底用多少，需要自己評估

如果你要檢視當前布隆過濾器的容量大小，就用 CF.INFO 命令，比如：

127.0.0.1:6379> CF.INFO clip_url_hash
 1) "Size"
 2) "16777272"
 3) "Number of buckets"
 4) "8388608"
 5) "Number of filters"
 6) "1"
 7) "Number of items inserted"
 8) "21436"
 9) "Number of items deleted"
10) "0"
11) "Bucket size"
12) "2"
13) "Expansion rate"
14) "1"
15) "Max iterations"
16) "20"

如果你是用 python, 可以按照下面的程式碼來建立新的 key

from redisbloom.client import Client
import settings

client = Client(
    host=settings.REDIS_CONFIG.host,
    port=settings.REDIS_CONFIG.port,
    db=settings.REDIS_CONFIG.db
)

def create_filter():
    client.cfCreate(key, 1000000000)

redis rebloom 報錯 Maximum expansions reached

相關文章