mac m1 arm 安裝 confluent-kafka 報錯解決方案

ponponon發表於2021-10-30
                    'c.          bot@mbp13m1.local 
                 ,xNMM.          ----------------- 
               .OMMMMo           OS: macOS 11.6 20G165 arm64 
               OMMM0,            Host: MacBookPro17,1 
     .;loddo:' loolloddol;.      Kernel: 20.6.0 
   cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 3 days, 22 hours, 18 mins 
 .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 44 (brew) 
 XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.8 
;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: 1440x900, 2560x1440 
:MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua 
.MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor 
 kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Light) 
 .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: pycharm 
  .XMMMMMMMMMMMMMMMMMMMMMMMMK.   CPU: Apple M1 
    kMMMMMMMMMMMMMMMMMMMMMMd     GPU: Apple M1 
     ;KMMMMMMMWXXWMMMMMMMk.      Memory: 1616MiB / 8192MiB 
       .cooc,.    .,coo:.
                                                         
                                                         

在 Linux 上沒有問題,但是在 Mac 上卻不行了,安裝 confluent-kafka 會報找不到相關標頭檔案的錯誤,因為 confluent-kafka 使用了用 C語言編寫的 librdkafka 作為基礎,而 Mac 糟糕的標頭檔案管理方式讓一切變得糟糕。

使用下面的命令安裝 confluent-kafka 的時候報錯了

pip install confluent-kafka

報錯內容如下:

ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"'; __file__='"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-record-7jy58dai/install-record.txt --single-version-externally-managed --compile --install-headers /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/confluent-kafka
         cwd: /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/
...
 In file included from /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/src/confluent_kafka/src/Admin.c:17:
    /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error: 'librdkafka/rdkafka.h' file not found
    #include <librdkafka/rdkafka.h>
             ^~~~~~~~~~~~~~~~~~~~~~
    1 error generated.
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"'; __file__='"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-record-7jy58dai/install-record.txt --single-version-externally-managed --compile --install-headers /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/confluent-kafka Check the logs for full command output.

解決辦法,安裝 librdkafka

brew install librdkafka

光著還不夠哦,此時還是找不到標頭檔案的,我們還需要讓 pip 可以找到 librdkafka 在哪裡(Linux 請無視)

Looks like Python is not making use of the homebrew installation location (which should be linked to /usr/local/), try fixing the homebrew package with brew link --overwrite librdkafka and if that doesn't work you can force it in the right direction by:

C_INCLUDE_PATH=/usr/local/Cellar/librdkafka/0.11.0/include LIBRARY_PATH=/usr/local/Cellar/librdkafka/0.11.0/lib pip install confluent_kafka

我安裝的 librdkafka 版本是 1.7.0

C_INCLUDE_PATH=/opt/homebrew/Cellar/librdkafka/1.7.0/include LIBRARY_PATH=/opt/homebrew/Cellar/librdkafka/1.7.0/lib pip install confluent_kafka

上面是一行命令,千萬不要拆開執行,不然無效

參考文章:
linux gcc標頭檔案搜尋路徑
Install librdkafka dependency on Mac OSX

相關文章