ollama 最快方式部署管理大模型

夜半钟声到客船發表於2024-08-29
大模型

github:https://github.com/ollama/ollama

模型地址:https://ollama.com/library/llama3.1

linux: 安裝

1.下載安裝指令碼

curl -fsSL https://ollama.com/install.sh | sh

2.修改啟動環境變數

如果是root 使用者記得改為root

vim /etc/systemd/system/ollama.service

[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/local/bin/ollama serve
User=root
Group=root
Restart=always
RestartSec=3
Environment="PATH=/root/.nvm/versions/node/v18.20.4/bin:/home/miniconda3/bin:/home/miniconda3/condabin:/usr/lib64/qt-3.3/bin:/root/perl5/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/usr/local/mysql/bin"
Environment="OLLAMA_DEBUG=1"
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment=" OLLAMA_KEEP_ALIVE=5h"
Environment="OLLAMA_MAX_LOADED_MODELS=10"
#export OLLAMA_MAX_QUEUE=100
Environment="OLLAMA_MODELS=/home/data/llm/ollama/models/"
[Install]
WantedBy=default.target

2.相關命令

(base) [root@ceph1 ~]# ollama 
Usage:
  ollama [flags]
  ollama [command]

Available Commands:
  serve       Start ollama
  create      Create a model from a Modelfile
  show        Show information for a model
  run         Run a model
  pull        Pull a model from a registry
  push        Push a model to a registry
  list        List models
  ps          List running models
  cp          Copy a model
  rm          Remove a model
  help        Help about any command

Flags:
  -h, --help      help for ollama
  -v, --version   Show version information

Use "ollama [command] --help" for more information about a command.

3. 執行模型執行

o ollama run llama3.1:70b

首次執行會下載模型到 環境變數Environment="OLLAMA_MODELS=/home/data/llm/ollama/models/"

第二次就不用了下載執行 開始執行

相關文章