在專案裡快速執行史上最強開源大模型「Llama 3.1」

英智未来發表於2024-08-13

7月份，Meta正式釋出迄今為止最強大的開源模型——Llama 3.1，包含了405B、70B和8B三個版本的模型。

作為開發者，如何在專案裡快速執行LLaMa3.1大模型？

今天，我們結合英智未來自己的LLM推理API平臺，給大家介紹一套免費實踐流程。

免費使用地址：https://cognihub.baystoneai.com

方式一：不使用流式傳輸，直接讓大模型返回最終結果請求：

# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": false,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智慧智匯API是什麼？"}    ]}'

應答：

{  "id": "chatcmpl-754",  "object": "chat.completion",  "created": 1723433933,  "model": "llama3.1",  "system_fingerprint": "fp_ollama",  "choices": [    {      "index": 0,      "message": {        "role": "assistant",        "content": "英智人工智慧智匯API（Application Programming Interface）是指一種開放的開發介面，讓第三方服務或應用能夠訪問和呼叫英智的人工智慧技術。透過這些API，開發者可以整合英智的人工智慧功能，如自然語言處理、影像識別和情緒分析，來提高自己的產品或服務。\n\n例如，一家移動應用開發商可能會使用英智的語音識別API，讓使用者能夠透過語音說話給應用進行互動。或者，一家教育平臺可能會利用英智的情緒分析API，對學生的情緒變化進行實時監測和反饋等應用。\n\n透過API介面，第三方開發者可以使用英智的人工智慧能力，為更加強大和有趣的應用或服務提供支援。這也促進了應用和服務之間的快速創新與整合。"      },      "finish_reason": "stop"    }  ],  "usage": {    "prompt_tokens": 31,    "completion_tokens": 186,    "total_tokens": 217  }}

方式二：設定stream（流式傳輸）請求：

# sh
export API_BASE_URL="https://api.baystoneai.com/cognihub/service/66b97dd5fb19cc7397384f42"export API_KEY="Sy0FY5eH1jOaKtcfVvIxN1zsgRWED4IgfoFA"
curl -X POST $API_BASE_URL/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer $API_KEY" -d '{    "model": "llama3.1",    "stream": true,    "messages": [        {"role": "system", "content": "You are a helpful assistant."},        {"role": "user", "content": "英智人工智慧智匯API是什麼？"}    ]}'

應答：

data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"英"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"人"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"工"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智慧"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"智"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"匯"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"API"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"是一"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"種"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"可以"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434271,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"為"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"網站"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"應用"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"程式"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"和"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"其他"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"裝置"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"提供"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"話"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"語"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"建議"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"、"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"文"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"本"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"分析"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"等"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"服務"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434272,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"的"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"軟體"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"接"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"口"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"。"},"finish_reason":null}]}
data: {"id":"chatcmpl-514","object":"chat.completion.chunk","created":1723434273,"model":"llama3.1","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":"stop"}]}
data: [DONE]

下面，給大家介紹一下「英智LLM推理API平臺」，以及平臺具有的優勢。

英智LLM推理API平臺

英智LLM推理API是一種基於人工智慧大語言模型的推理API服務，它可以幫助開發者簡單靈活地使用市面上的所有人工智慧大語言模型，為企業定製專屬的智慧化解決方案。目前已支援包括Llama 3.1、Mistral Large 2、Qwen 2等在內的主流開源大模型的最新版本，同時相容OpenAI API。

平臺優勢

免費註冊：申請服務的免費套餐後可獲取專屬您的“API_KEY”，即可接入本服務來免費使用API。
線上使用：支援線上使用開源大模型。
相容能力強：提供API呼叫方式，與OpenAI API介面相容(只要支援OpenAI API的應用，都可以使用)。
推理速度快：基於NVIDIA高效能GPU算力，推理速度更快。
自然語言處理：透過對大量文字資料進行分析和理解，實現對自然語言的智慧處理，如情感分析、文字分類、關鍵詞提取等。
語義理解：透過對自然語言的深入理解，實現對使用者意圖的準確識別，從而提供更精準的資訊服務。
智慧問答：透過對大量知識庫的學習和理解，實現對使用者提問的智慧回答，提高使用者滿意度。
文字生成：透過對大量文字資料的學習，實現對特定主題或場景的文字生成，如新聞稿、產品描述等。
語音識別與合成：透過對語音資料的分析，實現對語音的識別和合成，提高人機互動的自然度和便捷性。
影像識別與處理：透過對影像資料的分析和理解，實現對影像內容的識別和處理，如人臉識別、物體檢測等。

免費使用，請訪問：https://cognihub.baystoneai.com

加“英智未來”好友，免費體驗更多AI產品

本地快速安裝執行史上最強開源LLaMa3大模型
2024-08-21
大模型
在Windows電腦上快速執行AI大語言模型-Llama3
2024-04-20
WindowsAI模型
如何擁有自己的專屬GPT-本地部署目前最強大模型llama3
2024-04-26
GPT大模型
5分鐘執行最新llama-3大模型
2024-04-22
大模型
10個最新開源大模型Llama 3應用
2024-04-22
大模型
阿里開源的32B大模型到底強在哪裡？
2024-04-10
阿里大模型
史上最強 iPhone 越獄工具開源：功能永久有效！
2020-08-17
iPhone
擊敗GPT-4o的開源模型如何煉成？關於Llama 3.1 405B，Meta都寫在這篇論文裡了
2024-07-24
GPT模型
非Transformer架構站起來了！首個純無注意力大模型，超越開源巨頭Llama 3.1
2024-08-13
ORM架構大模型
手把手教你本地執行Meta最新大模型：Llama3.1，可是它說自己是ChatGPT？
2024-07-26
大模型ChatGPT
人工智慧大模型之開源大語言模型彙總（國內外開源專案模型彙總）
2024-06-21
人工智慧大模型
本週DBRX等八款大模型開源專案
2024-04-05
大模型
開源史上最成功的八個開源產品
2013-03-18
在Docker容器中執行GUI圖形應用的開源專案
2022-04-01
DockerGUI
歡迎 Llama 3：Meta 的新一代開源大語言模型
2024-04-23
模型
開源史上的8大交易
2010-09-13
英偉達開源最強通用模型Nemotron-4 340B
2024-06-16
模型
idea執行springboot開源專案Guns
2018-08-15
IdeaSpring Boot
首個超越GPT4o級開源模型！Llama 3.1洩密：4050億引數，下載連結、模型卡都有了
2024-07-23
GPT模型
無快不破，在本地 docker 執行 IDEA 裡面的專案？
2021-12-24
DockerIdea
2017年度30大最驚豔的開源機器學習專案
2018-01-08
機器學習
國產最強語音大模型誕生，MaskGCT宣佈開源，聲音效果媲美人類
2024-10-30
大模型GC
史上最全的大廠Mysql面試題在這裡！
2019-01-15
MySql面試題
在Linux上以容器方式執行安卓系統的開源專案：Waydroid
2021-09-10
Linux安卓
華為開源專案ServiceComb快速入門
2018-04-10
Java開源專案Hibernate快速入門
2007-01-01
Java
最適合練手的10大機器學習開源專案，趕緊收藏！
2018-05-08
機器學習
LLAMA3.1資料處理
2024-07-25
史上最強Tableau Server 安裝教程
2021-07-18
Server
搞人工智慧開源大語言模型GPT2、Llama的正確姿勢
2024-11-02
人工智慧模型GPT
開源軟體需順暢執行開源專案要怎樣實施(轉)
2007-08-15
llama.cpp 在 intel cpu 執行最好的後端
2024-07-13
Intel後端
15億引數！史上最強通用NLP模型誕生：狂攬7大資料集最佳紀錄
2019-02-19
模型大資料
依賴開源專案的公司離不開強有力的行為準則
2014-12-05
盤點大廠的那些開源專案 - 微眾銀行
2021-02-18
如何找到並快速上手一個開源專案
2024-07-01
在騰訊雲上部署開源專案
2018-11-20
Asp.NetCore3.1開源專案升級為.Net6.0
2021-12-20
ASP.NETNetCore

在專案裡快速執行史上最強開源大模型「Llama 3.1」

英智LLM推理API平臺

平臺優勢

相關文章