零門檻AI 繪圖:教你為客戶定製 ComfyUI Serverless API 應用

Serverless社区發表於2024-07-31

作者:鷗弋、筱姜

2023年下半年,ComfyUI 以其快速、流暢的影像生成能力,結合多樣的自定義節點,迅速在創作者中流行起來。ComfyUI 的亮點就是能夠批次化生成影像,一鍵載入大量工作流,讓使用者可以輕鬆實現人像生成、背景替換、風格遷移和影像動畫化等功能。越來越多的企業及個人開發者希望藉助 ComfyUI 能力進行 AI 繪畫領域創業或者業務上新,獲得高流量及商業價值,但使用原生的 ComfyUI 仍然存在一些問題:

1.顯示卡資源昂貴且難以購買:GPU 卡池管理技術門檻高:高效能的 GPU 資源不僅價格昂貴,而且往往難以大規模採購。此外,GPU 卡池的有效管理和維護需要複雜的技術支援,也帶來了額外的挑戰。
2.難以應對高併發:原生的 ComfyUI 出圖需要排隊,併發處理能力有限。在面對高併發場景時,尤其是併發請求具有大的波動性時,資源配置難以精確預測,從而可能導致系統錯誤和業務中斷。
3.門檻高,難以對外透出:ComfyUI 擁有一定的門檻,對於普通的創作者而言幾乎無法使用,需要對其進行二次包裝才能讓更多使用者享受到 AI 的便捷。

為了幫助使用者高效率、低成本應對企業級複雜場景,以下介紹 ComfyUI API Serverless 版解決方案,透過使用該方案,使用者可以充分利用 ComfyUI + Serverless 技術優勢快速開發上線 AI 繪畫應用,期待為廣大開發者 AI 繪畫創業及變現提供思路。
相關文章:AI 繪畫平臺難開發,難變現?試試 Stable Diffusion API Serverless 版解決方案

阿里雲X優酷聯名發起的「Creat@AI江湖創作大賽」使用本文章中的解決方案,基於函式計算FC 一鍵部署 AI 繪圖平臺,1分鐘實現 “破次元壁合照”、5分鐘實現 Stable Diffusion、ComfyUI 部署,生成以“少年江湖“為主題的畫作贏萬元獎金。
活動連結:https://developer.aliyun.com/plan/create/snbm
截圖2024-07-30 14.12.49.png

方案優勢

在以往的活動中,我們也面臨了很多非技術相關的使用者期望享受 AI 的魅力。結合實際需要我們給出了 Serverless 化的 ComfyUI 實踐案例,解決了上述問題。

  • 部署簡單:提供基礎 ComfyUI 映象,不需要修改時一鍵即可拉起出圖,需要修改時也只需要修改 ComfyUI 映象地址即可
  • 彈性 GPU:函式計算提供了 GPU 彈性的能力,根據實際請求控制例項個數,有突發流量時自動彈新例項承接請求,完全不需要增加額外的關注
  • 按量付費:函式計算的按量例項為毫秒級粒度的計費策略,用多久就收多少錢,確保每分錢都花在刀刃上
  • ComfyUI Serverless 化改造:對原本不適應 Serverless 彈效能力的 ComfyUI 改造,使其可以支援非同步、併發、彈性等各種 Serverless 能力
  • 前後端聯動:活動開源了一個支援自定義引數,並且併發出圖的前端頁面,可直接提供給客戶使用

應用場景

ComfyUI 提供了非常高的自由度和靈活性,支援定製化工作流,並且可以重複使用,批次出圖,特別適用於需要創意影像生成場景:

  • 藝術創作與設計:藝術家和設計師可以利用 ComfyUI 生成獨特的藝術作品,包括概念藝術、插畫、海報設計等。透過 ComfyUI,他們可以根據自己的創意想法生成初步的影像草稿,然後再進一步細化和完善。
  • 內容製作與營銷:在社交媒體、廣告和營銷領域,ComfyUI 可用於快速生成符合品牌風格的視覺素材,用於社交媒體內容、廣告橫幅、海報等
  • 遊戲開發:遊戲開發者可能利用 ComfyUI 自動生成遊戲內的景觀或建築物的紋理,減少手工製作這些元素所需的時間和成本。
  • 視覺特效與影視後期:電影和電視行業的視覺特效團隊可以使用 ComfyUI 來輔助建立逼真的背景、特殊效果或修復舊影片中的畫面缺陷。

透過 API 介面呼叫 ComfyUI 解決方案

常規的 ComfyUI 出圖的流程大致如下

  • 呼叫 /prompt 介面,發起出圖任務
  • 透過 WebSocket 獲取出圖進度

由於在 Serverless 場景下,無請求的時候例項會被凍結,因此 WebSocket 請求是必須要存在的,且需要保持連線到出圖完成。

在併發請求數比較大的情況下,我們往往期望可以利用 Serverless 的彈性,動態建立多個函式例項處理出圖任務。但由於 ComfyUI 本身是“有狀態”的,難以確保出圖的請求和獲取狀態的請求固定打到同一個例項上,這可能會導致介面的呼叫不符合預期。

為了讓 ComfyUI 更加適配 Serverless 模式,需要針對 ComfyUI 進行一定的改造。
參考 fc-comfyui/src/images/agent 的程式碼,在 ComfyUI 映象裡內建 agent 程式,負責轉換 ComfyUI 請求並且拉起 ComfyUI。
截圖2024-07-30 13.59.35.png

注意!
我們提供的程式碼僅用於運營活動使用,作為 Serverless 方式呼叫的實踐參考。
功能未經過嚴格測試,請根據實際的業務需要開發或調整相關的程式碼,並構建 ComfyUI 映象。

目前提供的 Agent 能力介紹

開啟 Agent 能力,需要增加環境變數

  • USE_AGENT1

當透過 Agent 的 API 呼叫時,建議您調整單例項併發度為 1 ~ 5,確保併發請求儘量使用單獨的例項,提高出圖效率

資料型別

出圖 Prompt

與 ComfyUI 在 Dev Mode 匯出的檔案一致

type TPromptNode struct {
	Inputs    map[string]any `json:"inputs"`
	ClassType string         `json:"class_type"`
	Meta      map[string]any `json:"_meta"`
}

type TPrompt map[string]TPromptNode

LoadImage 節點的引數做了特殊處理,如果內容為 base64 或 http 地址,會自動將對應的檔案上傳,並轉換為 ComfyUI 可識別的形式

進度

// key 為 node id 的 map 物件
type TProgress map[string]TProgressNode

type TProgressNode struct {
	Max         int                  `json:"max"` // 進度的最大值
	Value       int                  `json:"value"` // 當前進度
	Start       int64                `json:"start"` // 開始時間
	LastUpdated int64                `json:"last_updated"` // 最後一次更新時間
	Images      []TProgressNodeImage `json:"images"` // 當前節點輸出的圖片資訊(路徑)
	Results     []string             `json:"results,omitempty"` // 當前節點輸出的圖片 base64
}

介面

出圖請求(HTTP 同步)

路徑:/api/run
Body:json 格式的 prompt 資料
返回值:最後一次的進度(包含圖片資訊)

當需要非同步請求時,需要增加 X-Fc-Invocation-Typetask-id,前者告知 FC 非同步形式呼叫,後者用於記錄當前任務的唯一 id,方便後續獲取狀態

curl http://xxxxx/api/run -v \
	-H 'X-Fc-Invocation-Type: Async' \
  -H "task-id: abcdefg" \
 	-XPOST \
  -d '{
    "3": {
        "inputs": {
            "seed": 1586995582004891,
            "steps": 17,
            "cfg": 6,
            "sampler_name": "dpm_2",
            "scheduler": "karras",
            "denoise": 1,
            "model": [
                "33",
                0
            ],
            "positive": [
                "31",
                0
            ],
            "negative": [
                "32",
                0
            ],
            "latent_image": [
                "5",
                0
            ]
        },
        "class_type": "KSampler",
        "_meta": {
            "title": "KSampler"
        }
    },
    "4": {
        "inputs": {
            "ckpt_name": "majicMIX realistic_v7.safetensors"
        },
        "class_type": "CheckpointLoaderSimple",
        "_meta": {
            "title": "Load Checkpoint"
        }
    },
    "5": {
        "inputs": {
            "width": 1024,
            "height": 784,
            "batch_size": 1
        },
        "class_type": "EmptyLatentImage",
        "_meta": {
            "title": "Empty Latent Image"
        }
    },
    "6": {
        "inputs": {
            "text": "2 human\nhi quality,detailed",
            "clip": [
                "4",
                1
            ]
        },
        "class_type": "CLIPTextEncode",
        "_meta": {
            "title": "CLIP Text Encode (Prompt)"
        }
    },
    "8": {
        "inputs": {
            "samples": [
                "3",
                0
            ],
            "vae": [
                "4",
                2
            ]
        },
        "class_type": "VAEDecode",
        "_meta": {
            "title": "VAE Decode"
        }
    },
    "9": {
        "inputs": {
            "filename_prefix": "ComfyUI",
            "images": [
                "8",
                0
            ]
        },
        "class_type": "SaveImage",
        "_meta": {
            "title": "Save Image"
        }
    },
    "10": {
        "inputs": {
            "image": "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/default.png",
            "upload": "image"
        },
        "class_type": "LoadImage",
        "_meta": {
            "title": "Load Image"
        }
    },
    "11": {
        "inputs": {
            "image": "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/百里東君.png",
            "upload": "image"
        },
        "class_type": "LoadImage",
        "_meta": {
            "title": "Load Image",
            "edit": []
        }
    },
    "12": {
        "inputs": {
            "image": "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/background.png",
            "upload": "image"
        },
        "class_type": "LoadImage",
        "_meta": {
            "title": "Load Image"
        }
    },
    "13": {
        "inputs": {
            "image": "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/mask.png",
            "upload": "image"
        },
        "class_type": "LoadImage",
        "_meta": {
            "title": "Load Image"
        }
    },
    "15": {
        "inputs": {
            "threshold_r": 0.15,
            "threshold_g": 0.15,
            "threshold_b": 0.15,
            "remove_isolated_pixels": 0,
            "fill_holes": false,
            "image": [
                "13",
                0
            ]
        },
        "class_type": "MaskFromRGBCMYBW+",
        "_meta": {
            "title": "🔧 Mask From RGB/CMY/BW"
        }
    },
    "21": {
        "inputs": {
            "image_weight": 0.8,
            "prompt_weight": 1,
            "weight_type": "linear",
            "start_at": 0,
            "end_at": 1,
            "image": [
                "10",
                0
            ],
            "mask": [
                "15",
                0
            ],
            "positive": [
                "24",
                0
            ],
            "negative": [
                "25",
                0
            ]
        },
        "class_type": "IPAdapterRegionalConditioning",
        "_meta": {
            "title": "IPAdapter Regional Conditioning"
        }
    },
    "22": {
        "inputs": {
            "image_weight": 1,
            "prompt_weight": 1,
            "weight_type": "linear",
            "start_at": 0,
            "end_at": 1,
            "image": [
                "11",
                0
            ],
            "mask": [
                "15",
                1
            ],
            "positive": [
                "26",
                0
            ],
            "negative": [
                "25",
                0
            ]
        },
        "class_type": "IPAdapterRegionalConditioning",
        "_meta": {
            "title": "IPAdapter Regional Conditioning"
        }
    },
    "23": {
        "inputs": {
            "image_weight": 0.7000000000000001,
            "prompt_weight": 1,
            "weight_type": "linear",
            "start_at": 0,
            "end_at": 1,
            "image": [
                "12",
                0
            ],
            "mask": [
                "15",
                6
            ]
        },
        "class_type": "IPAdapterRegionalConditioning",
        "_meta": {
            "title": "IPAdapter Regional Conditioning"
        }
    },
    "24": {
        "inputs": {
            "text": "illustration of a body with black hair, presented in high definition with intricate details",
            "clip": [
                "4",
                1
            ]
        },
        "class_type": "CLIPTextEncode",
        "_meta": {
            "title": "CLIP Text Encode (Prompt)"
        }
    },
    "25": {
        "inputs": {
            "text": "(worst quality:1.6),(low quality:1.6),(lowres:1.6),(NSFW:1.5),watermark,monochrome,disconnected limbs,malformed limbs,extra limb,mutated hands,fused fingers,too many fingers,extra arms,missing fingers,bad hands,bad feet,mutated hands and fingers,malformed hands,extra legs,floating limbs,missing limb,mutation,mutated,deformed,bad body,poorly drawn hands,(badhandv4),(naked),(nude),",
            "clip": [
                "4",
                1
            ]
        },
        "class_type": "CLIPTextEncode",
        "_meta": {
            "title": "CLIP Text Encode (Prompt)"
        }
    },
    "26": {
        "inputs": {
            "text": "anime Aillustration of 1 boy with black hair, depicted in high definition showcasing rich details, in 8k resolution.",
            "clip": [
                "4",
                1
            ]
        },
        "class_type": "CLIPTextEncode",
        "_meta": {
            "title": "CLIP Text Encode (Prompt)"
        }
    },
    "28": {
        "inputs": {
            "params_1": [
                "21",
                0
            ],
            "params_2": [
                "22",
                0
            ],
            "params_3": [
                "23",
                0
            ]
        },
        "class_type": "IPAdapterCombineParams",
        "_meta": {
            "title": "IPAdapter Combine Params"
        }
    },
    "31": {
        "inputs": {
            "conditioning_1": [
                "21",
                1
            ],
            "conditioning_2": [
                "22",
                1
            ],
            "conditioning_3": [
                "6",
                0
            ],
            "conditioning_4": [
                "47",
                0
            ]
        },
        "class_type": "ConditioningCombineMultiple+",
        "_meta": {
            "title": "🔧 Conditionings Combine Multiple "
        }
    },
    "32": {
        "inputs": {
            "conditioning_1": [
                "47",
                1
            ],
            "conditioning_2": [
                "22",
                2
            ],
            "conditioning_3": [
                "25",
                0
            ]
        },
        "class_type": "ConditioningCombineMultiple+",
        "_meta": {
            "title": "🔧 Conditionings Combine Multiple "
        }
    },
    "33": {
        "inputs": {
            "combine_embeds": "concat",
            "embeds_scaling": "V only",
            "model": [
                "4",
                0
            ],
            "ipadapter": [
                "34",
                1
            ],
            "ipadapter_params": [
                "28",
                0
            ]
        },
        "class_type": "IPAdapterFromParams",
        "_meta": {
            "title": "IPAdapter from Params"
        }
    },
    "34": {
        "inputs": {
            "preset": "PLUS (high strength)",
            "model": [
                "4",
                0
            ]
        },
        "class_type": "IPAdapterUnifiedLoader",
        "_meta": {
            "title": "IPAdapter Unified Loader"
        }
    },
    "43": {
        "inputs": {
            "clip_name": "CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors"
        },
        "class_type": "CLIPVisionLoader",
        "_meta": {
            "title": "Load CLIP Vision"
        }
    },
    "45": {
        "inputs": {
            "ipadapter_file": "ip-adapter-plus_sd15.safetensors"
        },
        "class_type": "IPAdapterModelLoader",
        "_meta": {
            "title": "IPAdapter Model Loader"
        }
    },
    "46": {
        "inputs": {
            "provider": "CPU"
        },
        "class_type": "IPAdapterInsightFaceLoader",
        "_meta": {
            "title": "IPAdapter InsightFace Loader"
        }
    },
    "47": {
        "inputs": {
            "strength": 0.8,
            "start_percent": 0,
            "end_percent": 1,
            "positive": [
                "21",
                1
            ],
            "negative": [
                "21",
                2
            ],
            "control_net": [
                "48",
                0
            ],
            "image": [
                "49",
                0
            ]
        },
        "class_type": "ControlNetApplyAdvanced",
        "_meta": {
            "title": "Apply ControlNet (Advanced)"
        }
    },
    "48": {
        "inputs": {
            "control_net_name": "control_v11p_sd15_openpose_fp16.safetensors"
        },
        "class_type": "ControlNetLoader",
        "_meta": {
            "title": "Load ControlNet Model"
        }
    },
    "49": {
        "inputs": {
            "detect_hand": "enable",
            "detect_body": "enable",
            "detect_face": "enable",
            "resolution": 512,
            "image": [
                "10",
                0
            ]
        },
        "class_type": "OpenposePreprocessor",
        "_meta": {
            "title": "OpenPose Pose"
        }
    }
}'


{"":{"max":0,"value":0,"start":0,"last_updated":1722234889,"images":null},"10":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"11":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"12":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"13":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"15":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"21":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"22":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"23":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"24":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"25":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"26":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"28":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"3":{"max":17,"value":17,"start":1722234848,"last_updated":1722234889,"images":null},"31":{"max":1,"value":0,"start":1722234848,"last_updated":1722234848,"images":null},"32":{"max":1,"value":0,"start":1722234848,"last_updated":1722234848,"images":null},"33":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"34":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"4":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"43":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"45":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"46":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"47":{"max":1,"value":0,"start":1722234848,"last_updated":1722234848,"images":null},"48":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"49":{"max":1,"value":1,"start":1722234846,"last_updated":1722234848,"images":null},"5":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"6":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"8":{"max":1,"value":0,"start":1722234889,"last_updated":1722234889,"images":null},"9":{"max":1,"value":0,"start":1722234889,"last_updated":1722234889,"images":[{"filename":"ComfyUI_00004_.png","subfolder":"","type":"output"}]}}

出圖請求(WebSocket)

路徑:/api/run/ws
Message:

  • 客戶端 -> 服務端:僅傳送一次,json 格式的 prompt 資訊
  • 服務端 -> 客戶端:中間狀態

獲取狀態

路徑:/api/run/ws?id=
Query 引數:

  • id:task id
curl http://xxxxx/api/status?id=abcdefg -v


{"":{"max":0,"value":0,"start":0,"last_updated":1722234889,"images":null},"10":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"11":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"12":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"13":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"15":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"21":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"22":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"23":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"24":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"25":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"26":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"28":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"3":{"max":17,"value":17,"start":1722234848,"last_updated":1722234889,"images":null},"31":{"max":1,"value":0,"start":1722234848,"last_updated":1722234848,"images":null},"32":{"max":1,"value":0,"start":1722234848,"last_updated":1722234848,"images":null},"33":{"max":1,"value":0,"start":1722234844,"last_updated":1722234844,"images":null},"34":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"4":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"43":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"45":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"46":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"47":{"max":1,"value":0,"start":1722234848,"last_updated":1722234848,"images":null},"48":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"49":{"max":1,"value":1,"start":1722234846,"last_updated":1722234848,"images":null},"5":{"max":0,"value":0,"start":0,"last_updated":0,"images":null},"6":{"max":0,"value":0,"start":0,"last_updated":0,"imag* Connection #0 to host photo-b-comfyui-ibiwqxodsh.cn-hangzhou.fcapp.run left intact
es":null},"8":{"max":1,"value":0,"start":1722234889,"last_updated":1722234889,"images":null},"9":{"max":1,"value":0,"start":1722234889,"last_updated":1722234889,"images":[{"filename":"ComfyUI_00004_.png","subfolder":"","type":"output"}]}}

其他

原樣轉發至 ComfyUI

呼叫方式

同步呼叫

/api/run/api/run/ws 都是同步介面,直接呼叫即可,區別在於是否需要出圖進度

  • 在 WebSocket 內部獲取:只呼叫 /api/run/ws
  • 不關心出圖進度 / 起另一個執行緒獲取進度:使用 /api/run + /api/status

:當選擇 /api/run + /api/status 方式時,您需要掛載一個 NAS 例項或改造程式碼,將狀態存放至 OTS 等資料庫,否則在多例項時無法獲取進度

非同步呼叫

呼叫 /api/run 介面,並且新增 HTTP Header,藉助函式計算自帶的能力,將請求轉換為非同步形式

  • Key:X-Fc-Invocation-Type
  • Value:Async

:當選擇非同步呼叫時,您需要掛載一個 NAS 例項或改造程式碼,將狀態存放至 OTS 等資料庫,否則在多例項時無法獲取進度

二次開發

我們提供的 agent 僅用作參考,正式使用時,請根據業務需要進行二次開發

狀態儲存

src/images/agent/pkg/store/fs.go 中,我們實現了基於檔案系統的狀態儲存,您只需要掛載 NAS 系統,確保檔案可被正常持久化,既可以在多個例項之間共享狀態檔案,確保可以正確拿到狀態資訊
更好的做法是,將狀態資訊寫入到 OTS、MySQL 等資料庫中,您只需要仿照 fs.go 實現 Stroe 介面針對其他資料庫的實現即可

// Store KV 資料儲存
type Store interface {
    // Save 儲存 value 到 key
    Save(key string, value string) error
    // Load 從 key 載入 value
    Load(key string) (string, error)
}

Output 節點

目前,agent 僅針對 SaveImage 節點做了特殊處理,提取其中的圖片資訊。對於特殊的業務需要,您可能需要更加定製化的工作流處理,如

  • 增加更多對於 Output 的解析
  • 不解析圖片節點,而是藉助於其他介面獲取圖片檔案
case "execution_error", "executed":
			// 節點執行結束
			log.Debugf("%s node %s finished", logPrefix, nodeid)

			// 節點已完成時,修改下 Max 和 Value 至少為 1
			if currentNodeProgress.Max == 0 && currentNodeProgress.Value == 0 {
				currentNodeProgress.Max = 1
				currentNodeProgress.Value = 1
			}

			if promptNode.ClassType == "SaveImage" && msg.Data.Output.Images != nil && len(msg.Data.Output.Images) > 0 {
				// 如果是圖片節點,則記錄一下圖片資料
				if currentNodeProgress.Images == nil {
					currentNodeProgress.Images = make([]store.TProgressNodeImage, 0, len(msg.Data.Output.Images))
				}

				for _, img := range msg.Data.Output.Images {
					currentNodeProgress.Images = append(currentNodeProgress.Images, store.TProgressNodeImage{
						Filename:  img.Filename,
						SubFolder: img.SubFolder,
						Type:      img.Type,
					})
				}
			}

前端功能整合

與 Agent 對應,我們也給出了一份前端頁面
devsapp/fc-comfyui-couple-photo

在這裡,我們針對 ComfyUI 的 prompt 做了一些特殊的約定,以適應自定義需要。
以函式計算支援活動 “阿里雲X優酷江湖創作大賽” 為例,我們提供了預定義的 prompt 檔案

[
  {
    "title": "破次元壁合照",
    "prompt": {},
    "params": [
      {
        "type": "group",
        "title": "STEP 1 - 上傳您的照片",
        "children": [
          {
            "type": "image",
            "id": "10",
            "key": "image",
            "title": "參考圖",
            "description": "請上傳您的照片,幫助模型理解您的樣貌。請儘量選擇背景簡單、主體突出的半身照,不要佩戴墨鏡、帽子等可能影響您特徵的衣物。"
          },
          {
            "type": "string",
            "id": "24",
            "key": "text",
            "title": "參考形象描述",
            "description": "為了確保模型更好地理解您的特點,您可以使用提示詞來加強模型對您的印象(請使用因為描述)。"
          }
        ]
      },
      {
        "type": "image",
        "id": "11",
        "key": "image",
        "title": "STEP 2 - 選擇角色",
        "description": "請選擇您希望合照的角色。",
        "options": [
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/百里東君.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/司空長風.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/玥瑤.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/葉鼎之.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/易文君.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/南宮春水.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/蕭若風.png"
        ]
      },
      {
        "type": "image",
        "id": "12",
        "key": "image",
        "title": "STEP 3 - 上傳背景圖",
        "description": "請上傳您期望的合影地點的圖片,這將作為背景圖片的參考。"
      }
    ]
  },
  {
    "title": "背景替換",
    "prompt": {},
    "params": [
      {
        "type": "image",
        "id": "10",
        "key": "image",
        "title": "STEP 1 - 選擇角色",
        "description": "請選擇您希望合照的角色。",
        "options": [
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/百里東君.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/司空長風.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/玥瑤.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/葉鼎之.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/易文君.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/南宮春水.png",
          "https://serverless-tool-images.oss-cn-hangzhou.aliyuncs.com/aigc/json/couple/蕭若風.png"
        ]
      },
      {
        "type": "image",
        "id": "12",
        "key": "image",
        "title": "STEP 2 - 上傳背景圖",
        "description": "請上傳您期望的合影地點的圖片,這將作為背景圖片的參考。"
      }
    ]
  }
]

透過 params 欄位,約定了如何渲染頁面並允許使用者填入自己的引數

export type ComfyUIPromptEditPanel = {
  type: 'image' | 'select' | 'number' | 'string' | 'group'; // 資料型別
  id?: string; // 對應 prompt 中的 node id
  key: string; // 要修改的引數
  title: string; // 標題
  description?: string; // 描述
  options?: string[] | string; // 可選項
  min?: number; // 最小值
  max?: number; // 最大值
  step?: number; // 調整步數
  hidden?: boolean; // 是否隱藏
  children?: ComfyUIPromptEditPanel[]; // group 型別的子節點
};

一些其他約定:

  • 如果 seed 欄位為 -1,則會被替換為隨機數

如果您也希望建立自己的 ComfyUI 自定義頁面提供給自己的客戶,可以參考相關的前端程式碼。

最佳實踐

為了方便大家直觀體驗一下該解決方案成效,函式計算Serverless 應用中心上線基於 ComfyUI Serverless API 解決方案搭建的 應用-【少年白馬專屬】破次元壁合照 AI 繪畫平臺,作為一個實驗demo 開放體驗,期待為廣大開發者 AI 繪畫創業及變現提供一些有益思考。直接參加體驗活動,送好禮!
活動連結:https://developer.aliyun.com/plan/create/snbm
截圖2024-07-30 14.07.49.png

更多內容關注 Serverless 微信公眾號(ID:serverlessdevs),彙集 Serverless 技術最全內容,定期舉辦 Serverless 活動、直播,使用者最佳實踐。

相關文章