stable diffusion ControlNet使用介紹與進階技巧

破晓一代發表於2024-07-02

ControlNet是什麼？固定構圖、定義姿勢、描繪輪廓、單憑線稿就能生成一張豐滿精緻的插畫……它幾乎無所不能。

有人把它稱為AI繪畫界的“革命性”突破，但在我看來，它不過是StableDiffusion邁向“工業化”的第一步。

ControlNet擴充套件與模型下載地址

擴充套件地址：

https://github.com/Mikubill/sd-webui-controlnet

1.1版本模型地址：

（下載地址）https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main

（模型介紹）https://github.com/lllyasviel/ControlNet-v1-1-nightly

1.14版本後更新的新模型（包括社群模型）與XL模型的地址：https://huggingface.co/lllyasviel/sd_control_collection/tree/main

*模型下載後，一般放置於stable-diffusion-webui（根目錄）\extensions\sd-webui-controlnet\models or stable-diffusion-webui（根目錄）\models\ControlNet

前處理器下載地址（缺東西在裡面找）：

https://huggingface.co/lllyasviel/Annotators/tree/main

*前處理器下載後，一般放置於stable-diffusion-webui（根目錄）\extensions\sd-webui-controlnet\annotator\downloads

2024-06-29T12:36:32.png

生成資訊

注：生成資訊也可以讀取到ControlNet，但無法直接一鍵應用。

你可以從最後這些欄位中，檢視這張圖片用了幾個ControlNet，權重、引導步數分別是多少。

2024-06-29T12:36:51.png

以上介紹內容參考：https://nenly.notion.site/10-ControlNet-a802ab59c48d45f8ae4ad1cfea0a0d4d

正文

AI二維碼+藝術字+光影光效+創意Logo生成方法

藝術字實現介紹

在ps中製作一個白底黑字的圖片

2024-06-30T01:37:59.png

拉入到ControlNet

2024-06-30T01:40:07.png

生成引數參考

stone, rock, coal,sand,from above,
high quality, highres, masterpiece, solid background
Negative prompt: (worst quality:2),(low quality:2),(normal quality:2),lowres,watermark,monochrome
Steps: 20, Sampler: Euler, Schedule type: Automatic, CFG scale: 7, Seed: 2109960192, Size: 960x540, Model hash: 038ba203d8, Model: 二次元：AbyssOrangeMix2_sfw, ControlNet 0: "Module: depth_midas, Model: control_v11f1p_sd15_depth [cfd03158], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Version: 1.9.4

2024-06-30T01:44:04.png

使用不同的模型,logo，ControlNet預處理和提示詞（不同材質）可以生成不同風格的圖片

2024-06-30T01:49:14.png

2024-06-30T02:02:00.png

介紹圖片融字實現

兩個新模型：brightness（亮度模型） & illumination（明度模型）

2024-06-30T02:01:19.png

模型下載連結（來自nenly網盤）：https://pan.baidu.com/share/init?surl=0rzgzIjzad7AKmj-w8zO_w&pwd=nely

修改ControlNet引數如下（這裡用invert是交換黑白部分）

2024-06-30T02:11:59.png

生成效果

2024-06-30T02:15:55.png

引數

gorgeous lips, cinematic, (masterpiece), (best quality), (ultra-detailed), very aesthetic, illustration, perfect composition, intricate details, absurdres, detailed face, (anime, masterpiece, intricate:1.3), (best quality, hires textures, high detail:1.2), (4k),(incredibly detailed:1.4)
Masterpiece, highest quality, high resolution, cinematic lighting, anime illustration, (rabbit ears), rabbit ears, ear fluff, girl looking up at the viewer, girl looking back in the forest, angle looking down on girl, bust shot, solo, angle looking down from above, hairs between eyes, long sideburns, red eyes, long hair, silver hair, hair swaying in the wind, petite, ((short)), young, red ribbon on chest,
BREAK
White maid outfit, maid headband, puff sleeves, short sleeves, shadow on chest, long skirt, frills,
BREAK
Birch forest, basket, sunlight filtering through the trees, flowers, white flowers, background of square in forest, detailed background, lens flare, chromatic aberration, depth of field,
Negative prompt: loli, child, longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, worst quality, low quality, normal quality, watermark, artist name, signature
Steps: 30, Sampler: Euler, Schedule type: Automatic, CFG scale: 8, Seed: 4076082077, Size: 960x540, Model hash: 038ba203d8, Model: 二次元：AbyssOrangeMix2_sfw, VAE hash: df3c506e51, VAE: kl-f8-anime2.ckpt, ControlNet 0: "Module: invert (from white bg & black line), Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.5, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0.1, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Version: 1.9.4

2024-06-30T02:16:53.png

將文字新增在人物衣服上

例如這張圖

Photo of a stunning country 22 year old woman, freckles, beauty spots, slightly curvy, flannel shirt open, winter hat, hard shadows, soft light, summer,full body,
Negative prompt: (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers:1.4), (deformed, distorted, disfigured:1.3), poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation
Steps: 8, Sampler: DPM++ 2M SDE, Schedule type: Exponential, CFG scale: 1.5, Seed: 1811433106, Size: 675x1013, Model hash: f47e942ad4, Model: realisticVisionV60B1_v51HyperVAE, VAE hash: df3c506e51, VAE: kl-f8-anime2.ckpt, Version: 1.9.4

2024-06-30T03:04:43.png