Rust Vulkan 逐幀渲染流程介紹

熊皮皮發表於2018-12-04

2019.5.10 修改：渲染到檢視

gfx-hal介面以1:1模仿Vulkan，下面改用Vulkan介面作說明。由於Vulkan介面粒度過細，比OpenGL / ES難學。根據個人經驗，對於移動端圖形開發者，照著OpenGL ES的介面講解Vulkan可降低學習難度。從逐幀渲染部分開始學習，跳過這些資料結構的初始化過程，有利於把握Vulkan的核心流程。

OpenGL / ES 逐幀渲染流程示例

// 準備渲染目標環境
glBindFramebuffer();
glFramebufferTexture2D(); glCheckFramebufferStatus(); // 假如渲染到紋理
glViewport(x, y, width, height);
// 準備Shader需要讀取的資料
glUseProgram(x);
glBindBuffer(i)
loop i in 0..VertexVarCount {
    glEnableVertexAttribArray(i);
    glVertexAttribPointer(i, ...); 
}
loop i in 0..UniformVarCount {
    switch UniformType {
        case NoTexture: glUniformX(i, data); break;
        case Texture: {
            glActiveTexture(j);
            glBindTexture(type, texture_name);
            glUniform1i(location, j);
            break;
        }
        default:ERROR();
    }
}
// 配置其他Fragment操作，比如glBlend, glStencil
glDrawArrays/Elements/ArraysInstanced...
// 到此完成Draw Call，視情況呼叫EGL函式交換前後幀緩衝區，非GL函式，
// 渲染到紋理則無此操作。
// 為了不干擾後續繪製，恢復剛才設定的Framebuffer操作為預設值。
eglSwapbuffers()/[EAGLContext presentRenderbuffer]; 
複製程式碼

可見，OpenGL / ES的介面遮蔽了絕大部分細節，整體程式碼量顯得很少，但初學時也不好理解，用久了就成套路，覺得就該這樣，以致於第一次接觸Vulkan發現很多細節之前完全不瞭解，有點懵。

OpenGL / ES形成套路後的缺點是，出錯的第一時間很難定位出是專案程式碼的問題，比如狀態機沒設定好，還是驅動的問題，在iOS上還好，Android真是眼黑。我猜你會說有各廠家的Profile工具和Google gapid，都特麼不好用，高通的技術支援建議我們用Android 8.0 + Root裝置。但是，往往出問題的都是Android 4.x。

渲染到檢視

gfx-hal(Vulkan)逐幀渲染到檢視的核心呼叫流程如下所示：

-> CommandPool -> ComanndBuffer 

-> Submit -> Submission-> QueueGroup -> CommandQueue -> GraphicsHardware
複製程式碼

說明：

CommandQueue：用於執行不同型別任務的佇列，比如渲染任務、計算任務。
QueueGroup：CommandQueue集合
GraphicsHardware：圖形硬體

具體流程程式碼：

重置Fence，給後面提交Submission到佇列使用。
```
device.reset_fence(&frame_fence);
複製程式碼
```
重置CommandPool，即重置從此Pool中建立的CommandBuffer。如果CommandBuffer還在中，需要開發者實現資源同步操作。
```
command_pool.reset();
複製程式碼
```

從Swapchain獲取Image索引

let frame = swap_chain.acquire_image(!0, FrameSync::Semaphore(&mut frame_semaphore));
複製程式碼

通過CommandPool建立、配置CommandBuffer，命令錄製結束後得到有效的Submit物件

let mut cmd_buffer = command_pool.acquire_command_buffer(false);
// 一系列類似OpenGL / ES的Fragment操作、繫結資料到Program的配置
// 兩個值得注意的Pipeline操作
cmd_buffer.bind_graphics_pipeline(&pipeline);
cmd_buffer.bind_graphics_descriptor_sets(&pipeline_layout, 0, Some(&desc_set), &[]);
// 聯合RenderPass的操作
let mut encoder = cmd_buffer.begin_render_pass_inline(&render_pass,...);
let submit = cmd_buffer.finish()
複製程式碼

通過Submit建立Submission

let submission = Submission::new()
    .wait_on(&[(&frame_semaphore, PipelineStage::BOTTOM_OF_PIPE)])
    .submit(Some(submit));
複製程式碼

提交Submission到佇列

queue.submit(submission, Some(&mut frame_fence));
複製程式碼

等待GPU執行完成

device.wait_for_fence(&frame_fence, !0);
複製程式碼

交換前後幀緩衝區，相當於eglSwapbuffers

swap_chain.present(&mut queue_group.queues[0], frame, &[])
複製程式碼

詳細介紹CommandBuffer的配置

OpenGL / ES 2/3.x沒CommandPool與CommandBuffer資料結構，除了最新的OpenGL小版本才加入了SPIR-V和Command，但OpenGL ES還沒更新。Metal的CommandBuffer介面定義不同於Vulkan。Metal建立MTLCommandBuffer，由Buffer與RenderPassDescriptor一起建立出 Enconder，然後打包本次渲染相關的資源，最後提交Buffer到佇列讓GPU執行。Vulkan基本把Metal的Encoder操作放到CommandBuffer，只留了很薄的Encoder操作。

總體流程：

由Command Pool分配可用Command Buffer
配置viewport等資訊
配置頂點資料緩衝區
配置Uniform與Buffer的對應關係
設定輸出目標RenderPass
設定繪製方式，draw/draw_indexed/draw_indirect等等
結束配置

程式碼示例如下：

let submit = {
    // 從緩衝區中取出一個實際為RawCommandBuffer的例項，
    // 加上執行緒安全物件，組裝成執行緒安全的CommandBuffer例項，
    // 這是HAL的程式設計“套路”，還有很多這類資料結構
    let mut cmd_buffer = command_pool.acquire_command_buffer(false);
    cmd_buffer.begin()

    cmd_buffer.set_viewports(0, &[viewport]);
    cmd_buffer.set_scissors(0, &[viewport.rect]);
    cmd_buffer.bind_graphics_pipeline(&pipeline);
    cmd_buffer.bind_vertex_buffers(0, pso::VertexBufferSet(vec![(&vertex_buffer, 0)]));
    cmd_buffer.bind_graphics_descriptor_sets(&pipeline_layout, 0, Some(&desc_set)); //TODO

    // encoder
    cmd_buffer.begin_render_pass_inline(
            &render_pass,
            &framebuffers[frame.id()],
            viewport.rect,
            &[command::ClearValue::Color(command::ClearColor::Float([0.8, 0.8, 0.8, 1.0]))],
        ).draw(0..6, 0..1);

    cmd_buffer.finish()
};
複製程式碼

這段程式碼顯示了CommandBuffer兩個很關鍵的操作：bind_graphics_pipeline(GraphicsPipeline)和bind_graphics_descriptor_sets(PipelineLayout, DescriptorSet)。GraphicsPipeline相當於OpenGL / ES的Program，PipelineLayout和DescriptorSet描述了Shader的Uniform變數如何讀取Buffer的資料。

渲染到紋理

渲染到紋理(Render to Texture, RTT)場景沒Swapchain，需配置RenderPass.Attachment.format為紋理的畫素格式。接著Submmit到Queue，流程就結束了，無需且無法呼叫swap_chain.present()。如果要獲取該CommandBuffer的GPU操作結束事件或耗時，新增相應的回撥函式給CommandBuffer即可。

配置`RenderPass.Attachment.format`

let render_pass = {
    let attachment = Attachment {
        format: Some(format), /// 紋理的畫素格式
        samples: 1,
        ops: AttachmentOps::new(AttachmentLoadOp::Clear, AttachmentStoreOp::Store),
        stencil_ops: AttachmentOps::DONT_CARE,
        layouts: Layout::Undefined..Layout::Present,
    };

    let subpass = SubpassDesc {
        colors: &[(0, Layout::ColorAttachmentOptimal)], /// 匹配了 Vulkan 要求
        depth_stencil: None,  /// 同上應該使用 Optimal
        inputs: &[],
        resolves: &[],
        preserves: &[],
    };

    let dependency = SubpassDependency {
        passes: SubpassRef::External..SubpassRef::Pass(0),
        stages: PipelineStage::COLOR_ATTACHMENT_OUTPUT..PipelineStage::COLOR_ATTACHMENT_OUTPUT,
        accesses: Access::empty()..(Access::COLOR_ATTACHMENT_READ | Access::COLOR_ATTACHMENT_WRITE),
    };

    device.create_render_pass(&[attachment], &[subpass], &[dependency])
          .expect("Can't create render pass")
};
複製程式碼

提交到CommandQueue

和渲染到檢視一樣提交即可，少一步swap_chain.present()。如何驗證到這步就夠了呢？看原始碼是一種方案。如果是Metal，用Xcode Capture GPU Frame也是一種方案。如何對Cargo專案進行Xcode Capture GPU Frame？參考我另一個文件：Xcode External Build System 失敗的 Capture GPU Frame 經歷、解決方案與覆盤。

// ... lots of previous stuff
queue.submit(submission, Some(&mut frame_fence));
device.wait_for_fence(&frame_fence, !0);
複製程式碼

紋理依賴渲染

比如，DrawCall 1輸出紋理為 DrawCall 2的一個輸入紋理，DrawCall 2輸出紋理為 DrawCall 3的一個輸入紋理，這類場景在直播、短視訊業務中很常見。

// **************** 準備 RenderPass ***************
// 建立新 RenderPass 前先釋放老的 RenderPass
vkDestroyRenderPass(vkDevice, mVkRenderPass, nullptr);

VkSubpassDescription subpass;
subpass.pipelineBindPoint       = mVkPipelineBindPoint; // 初始化為 VK_PIPELINE_BIND_POINT_GRAPHICS
subpass.flags                   = 0;
subpass.colorAttachmentCount    = colorFormat        != VK_FORMAT_UNDEFINED ? 1             : 0;
subpass.pColorAttachments       = colorFormat        != VK_FORMAT_UNDEFINED ? &color        : nullptr;
subpass.pDepthStencilAttachment = depthstencilFormat != VK_FORMAT_UNDEFINED ? &depthstencil : nullptr;
subpass.pResolveAttachments     = nullptr;
subpass.inputAttachmentCount    = 0;
subpass.pInputAttachments       = nullptr;
subpass.preserveAttachmentCount = 0;
subpass.pPreserveAttachments    = nullptr;

VkRenderPassCreateInfo info;
info.sType            = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO;
info.pNext            = nullptr;
info.flags            = 0;
info.attachmentCount  = static_cast<uint32_t>(attachments.size());
info.pAttachments     = attachments.data();
info.subpassCount     = 1;
info.pSubpasses       = &subpass;
info.dependencyCount  = 0;
info.pDependencies    = nullptr;

VkResult err = vkCreateRenderPass(vkDevice, &info, nullptr, &mVkRenderPass);
// 錯誤處理 (err != VK_ERROR_OUT_OF_HOST_MEMORY && err != VK_ERROR_OUT_OF_DEVICE_MEMORY);

// **************** 準備 Framebuffer ***************
// 建立新 Framebuffer 前先釋放老的 Framebuffer
vkDestroyFramebuffer(vkDevice, mVkFramebuffer, nullptr);

VkFramebufferCreateInfo info;
info.sType           = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO;
info.pNext           = nullptr;
info.flags           = 0;
info.renderPass      = *renderpass;
info.width           = width;
info.height          = height;
info.layers          = 1;
info.attachmentCount = static_cast<uint32_t>(imageViews->size());
info.pAttachments    = imageViews->data();
VkResult err = vkCreateFramebuffer(vkDevice, &info, nullptr, &mVkFramebuffer);
// 錯誤處理 (err != VK_ERROR_OUT_OF_HOST_MEMORY && err != VK_ERROR_OUT_OF_DEVICE_MEMORY);

// **************** 開始 CommandBuffer ***************
VkCommandBufferBeginInfo info;
info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
info.pNext = nullptr;
info.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; // glDraw* 只使用一次？
info.pInheritanceInfo = nullptr;

VkResult err = vkBeginCommandBuffer(mVkCommandBuffers.commandBuffer[mActiveCmdBuffer], &info);
// 錯誤處理 (err != VK_SUCCESS)
mVkCommandBuffers.commandBufferState[mActiveCmdBuffer] = CMD_BUFFER_RECORDING_STATE;

// **************** 開始 RenderPass ***************
VkRenderPassBeginInfo info;
info.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO;
info.pNext = nullptr;
info.framebuffer = *framebuffer;
info.renderPass = mVkRenderPass;
info.renderArea = mVkRenderArea;
info.clearValueCount = 2;
info.pClearValues = mVkClearValues;

const VkSubpassContents subpassContents = VK_SUBPASS_CONTENTS_SECONDARY_COMMAND_BUFFERS;
vkCmdBeginRenderPass(*activeCmdBuffer, &info, subpassContents);     


// **************** 分配 Secondary CommandBuffers ***************
VkCommandBuffer *commandBuffers = new VkCommandBuffer;
VkCommandBufferAllocateInfo cmdAllocInfo;
cmdAllocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
cmdAllocInfo.pNext = nullptr;
cmdAllocInfo.commandPool = mVkCmdPool;
cmdAllocInfo.level = VK_COMMAND_BUFFER_LEVEL_SECONDARY;
cmdAllocInfo.commandBufferCount = numOfBuffers; /// 值為 1

VkResult err = vkAllocateCommandBuffers(vkDevice, &cmdAllocInfo, commandBuffers);
// 錯誤處理 (err != VK_SUCCESS)
      
// **************** 開始 Secondary CommandBuffers ***************    
VkCommandBufferInheritanceInfo inheritanceInfo;
inheritanceInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_INHERITANCE_INFO;
inheritanceInfo.pNext = nullptr;
inheritanceInfo.renderPass = renderPass;
inheritanceInfo.subpass = 0;
inheritanceInfo.framebuffer = framebuffer;
inheritanceInfo.occlusionQueryEnable = VK_FALSE;
inheritanceInfo.queryFlags = 0;
inheritanceInfo.pipelineStatistics = 0;

VkCommandBufferBeginInfo cmdBeginInfo;
cmdBeginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
cmdBeginInfo.pNext = nullptr;
cmdBeginInfo.flags = VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT;
cmdBeginInfo.pInheritanceInfo = &inheritanceInfo;

VkResult err = vkBeginCommandBuffer(*cmdBuffer, &cmdBeginInfo);
// 錯誤處理 (err != VK_SUCCESS)
複製程式碼

HAL相關資料結構定義

FrameSync定義

/// Synchronization primitives which will be signalled once a frame got retrieved.
///
/// The semaphore or fence _must_ be unsignalled.
pub enum FrameSync<'a, B: Backend> {
    /// Semaphore used for synchronization.
    ///
    /// Will be signaled once the frame backbuffer is available.
    Semaphore(&'a B::Semaphore),

    /// Fence used for synchronization.
    ///
    /// Will be signaled once the frame backbuffer is available.
    Fence(&'a B::Fence),
}
複製程式碼

CommandBuffer（關鍵資料結構）

/// A strongly-typed command buffer that will only implement methods that are valid for the operations
/// it supports.
pub struct CommandBuffer<'a, B: Backend, C, S: Shot = OneShot, L: Level = Primary> {
    pub(crate) raw: &'a mut B::CommandBuffer,
    pub(crate) _marker: PhantomData<(C, S, L)>
}
複製程式碼

Submit

/// Thread-safe finished command buffer for submission.
pub struct Submit<B: Backend, C, S, L>(pub(crate) B::CommandBuffer, pub(crate) PhantomData<(C, S, L)>);
impl<B: Backend, C, S, L> Submit<B, C, S, L> {
    fn new(buffer: B::CommandBuffer) -> Self {
        Submit(buffer, PhantomData)
    }
}
unsafe impl<B: Backend, C, S, L> Send for Submit<B, C, S, L> {}
複製程式碼

Submission

/// Submission information for a command queue, generic over a particular
/// backend and a particular queue type.
pub struct Submission<'a, B: Backend, C> {
    cmd_buffers: SmallVec<[Cow<'a, B::CommandBuffer>; 16]>,
    wait_semaphores: SmallVec<[(&'a B::Semaphore, pso::PipelineStage); 16]>,
    signal_semaphores: SmallVec<[&'a B::Semaphore; 16]>,
    marker: PhantomData<C>,
}

/////////////////////////////// submit介面 /////////////////////////////////
/// Append a new list of finished command buffers to this submission.
///
/// All submits for this call must be of the same type.
/// Submission will be automatically promoted to to the minimum required capability
/// to hold all passed submits.
pub fn submit<I, K>(mut self, submits: I) -> Submission<'a, B, <(C, K) as Upper>::Result>
where
    I: IntoIterator,
    I::Item: Submittable<'a, B, K, Primary>,
    (C, K): Upper
{
    self.cmd_buffers.extend(submits.into_iter().map(
        |s| { unsafe { s.into_buffer() } }
    ));
    Submission {
        cmd_buffers: self.cmd_buffers,
        wait_semaphores: self.wait_semaphores,
        signal_semaphores: self.signal_semaphores,
        marker: PhantomData,
    }
}
複製程式碼

Vulkan與Metal的CommandBuffer複用與效能等問題討論

根據實踐，持續更新。

CommandBuffer重用

Metal的CommandBuffer一旦Commit到Queue，則不能再次使用。Vulkan可多次提交。

After a command buffer has been committed for execution, the only valid operations on the command buffer are to wait for it to be scheduled or completed (using synchronous calls or handler blocks) and to check the status of the command buffer execution. When used, scheduled and completed handlers are blocks that are invoked in execution order. These handlers should perform quickly; if expensive or blocking work needs to be scheduled, defer that work to another thread.

In a multithreaded app, it’s advisable to break your overall task into subtasks that can be encoded separately. Create a command buffer for each chunk of work, then call the enqueue() method on these command buffer objects to establish the order of execution. Fill each buffer object (using multiple threads) and commit them. The command queue automatically schedules and executes these command buffers as they become available.

developer.apple.com/documentati…

提交到佇列的函式名區別

提交CommandBuffer到Queue，Metal和Vulkan用了不同的單詞。Metal = commit()，Vulkan = submit()。

Rust圖形庫gfx-hal 逐幀渲染流程介紹
2019-03-04
Rust
GCanvas 渲染引擎介紹
2017-07-31
GCCanvas
Rust 屬性介紹
2020-03-03
Rust
Rust 註釋介紹
2021-10-08
Rust
Godot的Vulkan渲染器正在成型
2019-11-09
Go
CSS3 animation逐幀動畫
2018-11-22
CSSS3動畫
Chromimum Android渲染介紹
2017-03-14
Android
rustyline 介紹
2020-07-10
Rust
CSS3動畫之逐幀動畫
2019-04-02
CSSS3動畫
css3實現逐幀動畫
2018-06-29
CSSS3動畫
css3 實現逐幀動畫
2016-05-29
CSSS3動畫
以OpenGL/ES視角介紹gfx-hal(Vulkan) Framebuffer介面使用
2019-02-26
以OpenGL/ES視角介紹gfx-hal(Vulkan) Texture介面使用
2018-11-28
Scrapy 框架介紹之 Puppeteer 渲染
2020-05-27
框架
瀏覽器將html程式碼渲染成頁面流程簡單介紹
2017-04-09
瀏覽器HTML
Rust 資料型別介紹
2021-10-09
Rust資料型別
幀動畫記憶體OOM？不存在的！—— SurfaceView逐幀解析
2019-05-03
動畫記憶體OOMView
簡單介紹Rust中的workspace
2022-04-11
Rust
CSS技巧：逐幀動畫抖動解決方案
2017-08-16
CSS動畫
Android動畫效果之Frame Animation（逐幀動畫）
2016-08-04
Android動畫
渲染樹與css解析詳細介紹
2018-08-06
CSS
HTTPS流程簡單介紹
2016-11-10
HTTP
「譯」 MotionLayout 介紹 (Part IV) 深入理解關鍵幀
2018-12-25
005 Rust 非同步程式設計，Pin 介紹
2020-06-19
Rust非同步程式設計
Nginx 架構——【核心流程+模組介紹】
2019-03-25
Nginx架構
mysqldump執行流程步驟介紹
2017-03-28
MySql
Mysqldump備份的基本流程介紹
2017-03-24
MySql
006 Rust 非同步程式設計，Stream 介紹
2020-07-01
Rust非同步程式設計
013 Rust 網路程式設計，SMTP 介紹
2020-05-11
Rust程式設計
005 Rust 網路程式設計，ipnet 介紹
2020-04-25
Rust程式設計
017 Rust 網路程式設計，TFTP 介紹
2020-05-19
Rust程式設計FTP
SAP的月結相關流程介紹
2007-09-15
Flutter渲染流程解析
2020-05-28
Flutter
Rust 流程控制
2020-12-19
Rust
瀏覽器渲染頁面過程簡單介紹
2017-04-14
瀏覽器
006 Rust 網路程式設計，mio 庫介紹
2020-04-25
Rust程式設計
015 Rust 網路程式設計，FTP 介紹
2020-05-17
Rust程式設計FTP
流程編排、如此簡單-通用流程編排元件JDEasyFlow介紹
2022-11-25
元件