[ARKit]4-著色器Shader的應用

蘋果API搬運工發表於2018-02-23

說明

本文程式碼地址

ARKit系列文章目錄

學習ARKit前,需要先學習SceneKit,參考SceneKit系列文章目錄

更多iOS相關知識檢視github上WeekWeekUpProject

上一篇我們已經初步示範了:如何讀取著色器String,並通過shaderModifiers載入.
Shader型別SCNShaderModifierEntryPoint有geometry,surface,lightingModel,fragment.

使用示例

我們以surface型別為例.swift
如何給著色器傳參呢??直接使用KVC...

private func setupShader() {
    guard let path = Bundle.main.path(forResource: "skin", ofType: "shaderModifier", inDirectory: "art.scnassets"),
    let shader = try? String(contentsOfFile: path, encoding: String.Encoding.utf8) else {
        return
    }
    
    skin.shaderModifiers = [SCNShaderModifierEntryPoint.surface: shader]
    
    skin.setValue(Double(0), forKey: "blendFactor")
    skin.setValue(NSValue(scnVector3: SCNVector3Zero), forKey: "skinColorFromEnvironment")
    
    let sparseTexture = SCNMaterialProperty(contents: UIImage(named: "art.scnassets/textures/chameleon_DIFFUSE_BASE.png")!)
    skin.setValue(sparseTexture, forKey: "sparseTexture")
}

複製程式碼

OC

    NSURL *url = [[NSBundle mainBundle] URLForResource:@"skin" withExtension:@"shaderModifier"];
    NSError *error;
    NSString *shaderSource = [[NSString alloc] initWithContentsOfURL:url
                                                            encoding:NSUTF8StringEncoding
                                                               error:&error];
    if (!shaderSource) {
        // Handle the error
        NSLog(@"Failed to load shader source code, with error: %@", [error localizedDescription]);
    } else {
        skin.shaderModifiers = @{ SCNShaderModifierEntryPointSurface : shader };
    }

    [skin setValue:@(0) forKey:@"blendFactor"];
    [skin setValue:[NSValue valueWithSCNVector3:SCNVector3Zero] forKey:@"skinColorFromEnvironment"];
    SCNMaterialProperty *sparseTexture = [SCNMaterialProperty materialPropertyWithContents:[NSImage imageNamed:@"art.scnassets/textures/chameleon_DIFFUSE_BASE.png"]];
    [skin setValue:sparseTexture forKey:@"sparseTexture"];
複製程式碼

shader型別詳解

  • SCNShaderModifierEntryPointGeometry : 用來處理幾何體形變.

輸入為結構體geometry:

struct SCNShaderGeometry {
   float4 position;
   float3 normal;
   float4 tangent;
   float4 color;
   float2 texcoords[kSCNTexcoordCount];
} _geometry;

Access: ReadWrite 訪問許可權:讀寫
Stages: Vertex shader only 只限於頂點著色器中
複製程式碼

kSCNTexcoordCount是個整型常量,表示用到的頂點座標數.
其中的向量及座標(position, normal and tangent)為模型空間.
著色器示例,正弦變形:

GLSL
 uniform float Amplitude = 0.1;

 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time);

Metal Shading Language
 #pragma arguments
 float Amplitude;
 
 #pragma body
 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(scn_frame.time);
複製程式碼
  • SCNShaderModifierEntryPointSurface : 用來在光照應用前處理材質.

輸入為結構體surface:


struct SCNShaderSurface {
       float3 view;                     // Direction from the point on the surface toward the camera (V)
       float3 position;                 // Position of the fragment
       float3 normal;                   // Normal of the fragment (N)
       float3 geometryNormal;           // Geometric normal of the fragment (normal map is ignored)
       float3 tangent;                  // Tangent of the fragment
       float3 bitangent;                // Bitangent of the fragment
       float4 ambient;                  // Ambient property of the fragment
       float2 ambientTexcoord;          // Ambient texture coordinates
       float4 diffuse;                  // Diffuse property of the fragment. Alpha contains the opacity.
       float2 diffuseTexcoord;          // Diffuse texture coordinates
       float4 specular;                 // Specular property of the fragment
       float2 specularTexcoord;         // Specular texture coordinates
       float4 emission;                 // Emission property of the fragment
       float2 emissionTexcoord;         // Emission texture coordinates
       float4 multiply;                 // Multiply property of the fragment
       float2 multiplyTexcoord;         // Multiply texture coordinates
       float4 transparent;              // Transparent property of the fragment
       float2 transparentTexcoord;      // Transparent texture coordinates
       float4 reflective;               // Reflective property of the fragment
       float  metalness;                // Metalness property of the fragment
       float2 metalnessTexcoord;        // Metalness texture coordinates
       float  roughness;                // Roughness property of the fragment
       float2 roughnessTexcoord;        // Roughness texture coordinates
       float4 selfIllumination;         // Self Illumination property of the fragment. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emission` in previous versions.
       float2 selfIlluminationTexcoord; // Self Illumination texture coordinates. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emissionTexcoord` in previous versions.
       float  ambientOcclusion;         // Ambient Occlusion property of the fragment. Available macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiply` in previous versions.
       float2 ambientOcclusionTexcoord; // Ambient Occlusion texture coordinates. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiplyTexcoord` in previous versions.
       float  shininess;                // Shininess property of the fragment
       float  fresnel;                  // Fresnel property of the fragment
} _surface;

Access: ReadWrite 訪問許可權:讀寫
Stages: Fragment shader only 只限於片段著色器中
複製程式碼

其中的向量及座標為檢視空間.
著色器示例,產生黑白條紋:

GLSL
 uniform float Scale = 12.0;
 uniform float Width = 0.25;
 uniform float Blend = 0.3;

 vec2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0, 1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(vec4(1.0), vec4(vec3(0.0),1.0), f1);

Metal Shading Language
 #pragma arguments
 float Scale;
 float Width;
 float Blend;

 #pragma body
 float2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0, 1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(float4(1.0), float4(float3(0.0),1.0), f1);
複製程式碼
  • SCNShaderModifierEntryPointLightingModel : 提供自定義的光照方程.

輸入有多個,SCNShaderModifierEntryPointSurface中的所有結構體,結構體lightingContribution,結構體light:

All the structures available from the SCNShaderModifierEntryPointSurface entry point 
SCNShaderModifierEntryPointSurface中的所有結構體
Access: ReadOnly 訪問許可權:只讀
Stages: Vertex shader and fragment shader 頂點著色器及片段著色器中

struct SCNShaderLightingContribution {
       float3 ambient;
       float3 diffuse;
       float3 specular;
} _lightingContribution;

Access: ReadWrite 訪問許可權:讀寫
Stages: Vertex shader and fragment shader 頂點著色器及片段著色器中

struct SCNShaderLight {
       float4 intensity;
       float3 direction; // Direction from the point on the surface toward the light (L)
} _light;

Access: ReadOnly 訪問許可權:只讀
Stages: Vertex shader and fragment shader 頂點著色器及片段著色器中
複製程式碼

著色器示例,漫反射光照:

GLSL
uniform float WrapFactor = 0.5;

float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
vec3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

Metal Shading Language
#pragma arguments
float WrapFactor;

#pragma body
float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
float3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

複製程式碼
  • SCNShaderModifierEntryPointFragment : 用來最後修改顏色.

輸入為SCNShaderModifierEntryPointSurface中的結構體,及結構體output:

All the structures available from the SCNShaderModifierEntryPointSurface entry point
SCNShaderModifierEntryPointSurface中的所有結構體
Access: ReadOnly 訪問許可權:只讀
Stages: Fragment shader only 只限於片段著色器中

struct SCNShaderOutput {
       float4 color;
} _output;

Access: ReadWrite 訪問許可權:讀寫
Stages: Fragment shader only 只限於片段著色器中
複製程式碼

著色器示例,反轉最終的顏色:

GLSL
_output.color.rgb = vec3(1.0) - _output.color.rgb;

Metal Shading Language
_output.color.rgb = 1.0 - _output.color.rgb;
複製程式碼

著色器格式與引數

shaderModifiers相關說明:

// Custom GLSL uniforms declarations are of the form: [uniform type uniformName [= defaultValue]]
// 自定義GLSL uniforms宣告格式:[uniform 型別 全域性變數名 [= 預設值]]
uniform float myGrayAmount = 3.0;

// Custom Metal uniforms declarations require a #pragma and are of the form: [type name]
// 自定義Metal uniforms宣告格式要求#pragma,格式:[型別 全域性變數名]
#pragma arguments
float myGrayAmount;

// In Metal, you can also transfer varying values from the vertex shader (geometry modifier) to the fragment shader (surface/fragment modifier)
// 在Metal中,你將變數值從頂點著色器(geometry modifier)傳遞到片段著色器(surface/fragment modifier)中
// In one (or both) of the modifier, declare the varying values
// 在一個(或全部)的modifier中,宣告變數值
#pragma varyings
half3 myVec;

// Output varying values in the geometry modifier
// 在geometry modifier中輸出變數值
out.myVec = _geometry.normal.xyz * 0.5h + 0.5h;

// And use them in the fragment modifier
// 在fragment modifier中使用這些值
_output.color.rgb = saturate(in.myVec);

// Optional global function definitions (for Metal: references to uniforms in global functions are not supported).
// 可選的全域性函式定義(Metal中不支援在全域性函式中對uniforms的引用)
float mySin(float t) {
       return sin(t);
}

[#pragma transparent | opaque]
[#pragma body]

// the shader modifier code snippet itself
// 
float3 myColor = myGrayAmount;
_output.color.rgb += myColor;
複製程式碼

#pragma body指令:當宣告函式不在著色器程式碼中時,必須使用這個指令.

#pragma transparent指令:強制使用公式_output.color.rgb + (1 - _output.color.a) * dst.rgb;來渲染混合模式;其中dst表示當前片段顏色,rgb分量必須是自左乘的.

#pragma opaque指令:強制渲染為不透明的.忽略片段的alpha分量.

SCNGeometry和SCNMaterial類是相容key-value編碼的類,就是說你可以用KVC來賦值,哪怕key myAmplitude在當前類中並沒有被宣告,仍然可以賦值.
當在shader modifier中宣告myAmplitude uniform後,SceneKit會監聽接收者的myAmplitude key.當該值改變時,SceneKit會給uniform繫結一個新的值.
普通用NSValue包裝的標題型別都是支援的.

對於Metal:

  • MTLBuffer也支援作為values.
  • 在Metal shader中宣告的複雜資料型別(結構體)也是支援的.
    • 你可以用NSData將它們設定為一個整體.
    • 或者你可以使用單獨的結構體成員,將成員名稱作為key,對應型別的成員值作為value.

自定義uniforms可以使用顯式動畫.
在宣告或繫結自定義uniforms時,可用到的對應型別如下:

GLSL Metal Shading Language Objective-C
int int NSNumber, NSInteger, int
float float NSNumber, CGFloat, float, double
vec2 float2 CGPoint
vec3 float3 SCNVector3
vec4 float4 SCNVector4
mat4, mat44 float4x4 SCNMatrix4
sampler2D texture2d SCNMaterialProperty
samplerCube texturecube SCNMaterialProperty (with a cube map)

下列字首是SceneKit預設保留的,在自定義變數命名中不能使用:
u_
a_
v_

SceneKit宣告的內建uniforms共有以下這些:

GLSL Metal Shading Language 說明
float u_time float scn_frame.time The current time, in seconds
vec2 u_inverseResolution float2 scn_frame.inverseResolution 1.0 / screen size
mat4 u_viewTransform float4x4 scn_frame.viewTransform See SCNViewTransform
mat4 u_inverseViewTransform float4x4 scn_frame.inverseViewTransform
mat4 u_projectionTransform float4x4 scn_frame.projectionTransform See SCNProjectionTransform
mat4 u_inverseProjectionTransform float4x4 scn_frame.inverseProjectionTransform
mat4 u_normalTransform float4x4 scn_node.normalTransform See SCNNormalTransform
mat4 u_modelTransform float4x4 scn_node.modelTransform See SCNModelTransform
mat4 u_inverseModelTransform float4x4 scn_node.inverseModelTransform
mat4 u_modelViewTransform float4x4 scn_node.modelViewTransform See SCNModelViewTransform
mat4 u_inverseModelViewTransform float4x4 scn_node.inverseModelViewTransform
mat4 u_modelViewProjectionTransform float4x4 scn_node.modelViewProjectionTransform See SCNModelViewProjectionTransform
mat4 u_inverseModelViewProjectionTransform float4x4 scn_node.inverseModelViewProjectionTransform
mat2x3 u_boundingBox; float2x3 scn_node.boundingBox The bounding box of the current geometry, in model space, u_boundingBox[0].xyz and u_boundingBox[1].xyz being respectively the minimum and maximum corner of the box.

著色器程式碼示例

首先,我們新建個AR專案,Xcode會自動生成預設的專案,顯示一個小飛機. 我們對介面稍加改動,新增幾個開關,用來控制shader.

1291519893732_.pic.jpg

然後實現setupShader()方法來載入四個shader,並在viewDidLoad()中呼叫.這裡我們對著色器稍做修改,新增一個效果因子factor來控制效果是否顯示:

private func setupShader() {
    
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    let skin = shipNode?.geometry?.firstMaterial;
    // 為了方便觀察混合效果,我在各個示例shader中新增了一個因子factor,分別設定為0.0和1.0可以控制效果的開啟和關閉;預設為0.0--關閉;
    let geometryShader = """
        uniform float Amplitude = 0.1;
        uniform float GeometryFactor = 0.0;

        _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time) * GeometryFactor;
        """
    
    let surfaceShader = """
        uniform float Scale = 12.0;
        uniform float Width = 0.25;
        uniform float Blend = 0.3;
        uniform float SurfaceFactor = 0.0;

        vec2 position = fract(_surface.diffuseTexcoord * Scale);
        float f1 = clamp(position.y / Blend, 0.0, 1.0);
        float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
        f1 = f1 * (1.0 - f2);
        f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
        _surface.diffuse = _surface.diffuse * (1-SurfaceFactor) + mix(vec4(1.0), vec4(vec3(0.0),1.0), f1) * SurfaceFactor;
        """
    
    let lightShader = """
        uniform float WrapFactor = 0.5;
        uniform float LightFactor = 0.0;

        float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
        _lightingContribution.diffuse += (dotProduct * _light.intensity.rgb) * LightFactor;

        vec3 halfVector = normalize(_light.direction + _surface.view);
        dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
        _lightingContribution.specular += (dotProduct * _light.intensity.rgb) * LightFactor;
        """
    
    let fragmentShader = """
        uniform float FragmentFactor = 0.0;

        _output.color.rgb = (vec3(1.0) - _output.color.rgb) * FragmentFactor + (1-FragmentFactor) * _output.color.rgb;
        """
    
    skin?.shaderModifiers = [SCNShaderModifierEntryPoint.geometry: geometryShader,
                             SCNShaderModifierEntryPoint.surface: surfaceShader,
                             SCNShaderModifierEntryPoint.lightingModel: lightShader,
                             SCNShaderModifierEntryPoint.fragment: fragmentShader
    ]
    
}
複製程式碼

最後,只要在開關改變時,用KVC設定對應shader的因子值就可以了:

@IBAction func switchChange(_ sender: UISwitch) {
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    let skin = shipNode?.geometry?.firstMaterial;
    let factor = sender.isOn ? 1.0 : 0.0
    switch sender.tag {
    case 10:
        skin?.setValue(Double(factor), forKey: "GeometryFactor")
    case 11:
        skin?.setValue(Double(factor), forKey: "SurfaceFactor")
    case 12:
        skin?.setValue(Double(factor), forKey: "LightFactor")
    case 13:
        skin?.setValue(Double(factor), forKey: "FragmentFactor")
    default:
        print("switch")
    }
}
複製程式碼

效果如圖:

1331519894613_.pic_hd.jpg

相關文章