# 前言
---
  在這個美即真理、全民娛樂的時代，可愛有趣的人臉貼紙在各大美顏軟體中得到了廣泛的應用，現在已經不僅侷限於相機美顏類軟體中，在社交、娛樂類的app中對人臉貼紙、AR貼紙的需求也非常廣泛。本文詳細介紹了整合華為HMS ML kit人臉識別實現2d貼紙的整合過程，在後面的文章中我們還會介紹3D貼紙的開發過程，歡迎大家關注哦~

# 場景
---
  在美顏相機、美圖app以及社交類app(如抖音、微博、微信)等需要對拍照，或者對照片進行處理的app都會構建自己特有的貼紙的需求。

# 開發前準備
---
## 在專案級gradle裡新增華為maven倉
  開啟AndroidStudio專案級build.gradle檔案

![在這裡插入圖片描述](https://img-blog.csdnimg.cn/2020081216172488.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDcwODI0MA==,size_16,color_FFFFFF,t_70#pic_center)
  增量新增如下maven地址：

```
buildscript {
    {        
       maven {url 'http://developer.huawei.com/repo/'}
   }    
}
allprojects {
   repositories {      
       maven { url 'http://developer.huawei.com/repo/'}
   }
}
```

## 在應用級的build.gradle裡面加上SDK依賴
![在這裡插入圖片描述](https://img-blog.csdnimg.cn/20200812163452934.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDcwODI0MA==,size_16,color_FFFFFF,t_70#pic_center)

```
// Face detection SDK.
implementation 'com.huawei.hms:ml-computer-vision-face:2.0.1.300'
// Face detection model.
implementation 'com.huawei.hms:ml-computer-vision-face-shape-point-model:2.0.1.300'
```

## 在AndroidManifest.xml檔案裡面申請相機、訪問網路和儲存許可權
```

<uses-feature android:name="android.hardware.camera"></uses-feature>
<uses-permission android:name="android.permission.CAMERA"></uses-permission>

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"></uses-permission>

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"></uses-permission>
```

# 程式碼開發關鍵步驟
---
## 設定人臉檢測器

```
MLFaceAnalyzerSetting detectorOptions;
detectorOptions = new MLFaceAnalyzerSetting.Factory()
       .setFeatureType(MLFaceAnalyzerSetting.TYPE_UNSUPPORT_FEATURES)
       .setShapeType(MLFaceAnalyzerSetting.TYPE_SHAPES)
       .allowTracing(MLFaceAnalyzerSetting.MODE_TRACING_FAST)
       .create();
detector = MLAnalyzerFactory.getInstance().getFaceAnalyzer(detectorOptions);
```

## 這裡我們透過相機回撥拿到相機幀資料,並透過呼叫人臉檢測器拿到人臉輪廓點後寫入FacePointEngine供貼紙濾鏡使用

```
@Override
public void onPreviewFrame(final byte[] imgData, final Camera camera) {
   int width = mPreviewWidth;
   int height = mPreviewHeight;

long startTime = System.currentTimeMillis();
   //設定前後攝方向一致
   if (isFrontCamera()){
       mOrientation = 0;
   }else {
       mOrientation = 2;
   }
   MLFrame.Property property =
           new MLFrame.Property.Creator()
                   .setFormatType(ImageFormat.NV21)
                   .setWidth(width)
                   .setHeight(height)
                   .setQuadrant(mOrientation)
                   .create();

ByteBuffer data = ByteBuffer.wrap(imgData);
   // 呼叫人臉檢測介面
   SparseArray<mlface> faces = detector.analyseFrame(MLFrame.fromByteBuffer(data,property));
   //判斷是否獲取到人臉資訊
   if(faces.size()>0){
       MLFace mLFace = faces.get(0);
       EGLFace EGLFace = FacePointEngine.getInstance().getOneFace(0);
       EGLFace.pitch = mLFace.getRotationAngleX();
       EGLFace.yaw = mLFace.getRotationAngleY();
       EGLFace.roll = mLFace.getRotationAngleZ() - 90;
       if (isFrontCamera())
           EGLFace.roll = -EGLFace.roll;
       if (EGLFace.vertexPoints == null) {
           EGLFace.vertexPoints = new PointF[131];
       }
       int index = 0;
       // 獲取一個人的輪廓點座標並轉化到openGL歸一化座標系下的浮點值
       for (MLFaceShape contour : mLFace.getFaceShapeList()) {
           if (contour == null) {
               continue;
           }
           List<mlposition> points = contour.getPoints();

for (int i = 0; i < points.size(); i++) {
               MLPosition point = points.get(i);
               float x = ( point.getY() / height) * 2 - 1;
               float y = ( point.getX() / width ) * 2 - 1;
               if (isFrontCamera())
                   x = -x;
               PointF Point = new PointF(x,y);
               EGLFace.vertexPoints[index] = Point;
               index++;
           }
       }
       // 插入人臉物件
       FacePointEngine.getInstance().putOneFace(0, EGLFace);
       // 設定人臉個數
       FacePointEngine.getInstance().setFaceSize(faces!= null ? faces.size() : 0);
   }else{
       FacePointEngine.getInstance().clearAll();
   }
   long endTime = System.currentTimeMillis();
   Log.d("TAG","Face detect time: " + String.valueOf(endTime - startTime));
}
```

ML kit介面返回的人臉輪廓點情況如圖所示：
![在這裡插入圖片描述](https://img-blog.csdnimg.cn/20200812164136681.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDcwODI0MA==,size_16,color_FFFFFF,t_70#pic_center)

## 介紹如何設計貼紙,首先看一下貼紙數JSON資料定義

```
public class FaceStickerJson {

public int[] centerIndexList;   // 中心座標索引列表，有可能是多個關鍵點計算中心點
   public float offsetX;           // 相對於貼紙中心座標的x軸偏移畫素
   public float offsetY;           // 相對於貼紙中心座標的y軸偏移畫素
   public float baseScale;         // 貼紙基準縮放倍數
   public int startIndex;          // 人臉起始索引，用於計算人臉的寬度
   public int endIndex;            // 人臉結束索引，用於計算人臉的寬度
   public int width;               // 貼紙寬度
   public int height;              // 貼紙高度
   public int frames;              // 貼紙幀數
   public int action;              // 動作，0表示預設顯示，這裡用來處理貼紙動作等
   public String stickerName;      // 貼紙名稱，用於標記貼紙所在資料夾以及png檔案的
   public int duration;            // 貼紙幀顯示間隔
   public boolean stickerLooping;  // 貼紙是否迴圈渲染
   public int maxCount;            // 最大貼紙渲染次數
...
}
```

## 我們製作貓耳貼紙JSON檔案,透過人臉索引找到眉心84號點和鼻尖85號點分別貼上耳朵和鼻子,然後把它和圖片都放在assets目錄下

```
{
   "stickerList": [{
       "type": "sticker",
       "centerIndexList": [84],
       "offsetX": 0.0,
       "offsetY": 0.0,
       "baseScale": 1.3024,
       "startIndex": 11,
       "endIndex": 28,
       "width": 495,
       "height": 120,
       "frames": 2,
       "action": 0,
       "stickerName": "nose",
       "duration": 100,
       "stickerLooping": 1,
       "maxcount": 5
   }, {
       "type": "sticker",
       "centerIndexList": [83],
       "offsetX": 0.0,
       "offsetY": -1.1834,
       "baseScale": 1.3453,
       "startIndex": 11,
       "endIndex": 28,
       "width": 454,
       "height": 150,
       "frames": 2,
       "action": 0,
       "stickerName": "ear",
       "duration": 100,
       "stickerLooping": 1,
       "maxcount": 5
   }]
}
```

## 這裡渲染貼紙紋理我們使用GLSurfaceView,使用起來比TextureView簡單, 首先在onSurfaceChanged例項化貼紙濾鏡,傳入貼紙路徑並開啟相機

```
@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {

GLES30.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
   mTextures = new int[1];
   mTextures[0] = OpenGLUtils.createOESTexture();
   mSurfaceTexture = new SurfaceTexture(mTextures[0]);
   mSurfaceTexture.setOnFrameAvailableListener(this);

//將samplerExternalOES 輸入到紋理中
   cameraFilter = new CameraFilter(this.context);

//設定assets目錄下人臉貼紙路徑
   String folderPath ="cat";
   stickerFilter = new FaceStickerFilter(this.context,folderPath);

//建立螢幕濾鏡物件
   screenFilter = new BaseFilter(this.context);

facePointsFilter = new FacePointsFilter(this.context);
   mEGLCamera.openCamera();
}
```

## 然後在onSurfaceChanged初始化貼紙濾鏡

```
@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
   Log.d(TAG, "onSurfaceChanged. width: " + width + ", height: " + height);
   int previewWidth = mEGLCamera.getPreviewWidth();
   int previewHeight = mEGLCamera.getPreviewHeight();
   if (width > height) {
       setAspectRatio(previewWidth, previewHeight);
   } else {
       setAspectRatio(previewHeight, previewWidth);
   }
   // 設定畫面的大小,建立FrameBuffer,設定顯示尺寸
   cameraFilter.onInputSizeChanged(previewWidth, previewHeight);
   cameraFilter.initFrameBuffer(previewWidth, previewHeight);
   cameraFilter.onDisplaySizeChanged(width, height);

stickerFilter.onInputSizeChanged(previewHeight, previewWidth);
   stickerFilter.initFrameBuffer(previewHeight, previewWidth);
   stickerFilter.onDisplaySizeChanged(width, height);

screenFilter.onInputSizeChanged(previewWidth, previewHeight);
   screenFilter.initFrameBuffer(previewWidth, previewHeight);
   screenFilter.onDisplaySizeChanged(width, height);

facePointsFilter.onInputSizeChanged(previewHeight, previewWidth);
   facePointsFilter.onDisplaySizeChanged(width, height);
   mEGLCamera.startPreview(mSurfaceTexture);
}
```

## 最後透過onDrawFrame把貼紙繪製到螢幕

```
@Override
public void onDrawFrame(GL10 gl) {
   int textureId;
   // 清除螢幕和深度快取
   GLES30.glClear(GLES30.GL_COLOR_BUFFER_BIT | GLES30.GL_DEPTH_BUFFER_BIT);
   //更新獲取一張圖
   mSurfaceTexture.updateTexImage();
   //獲取SurfaceTexture轉化矩陣
   mSurfaceTexture.getTransformMatrix(mMatrix);
   //設定相機顯示轉化矩陣
   cameraFilter.setTextureTransformMatrix(mMatrix);

//繪製相機紋理
   textureId = cameraFilter.drawFrameBuffer(mTextures[0],mVertexBuffer,mTextureBuffer);
   //繪製貼紙紋理
   textureId = stickerFilter.drawFrameBuffer(textureId,mVertexBuffer,mTextureBuffer);
   //繪製到螢幕
   screenFilter.drawFrame(textureId , mDisplayVertexBuffer, mDisplayTextureBuffer);
   if(drawFacePoints){
       facePointsFilter.drawFrame(textureId, mDisplayVertexBuffer, mDisplayTextureBuffer);
   }
}
```
這樣我們的貼紙就畫到人臉上了.

# Demo效果
---
![在這裡插入圖片描述](https://img-blog.csdnimg.cn/20200812164703214.gif#pic_center)

---
原文連結：[https://developer.huawei.com/consumer/cn/forum/topicview?tid=0203324526929930082&fid=18](https://developer.huawei.com/consumer/cn/forum/topicview?tid=0203324526929930082&fid=18)

原作者：旭小夜</mlposition></mlface>

超簡單整合HMS ML Kit 人臉檢測實現可愛貼紙

相關文章