AI賦能音樂創作,人人都是音影片創作者

HMSCore 發表於 2022-11-24

華為HMS Core音訊編輯服務(Audio Editor Kit)依託自身AI技術的研發優勢,上線全新的歌聲合成音色及伴奏,給音影片創作者提供更多的創作可能。在短影片場景中,使用者自定義歌詞的歌聲結合影片讓使用者感受到身臨其境,自由表達自己的情緒;在虛擬偶像場景中,歌聲合成功能賦予虛擬歌手們演唱風格各異的歌曲,帶來創意無限。

HMS Core音訊編輯服務歌聲合成的AI Singer模型能力透過字級別輸入歌詞進行音素轉換,就可以為使用者創作音樂,也可預置曲目合成歌聲。透過自研音高模型,讓音高曲線在保持輸入曲譜的音高精準度的同時改善自然度,更接近人的真實演唱。使用最新的生成式模型,帶來更好的音色還原度、建模更多的演唱細節,同時高畫質聲碼器能夠真實還原48k高畫質音質。

另外,使用者透過自由調整顫音、滑音、呼吸音等功能,可根據情感需求調整歌聲演唱技巧。當前歌聲合成服務已開放了情流行女聲、國風女聲和民謠男聲音色,未來會持續更新更多音色。

華為HMS Core音訊編輯服務(Audio Editor Kit)讓機器“演唱”出真實度的歌聲,僅需簡單的整合獲得,以下是開發者應用整合音訊編輯服務歌聲合成能力的具體步驟。

開發步驟

1. 開發準備

1.1註冊成為開發者

在開發應用前需要在華為開發者聯盟網站上註冊成為開發者並完成實名認證,具體方法請參見帳號註冊認證。

1.2建立專案及應用

參見建立專案,然後在專案下建立應用完成應用的建立,特殊配置如下:

選擇平臺:選擇“Web”。

1.3開啟相關服務

使用Audio Editor Kit服務需要您在AppGallery Connect上開啟Audio Editor Kit服務開關,具體操作步驟請參見開啟服務開關。

2.歌聲合成功能整合

2.1同步介面(流式)

2.1.1獲取access_token鑑權資訊

使用開發者聯盟介面獲得的客戶端ID以及對應金鑰,傳送HTTPS POST請求,獲取查詢access_token。獲取方式請參見客戶端模式(Client Credentials)。

2.1.2呼叫同步介面(流式)

透過以上步驟獲取的access_token資訊,傳送HTTPS POST呼叫同步介面(流式)。

示例程式碼(Java)如下所示:

其中requestUrl = "https://audioeditor-api-drcn.cloud.huawei.com/v1/audioeditor/gateway/ai/ttsing/sync"。

請點選下載MusicXML檔案,並上傳:

     /**
     * 呼叫同步介面(流式)
     * @throws Exception IO異常
     */
    private static void syncTask() throws Exception {
        // 設定請求header
        PostMethod postMethod = new PostMethod(requestUrl);
        // 設定文字型別(String),例:"application/json;charset=utf-8"
        postMethod.setRequestHeader("Content-Type", contentType);
        // 設定請求ID(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("X-Request-ID", requestId);
        // 設定App包名(String),例:"com.huawei.demo"
        postMethod.setRequestHeader("X-Package-Name", pacageName);
        // 設定App所在國家(String),例:"cn"
        postMethod.setRequestHeader("X-Country-Code", countryCode);
        // 設定App標識(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("HMS-APPLICATION-ID", applicationId);
        // 設定證照指紋(String),例:"xxxxxxxxxxxxxxx"
        postMethod.setRequestHeader("certFingerprint", certFingerprint);
        // 設定動態獲取的AccessToken(String)
        postMethod.setRequestHeader("Authorization","Bearer " + accessToken);
        // 設定請求body
        Map<String, Object> bodyMap = new HashMap<>();
        Map<String, Object> dataMap = new HashMap<>();
        Map<String, Object> configMap = new HashMap<>();
        // filePath是MusicXML檔案路徑(含檔名、字尾)
        String lyricFilePath = "filePath";
        dataMap.put("lyric", FileUtils.readFileToString(new File(lyricFilePath), "UTF-8"));
        dataMap.put("language", "chinese");
        configMap.put("type", 1);
        configMap.put("outputEncoderFormat", 0);
        configMap.put("wordDurationForceAlign", "false");
        bodyMap.put("data", dataMap);
        bodyMap.put("config", configMap);
        RequestEntity requestEntity = new StringRequestEntity(JSONObject.toJSONString(bodyMap),"application/json" ,"UTF-8");
        postMethod.setRequestEntity(requestEntity);

        HttpClient httpClient = new HttpClient();
        int ret = httpClient.executeMethod(postMethod);
        if (ret == 200) {
            Header responseHeader = postMethod.getResponseHeader("content-type");
            if ("application/octet-stream".equals(responseHeader.getValue())) {
                InputStream rpsContent = postMethod.getResponseBodyAsStream();
                // filePath是要儲存檔案的路徑(含檔名、PCM檔案字尾)
                String filePath = "filePath";
                FileUtils.copyInputStreamToFile(rpsContent, new File(filePath));
            } else {
                String errorString = postMethod.getResponseBodyAsString();
                System.out.println(errorString);
            }
        } else {
            System.out.println("callApi failed: ret =" + ret + " rsp=" + postMethod.getResponseBodyAsString());
        }
    }

使用預置曲目輸入歌詞:

   /**
     * 呼叫同步介面(流式)
     * @throws Exception IO異常
     */
    private static void syncTask() throws Exception {
        
        // 設定請求header
        PostMethod postMethod = new PostMethod(requestUrl);
        // 設定文字型別(String),例:"application/json;charset=utf-8"
        postMethod.setRequestHeader("Content-Type", contentType);
        // 設定請求ID(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("X-Request-ID", requestId);
        // 設定App包名(String),例:"com.huawei.demo"
        postMethod.setRequestHeader("X-Package-Name", pacageName);
        // 設定App所在國家(String),例:"cn"
        postMethod.setRequestHeader("X-Country-Code", countryCode);
        // 設定App標識(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("HMS-APPLICATION-ID", applicationId);
        // 設定證照指紋(String),例:"xxxxxxxxxxxxxxx"
        postMethod.setRequestHeader("certFingerprint", certFingerprint);
        // 設定動態獲取的AccessToken(String)
        postMethod.setRequestHeader("Authorization","Bearer " + accessToken);
        // 設定請求body
        Map<String, Object> bodyMap = new HashMap<>();
        Map<String, Object> dataMap = new HashMap<>();
        Map<String, Object> configMap = new HashMap<>();
        String[] lyrics = {"跟隨心跳的節拍", "感受自由的暢快", "把煩惱通通拋開", "我們一起嗨", "調整呼吸的節拍", "保持最好的狀態", "奔向耀眼的未來", "哦康忙北北"};
        dataMap.put("lyrics", lyrics);
        dataMap.put("accompanimentId", "1");
        dataMap.put("isAutoFill", "false");
        dataMap.put("language", "chinese");
        configMap.put("type", 1);
        configMap.put("outputEncoderFormat", 0);
        configMap.put("wordDurationForceAlign", "false");
        bodyMap.put("data", dataMap);
        bodyMap.put("config", configMap);
        RequestEntity requestEntity = new StringRequestEntity(JSONObject.toJSONString(bodyMap),"application/json" ,"UTF-8");
        postMethod.setRequestEntity(requestEntity);

        HttpClient httpClient = new HttpClient();
        int ret = httpClient.executeMethod(postMethod);
        if (ret == 200) {
            Header responseHeader = postMethod.getResponseHeader("content-type");
            if ("application/octet-stream".equals(responseHeader.getValue())) {
                InputStream rpsContent = postMethod.getResponseBodyAsStream();
                // filePath是要儲存檔案的路徑(含檔名、PCM檔案字尾)
                String filePath = "filePath";
                FileUtils.copyInputStreamToFile(rpsContent, new File(filePath));
            } else {
                String errorString = postMethod.getResponseBodyAsString();
                System.out.println(errorString);
            }
        } else {
            System.out.println("callApi failed: ret =" + ret + " rsp=" + postMethod.getResponseBodyAsString());
        }
    }

注意:

上述程式碼中xxxxx對應的值請根據實際情況填寫,具體取值請參見同步介面(流式)

2.2非同步介面

2.2.1建立非同步任務

透過access_token資訊,傳送HTTPS POST建立歌聲合成非同步任務。

示例程式碼(Java)如下所示:

其中requestUrl = "https://audioeditor-api-drcn.cloud.huawei.com/v1/audioeditor/gateway/ai/ttsing/async/task/create"。

請點選下載MusicXML檔案,並上傳:

     /**
     * 呼叫建立非同步任務介面
     * @throws Exception IO異常
     */
    private static void creatAsyncTask() throws Exception {
        
        // 設定請求header
        PostMethod postMethod = new PostMethod(requestUrl);
        // 設定文字型別(String),例:"application/json;charset=utf-8"
        postMethod.setRequestHeader("Content-Type", contentType);
        // 設定請求ID(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("X-Request-ID", requestId);
        // 設定App包名(String),例:"com.huawei.demo"
        postMethod.setRequestHeader("X-Package-Name", pacageName);
        // 設定App所在國家(String),例:"cn"
        postMethod.setRequestHeader("X-Country-Code", countryCode);
        // 設定App標識(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("HMS-APPLICATION-ID", applicationId);
        // 設定證照指紋(String),例:"xxxxxxxxxxxxxxx"
        postMethod.setRequestHeader("certFingerprint", certFingerprint);
        // 設定動態獲取的AccessToken(String)
        postMethod.setRequestHeader("Authorization","Bearer " + accessToken);
        // 設定請求body
        Map<String, Object> bodyMap = new HashMap<>();
        Map<String, Object> dataMap = new HashMap<>();
        Map<String, Object> configMap = new HashMap<>();
        // filePath是MusicXML檔案路徑(含檔名、字尾)
        String lyricFilePath = "filePath";
        dataMap.put("lyric", FileUtils.readFileToString(new File(lyricFilePath), "UTF-8"));
        dataMap.put("language", "chinese");
        configMap.put("type", 1);
        configMap.put("outputEncoderFormat", 0);
        configMap.put("wordDurationForceAlign", "false");
        bodyMap.put("data", dataMap);
        bodyMap.put("config", configMap);
        RequestEntity requestEntity = new StringRequestEntity(JSONObject.toJSONString(bodyMap),"application/json" ,"UTF-8");
        postMethod.setRequestEntity(requestEntity);

        HttpClient httpClient = new HttpClient();
        int ret = httpClient.executeMethod(postMethod);
        String rpsContent = postMethod.getResponseBodyAsString();
        if (ret == 200) {
            System.out.println(rpsContent);
        } else {
            System.out.println("callApi failed: ret =" + ret + " rsp=" + rpsContent);
        }
    }

使用預置曲目輸入歌詞:

/**
     * 呼叫建立非同步任務介面
     * @throws Exception IO異常
     */
    private static void creatAsyncTask() throws Exception {
        
        // 設定請求header
        PostMethod postMethod = new PostMethod(requestUrl);
        // 設定文字型別(String),例:"application/json;charset=utf-8"
        postMethod.setRequestHeader("Content-Type", contentType);
        // 設定請求ID(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("X-Request-ID", requestId);
        // 設定App包名(String),例:"com.huawei.demo"
        postMethod.setRequestHeader("X-Package-Name", pacageName);
        // 設定App所在國家(String),例:"cn"
        postMethod.setRequestHeader("X-Country-Code", countryCode);
        // 設定App標識(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("HMS-APPLICATION-ID", applicationId);
        // 設定證照指紋(String),例:"xxxxxxxxxxxxxxx"
        postMethod.setRequestHeader("certFingerprint", certFingerprint);
        // 設定動態獲取的AccessToken(String)
        postMethod.setRequestHeader("Authorization","Bearer " + accessToken);
        // 設定請求body
        Map<String, Object> bodyMap = new HashMap<>();
        Map<String, Object> dataMap = new HashMap<>();
        Map<String, Object> configMap = new HashMap<>();
        String[] lyrics = {"跟隨心跳的節拍", "感受自由的暢快", "把煩惱通通拋開", "我們一起嗨", "調整呼吸的節拍", "保持最好的狀態", "奔向耀眼的未來", "哦康忙北北"};
        dataMap.put("lyrics", lyrics);
        dataMap.put("accompanimentId", "1");
        dataMap.put("isAutoFill", "false");
        dataMap.put("language", "chinese");
        configMap.put("type", 1);
        configMap.put("outputEncoderFormat", 0);
        configMap.put("wordDurationForceAlign", "false");
        bodyMap.put("data", dataMap);
        bodyMap.put("config", configMap);
        RequestEntity requestEntity = new StringRequestEntity(JSONObject.toJSONString(bodyMap),"application/json" ,"UTF-8");
        postMethod.setRequestEntity(requestEntity);

        HttpClient httpClient = new HttpClient();
        int ret = httpClient.executeMethod(postMethod);
        String rpsContent = postMethod.getResponseBodyAsString();
        if (ret == 200) {
            System.out.println(rpsContent);
        } else {
            System.out.println("callApi failed: ret =" + ret + " rsp=" + rpsContent);
        }
    }

注意:

上述程式碼中xxxxx對應的值請根據實際情況填寫,具體取值請參見建立非同步任務

2.2.2查詢非同步任務狀態

使用者建立非同步任務後,可以透過呼叫該介面,獲取任務處理狀態等資訊。任務處理完成後,會返回任務的下載地址,直接訪問該地址即可下載檔案。

透過access_token資訊,和建立非同步任務獲取到的taskId傳送HTTPS POST查詢歌聲合成非同步任務狀態。

示例程式碼(Java)如下所示:

其中requestUrl = "https://audioeditor-api-drcn.cloud.huawei.com/v1/audioeditor/gateway/ai/ttsing/async/task/status"。

  /**
     * 呼叫查詢非同步任務狀態介面
     * @param taskId 建立非同步任務獲取的taskId
     * @throws Exception IO異常
     */
    private static void queryAsyncTaskInfo(String taskId) throws Exception {
        
        // 設定請求header
        PostMethod postMethod = new PostMethod(requestUrl);
        // 設定文字型別(String),例:"application/json;charset=utf-8"
        postMethod.setRequestHeader("Content-Type", contentType);
        // 設定請求ID(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("X-Request-ID", requestId);
        // 設定App包名(String),例:"com.huawei.demo"
        postMethod.setRequestHeader("X-Package-Name", pacageName);
        // 設定App所在國家(String),例:"cn"
        postMethod.setRequestHeader("X-Country-Code", countryCode);
        // 設定App標識(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("HMS-APPLICATION-ID", applicationId);
        // 設定證照指紋(String),例:"xxxxxxxxxxxxxxx"
        postMethod.setRequestHeader("certFingerprint", certFingerprint);
        // 設定動態獲取的AccessToken(String)
        postMethod.setRequestHeader("Authorization","Bearer " + accessToken);
        // 設定請求body
        Map<String, Object> bodyMap = new HashMap<>();
        // taskId對應的值是建立非同步任務時返回的任務ID(taskId)
        bodyMap.put("taskId", taskId);
        RequestEntity requestEntity = new StringRequestEntity(JSONObject.toJSONString(bodyMap),"application/json" ,"UTF-8");
        postMethod.setRequestEntity(requestEntity);

        HttpClient httpClient = new HttpClient();
        int ret = httpClient.executeMethod(postMethod);
        String rpsContent = postMethod.getResponseBodyAsString();
        if (ret == 200) {
            System.out.println(rpsContent);
        } else {
            System.out.println("callApi failed: ret =" + ret + " rsp=" + rpsContent);
        }
    }

注意:

上述程式碼中xxxxx對應的值請根據實際情況填寫,具體取值請參見查詢非同步任務狀態

2.2.3取消非同步任務

使用者建立歌聲合成非同步任務後,可以透過呼叫此介面,取消指定非同步任務並刪除相應任務資料。

透過access_token資訊和建立非同步任務獲取到的taskId,傳送HTTPS POST取消非同步任務。

示例程式碼(Java)如下所示:

其中requestUrl = "https://audioeditor-api-drcn.cloud.huawei.com/v1/audioeditor/gateway/ai/ttsing/async/task/cancel"。

  /**
     * 呼叫取消非同步任務介面
     * @param taskId 建立非同步任務獲取的taskId
     * @throws Exception IO異常
     */
    private static void cancelAsyncTask(String taskId) throws Exception {
        
        // 設定請求header
        PostMethod postMethod = new PostMethod(requestUrl);
        // 設定文字型別(String),例:"application/json;charset=utf-8"
        postMethod.setRequestHeader("Content-Type", contentType);
        // 設定請求ID(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("X-Request-ID", requestId);
        // 設定App包名(String),例:"com.huawei.demo"
        postMethod.setRequestHeader("X-Package-Name", pacageName);
        // 設定App所在國家(String),例:"cn"
        postMethod.setRequestHeader("X-Country-Code", countryCode);
        // 設定App標識(String),例:"9af1aeda-531b-407a-80b4-65b40ef77bd6"
        postMethod.setRequestHeader("HMS-APPLICATION-ID", applicationId);
        // 設定證照指紋(String),例:"xxxxxxxxxxxxxxx"
        postMethod.setRequestHeader("certFingerprint", certFingerprint);
        // 設定動態獲取的AccessToken(String)
        postMethod.setRequestHeader("Authorization","Bearer " + accessToken);
        // 設定請求body
        Map<String, Object> bodyMap = new HashMap<>();
        // taskId對應的值是建立非同步任務時返回的任務ID(taskId)
        bodyMap.put("taskId", taskId);
        RequestEntity requestEntity = new StringRequestEntity(JSONObject.toJSONString(bodyMap),"application/json" ,"UTF-8");
        postMethod.setRequestEntity(requestEntity);

        HttpClient httpClient = new HttpClient();
        int ret = httpClient.executeMethod(postMethod);
        String rpsContent = postMethod.getResponseBodyAsString();
        if (ret == 200) {
            System.out.println(rpsContent);
        } else {
            System.out.println("callApi failed: ret =" + ret + " rsp=" + rpsContent);
        }
    }

注意:

上述程式碼中xxxxx對應的值請根據實際情況填寫,具體取值請參見取消非同步任務

除了歌聲合成能力,音訊編輯服務還提供音訊基礎剪輯、AI配音、音源分離、空間渲染、變聲降噪等音訊處理能力,更多資訊可以訪問官網獲得

瞭解更多詳情>>

訪問華為開發者聯盟官網
獲取開發指導文件
華為移動服務開源倉庫地址:GitHubGitee

關注我們,第一時間瞭解 HMS Core 最新技術資訊~