不能否認,微軟Azure在TTS(text-to-speech文字轉語音)這個人工智慧細分領域的影響力是統治級的,一如ChatGPT在NLP領域的隨心所欲,予取予求。君不見幾乎所有的抖音營銷號口播均採用微軟的語音合成技術,其影響力由此可見一斑,僅有的白璧微瑕之處就是價格略高,雖然國內也可以使用科大訊飛語音合成進行平替,但我們只想要最好的那一個,本次我們使用免費的開源庫edge-tts來實現文字轉語音操作,薅微軟edge的羊毛。
TTS文字轉語音基礎使用方式
首先安裝edge-tts庫:
pip3 install edge-tts
安裝成功後,直接在終端執行edge-tts命令:
edge-tts
顯示幫助選單即代表安裝成功:
➜ Downloads edge-tts
usage: edge-tts [-h] [-t TEXT] [-f FILE] [-v VOICE] [-l] [--rate RATE] [--volume VOLUME] [-O OVERLAPPING]
[--write-media WRITE_MEDIA] [--write-subtitles WRITE_SUBTITLES] [--proxy PROXY]
隨後輸入命令:
edge-tts --list-voices
該命令可以將Edge瀏覽器中,內建的語言角色列表列出來:
Name: af-ZA-AdriNeural
Gender: Female
Name: af-ZA-WillemNeural
Gender: Male
Name: am-ET-AmehaNeural
Gender: Male
Name: am-ET-MekdesNeural
Gender: Female
Name: ar-AE-FatimaNeural
Gender: Female
Name: ar-AE-HamdanNeural
Gender: Male
Name: ar-BH-AliNeural
Gender: Male
Name: ar-BH-LailaNeural
Gender: Female
Name: ar-DZ-AminaNeural
Gender: Female
Name: ar-DZ-IsmaelNeural
Gender: Male
Name: ar-EG-SalmaNeural
Gender: Female
Name: ar-EG-ShakirNeural
Gender: Male
Name: ar-IQ-BasselNeural
Gender: Male
Name: ar-IQ-RanaNeural
Gender: Female
Name: ar-JO-SanaNeural
Gender: Female
Name: ar-JO-TaimNeural
Gender: Male
Name: ar-KW-FahedNeural
Gender: Male
Name: ar-KW-NouraNeural
Gender: Female
Name: ar-LB-LaylaNeural
Gender: Female
Name: ar-LB-RamiNeural
Gender: Male
Name: ar-LY-ImanNeural
Gender: Female
Name: ar-LY-OmarNeural
Gender: Male
Name: ar-MA-JamalNeural
Gender: Male
Name: ar-MA-MounaNeural
Gender: Female
Name: ar-OM-AbdullahNeural
Gender: Male
Name: ar-OM-AyshaNeural
Gender: Female
Name: ar-QA-AmalNeural
Gender: Female
Name: ar-QA-MoazNeural
Gender: Male
Name: ar-SA-HamedNeural
Gender: Male
Name: ar-SA-ZariyahNeural
Gender: Female
Name: ar-SY-AmanyNeural
Gender: Female
Name: ar-SY-LaithNeural
Gender: Male
Name: ar-TN-HediNeural
Gender: Male
Name: ar-TN-ReemNeural
Gender: Female
Name: ar-YE-MaryamNeural
Gender: Female
Name: ar-YE-SalehNeural
Gender: Male
Name: az-AZ-BabekNeural
Gender: Male
Name: az-AZ-BanuNeural
Gender: Female
Name: bg-BG-BorislavNeural
Gender: Male
Name: bg-BG-KalinaNeural
Gender: Female
Name: bn-BD-NabanitaNeural
Gender: Female
Name: bn-BD-PradeepNeural
Gender: Male
Name: bn-IN-BashkarNeural
Gender: Male
Name: bn-IN-TanishaaNeural
Gender: Female
Name: bs-BA-GoranNeural
Gender: Male
Name: bs-BA-VesnaNeural
Gender: Female
Name: ca-ES-EnricNeural
Gender: Male
Name: ca-ES-JoanaNeural
Gender: Female
Name: cs-CZ-AntoninNeural
Gender: Male
Name: cs-CZ-VlastaNeural
Gender: Female
Name: cy-GB-AledNeural
Gender: Male
Name: cy-GB-NiaNeural
Gender: Female
Name: da-DK-ChristelNeural
Gender: Female
Name: da-DK-JeppeNeural
Gender: Male
Name: de-AT-IngridNeural
Gender: Female
Name: de-AT-JonasNeural
Gender: Male
Name: de-CH-JanNeural
Gender: Male
Name: de-CH-LeniNeural
Gender: Female
Name: de-DE-AmalaNeural
Gender: Female
Name: de-DE-ConradNeural
Gender: Male
Name: de-DE-KatjaNeural
Gender: Female
Name: de-DE-KillianNeural
Gender: Male
Name: el-GR-AthinaNeural
Gender: Female
Name: el-GR-NestorasNeural
Gender: Male
Name: en-AU-NatashaNeural
Gender: Female
Name: en-AU-WilliamNeural
Gender: Male
Name: en-CA-ClaraNeural
Gender: Female
Name: en-CA-LiamNeural
Gender: Male
Name: en-GB-LibbyNeural
Gender: Female
Name: en-GB-MaisieNeural
Gender: Female
Name: en-GB-RyanNeural
Gender: Male
Name: en-GB-SoniaNeural
Gender: Female
Name: en-GB-ThomasNeural
Gender: Male
Name: en-HK-SamNeural
Gender: Male
Name: en-HK-YanNeural
Gender: Female
Name: en-IE-ConnorNeural
Gender: Male
Name: en-IE-EmilyNeural
Gender: Female
Name: en-IN-NeerjaExpressiveNeural
Gender: Female
Name: en-IN-NeerjaNeural
Gender: Female
Name: en-IN-PrabhatNeural
Gender: Male
Name: en-KE-AsiliaNeural
Gender: Female
Name: en-KE-ChilembaNeural
Gender: Male
Name: en-NG-AbeoNeural
Gender: Male
Name: en-NG-EzinneNeural
Gender: Female
Name: en-NZ-MitchellNeural
Gender: Male
Name: en-NZ-MollyNeural
Gender: Female
Name: en-PH-JamesNeural
Gender: Male
Name: en-PH-RosaNeural
Gender: Female
Name: en-SG-LunaNeural
Gender: Female
Name: en-SG-WayneNeural
Gender: Male
Name: en-TZ-ElimuNeural
Gender: Male
Name: en-TZ-ImaniNeural
Gender: Female
Name: en-US-AnaNeural
Gender: Female
Name: en-US-AriaNeural
Gender: Female
Name: en-US-ChristopherNeural
Gender: Male
Name: en-US-EricNeural
Gender: Male
Name: en-US-GuyNeural
Gender: Male
Name: en-US-JennyNeural
Gender: Female
Name: en-US-MichelleNeural
Gender: Female
Name: en-US-RogerNeural
Gender: Male
Name: en-US-SteffanNeural
Gender: Male
Name: en-ZA-LeahNeural
Gender: Female
Name: en-ZA-LukeNeural
Gender: Male
Name: es-AR-ElenaNeural
Gender: Female
Name: es-AR-TomasNeural
Gender: Male
Name: es-BO-MarceloNeural
Gender: Male
Name: es-BO-SofiaNeural
Gender: Female
Name: es-CL-CatalinaNeural
Gender: Female
Name: es-CL-LorenzoNeural
Gender: Male
Name: es-CO-GonzaloNeural
Gender: Male
Name: es-CO-SalomeNeural
Gender: Female
Name: es-CR-JuanNeural
Gender: Male
Name: es-CR-MariaNeural
Gender: Female
Name: es-CU-BelkysNeural
Gender: Female
Name: es-CU-ManuelNeural
Gender: Male
Name: es-DO-EmilioNeural
Gender: Male
Name: es-DO-RamonaNeural
Gender: Female
Name: es-EC-AndreaNeural
Gender: Female
Name: es-EC-LuisNeural
Gender: Male
Name: es-ES-AlvaroNeural
Gender: Male
Name: es-ES-ElviraNeural
Gender: Female
Name: es-ES-ManuelEsCUNeural
Gender: Male
Name: es-GQ-JavierNeural
Gender: Male
Name: es-GQ-TeresaNeural
Gender: Female
Name: es-GT-AndresNeural
Gender: Male
Name: es-GT-MartaNeural
Gender: Female
Name: es-HN-CarlosNeural
Gender: Male
Name: es-HN-KarlaNeural
Gender: Female
Name: es-MX-DaliaNeural
Gender: Female
Name: es-MX-JorgeNeural
Gender: Male
Name: es-MX-LorenzoEsCLNeural
Gender: Male
Name: es-NI-FedericoNeural
Gender: Male
Name: es-NI-YolandaNeural
Gender: Female
Name: es-PA-MargaritaNeural
Gender: Female
Name: es-PA-RobertoNeural
Gender: Male
Name: es-PE-AlexNeural
Gender: Male
Name: es-PE-CamilaNeural
Gender: Female
Name: es-PR-KarinaNeural
Gender: Female
Name: es-PR-VictorNeural
Gender: Male
Name: es-PY-MarioNeural
Gender: Male
Name: es-PY-TaniaNeural
Gender: Female
Name: es-SV-LorenaNeural
Gender: Female
Name: es-SV-RodrigoNeural
Gender: Male
Name: es-US-AlonsoNeural
Gender: Male
Name: es-US-PalomaNeural
Gender: Female
Name: es-UY-MateoNeural
Gender: Male
Name: es-UY-ValentinaNeural
Gender: Female
Name: es-VE-PaolaNeural
Gender: Female
Name: es-VE-SebastianNeural
Gender: Male
Name: et-EE-AnuNeural
Gender: Female
Name: et-EE-KertNeural
Gender: Male
Name: fa-IR-DilaraNeural
Gender: Female
Name: fa-IR-FaridNeural
Gender: Male
Name: fi-FI-HarriNeural
Gender: Male
Name: fi-FI-NooraNeural
Gender: Female
Name: fil-PH-AngeloNeural
Gender: Male
Name: fil-PH-BlessicaNeural
Gender: Female
Name: fr-BE-CharlineNeural
Gender: Female
Name: fr-BE-GerardNeural
Gender: Male
Name: fr-CA-AntoineNeural
Gender: Male
Name: fr-CA-JeanNeural
Gender: Male
Name: fr-CA-SylvieNeural
Gender: Female
Name: fr-CH-ArianeNeural
Gender: Female
Name: fr-CH-FabriceNeural
Gender: Male
Name: fr-FR-DeniseNeural
Gender: Female
Name: fr-FR-EloiseNeural
Gender: Female
Name: fr-FR-HenriNeural
Gender: Male
Name: ga-IE-ColmNeural
Gender: Male
Name: ga-IE-OrlaNeural
Gender: Female
Name: gl-ES-RoiNeural
Gender: Male
Name: gl-ES-SabelaNeural
Gender: Female
Name: gu-IN-DhwaniNeural
Gender: Female
Name: gu-IN-NiranjanNeural
Gender: Male
Name: he-IL-AvriNeural
Gender: Male
Name: he-IL-HilaNeural
Gender: Female
Name: hi-IN-MadhurNeural
Gender: Male
Name: hi-IN-SwaraNeural
Gender: Female
Name: hr-HR-GabrijelaNeural
Gender: Female
Name: hr-HR-SreckoNeural
Gender: Male
Name: hu-HU-NoemiNeural
Gender: Female
Name: hu-HU-TamasNeural
Gender: Male
Name: id-ID-ArdiNeural
Gender: Male
Name: id-ID-GadisNeural
Gender: Female
Name: is-IS-GudrunNeural
Gender: Female
Name: is-IS-GunnarNeural
Gender: Male
Name: it-IT-DiegoNeural
Gender: Male
Name: it-IT-ElsaNeural
Gender: Female
Name: it-IT-IsabellaNeural
Gender: Female
Name: ja-JP-KeitaNeural
Gender: Male
Name: ja-JP-NanamiNeural
Gender: Female
Name: jv-ID-DimasNeural
Gender: Male
Name: jv-ID-SitiNeural
Gender: Female
Name: ka-GE-EkaNeural
Gender: Female
Name: ka-GE-GiorgiNeural
Gender: Male
Name: kk-KZ-AigulNeural
Gender: Female
Name: kk-KZ-DauletNeural
Gender: Male
Name: km-KH-PisethNeural
Gender: Male
Name: km-KH-SreymomNeural
Gender: Female
Name: kn-IN-GaganNeural
Gender: Male
Name: kn-IN-SapnaNeural
Gender: Female
Name: ko-KR-InJoonNeural
Gender: Male
Name: ko-KR-SunHiNeural
Gender: Female
Name: lo-LA-ChanthavongNeural
Gender: Male
Name: lo-LA-KeomanyNeural
Gender: Female
Name: lt-LT-LeonasNeural
Gender: Male
Name: lt-LT-OnaNeural
Gender: Female
Name: lv-LV-EveritaNeural
Gender: Female
Name: lv-LV-NilsNeural
Gender: Male
Name: mk-MK-AleksandarNeural
Gender: Male
Name: mk-MK-MarijaNeural
Gender: Female
Name: ml-IN-MidhunNeural
Gender: Male
Name: ml-IN-SobhanaNeural
Gender: Female
Name: mn-MN-BataaNeural
Gender: Male
Name: mn-MN-YesuiNeural
Gender: Female
Name: mr-IN-AarohiNeural
Gender: Female
Name: mr-IN-ManoharNeural
Gender: Male
Name: ms-MY-OsmanNeural
Gender: Male
Name: ms-MY-YasminNeural
Gender: Female
Name: mt-MT-GraceNeural
Gender: Female
Name: mt-MT-JosephNeural
Gender: Male
Name: my-MM-NilarNeural
Gender: Female
Name: my-MM-ThihaNeural
Gender: Male
Name: nb-NO-FinnNeural
Gender: Male
Name: nb-NO-PernilleNeural
Gender: Female
Name: ne-NP-HemkalaNeural
Gender: Female
Name: ne-NP-SagarNeural
Gender: Male
Name: nl-BE-ArnaudNeural
Gender: Male
Name: nl-BE-DenaNeural
Gender: Female
Name: nl-NL-ColetteNeural
Gender: Female
Name: nl-NL-FennaNeural
Gender: Female
Name: nl-NL-MaartenNeural
Gender: Male
Name: pl-PL-MarekNeural
Gender: Male
Name: pl-PL-ZofiaNeural
Gender: Female
Name: ps-AF-GulNawazNeural
Gender: Male
Name: ps-AF-LatifaNeural
Gender: Female
Name: pt-BR-AntonioNeural
Gender: Male
Name: pt-BR-FranciscaNeural
Gender: Female
Name: pt-PT-DuarteNeural
Gender: Male
Name: pt-PT-RaquelNeural
Gender: Female
Name: ro-RO-AlinaNeural
Gender: Female
Name: ro-RO-EmilNeural
Gender: Male
Name: ru-RU-DmitryNeural
Gender: Male
Name: ru-RU-SvetlanaNeural
Gender: Female
Name: si-LK-SameeraNeural
Gender: Male
Name: si-LK-ThiliniNeural
Gender: Female
Name: sk-SK-LukasNeural
Gender: Male
Name: sk-SK-ViktoriaNeural
Gender: Female
Name: sl-SI-PetraNeural
Gender: Female
Name: sl-SI-RokNeural
Gender: Male
Name: so-SO-MuuseNeural
Gender: Male
Name: so-SO-UbaxNeural
Gender: Female
Name: sq-AL-AnilaNeural
Gender: Female
Name: sq-AL-IlirNeural
Gender: Male
Name: sr-RS-NicholasNeural
Gender: Male
Name: sr-RS-SophieNeural
Gender: Female
Name: su-ID-JajangNeural
Gender: Male
Name: su-ID-TutiNeural
Gender: Female
Name: sv-SE-MattiasNeural
Gender: Male
Name: sv-SE-SofieNeural
Gender: Female
Name: sw-KE-RafikiNeural
Gender: Male
Name: sw-KE-ZuriNeural
Gender: Female
Name: sw-TZ-DaudiNeural
Gender: Male
Name: sw-TZ-RehemaNeural
Gender: Female
Name: ta-IN-PallaviNeural
Gender: Female
Name: ta-IN-ValluvarNeural
Gender: Male
Name: ta-LK-KumarNeural
Gender: Male
Name: ta-LK-SaranyaNeural
Gender: Female
Name: ta-MY-KaniNeural
Gender: Female
Name: ta-MY-SuryaNeural
Gender: Male
Name: ta-SG-AnbuNeural
Gender: Male
Name: ta-SG-VenbaNeural
Gender: Female
Name: te-IN-MohanNeural
Gender: Male
Name: te-IN-ShrutiNeural
Gender: Female
Name: th-TH-NiwatNeural
Gender: Male
Name: th-TH-PremwadeeNeural
Gender: Female
Name: tr-TR-AhmetNeural
Gender: Male
Name: tr-TR-EmelNeural
Gender: Female
Name: uk-UA-OstapNeural
Gender: Male
Name: uk-UA-PolinaNeural
Gender: Female
Name: ur-IN-GulNeural
Gender: Female
Name: ur-IN-SalmanNeural
Gender: Male
Name: ur-PK-AsadNeural
Gender: Male
Name: ur-PK-UzmaNeural
Gender: Female
Name: uz-UZ-MadinaNeural
Gender: Female
Name: uz-UZ-SardorNeural
Gender: Male
Name: vi-VN-HoaiMyNeural
Gender: Female
Name: vi-VN-NamMinhNeural
Gender: Male
Name: zh-CN-XiaoxiaoNeural
Gender: Female
Name: zh-CN-XiaoyiNeural
Gender: Female
Name: zh-CN-YunjianNeural
Gender: Male
Name: zh-CN-YunxiNeural
Gender: Male
Name: zh-CN-YunxiaNeural
Gender: Male
Name: zh-CN-YunyangNeural
Gender: Male
Name: zh-CN-liaoning-XiaobeiNeural
Gender: Female
Name: zh-CN-shaanxi-XiaoniNeural
Gender: Female
Name: zh-HK-HiuGaaiNeural
Gender: Female
Name: zh-HK-HiuMaanNeural
Gender: Female
Name: zh-HK-WanLungNeural
Gender: Male
Name: zh-TW-HsiaoChenNeural
Gender: Female
Name: zh-TW-HsiaoYuNeural
Gender: Female
Name: zh-TW-YunJheNeural
Gender: Male
Name: zu-ZA-ThandoNeural
Gender: Female
Name: zu-ZA-ThembaNeural
Gender: Male
一望而知,幾乎支援所有主流的通用語,Gender欄位為合成語音的性別,Male代表男性,Female代表女性,zh開頭的就是中文語音角色,這裡以微軟的小伊為例子:
edge-tts --voice zh-CN-XiaoyiNeural --text "你好啊,我是智慧語音助手" --write-media hello_in_cn.mp3
該命令含義是透過zh-CN-XiaoyiNeural角色合成語音:"你好啊,我是智慧語音助手"的內容,隨後將音訊流寫入hello_in_cn.mp3檔案。
程式返回:
Downloads edge-tts --voice zh-CN-XiaoyiNeural --text "你好啊,我是智慧語音助手" --write-media hello_in_cn.mp3
WEBVTT
00:00:00.100 --> 00:00:00.525
你好
00:00:00.525 --> 00:00:00.912
啊
00:00:01.050 --> 00:00:01.238
我
00:00:01.238 --> 00:00:01.375
是
00:00:01.387 --> 00:00:01.700
智慧
00:00:01.700 --> 00:00:02.050
語音
00:00:02.062 --> 00:00:02.550
助手
程式會自動將時間軸和語音文字匹配輸出,如此一來,連字幕檔案也有了,可謂是一舉兩得,一箭雙鵰。
與此同時,我們也可以調整合成語音的語速:
edge-tts --rate=-50% --voice zh-CN-XiaoyiNeural --text "你好啊,我是智慧語音助手" --write-media hello_in_cn.mp3
--rate引數可以透過加號或者減號同步加快或者減慢合成語音的語速。
亦或者,調整合成語音的音量:
edge-tts --volume=-50% --voice zh-CN-XiaoyiNeural --text "你好啊,我是智慧語音助手" --write-media hello_in_cn.mp3
--volume引數可以調整語音的音量。
遺憾的是,和微軟Azure官方的語音合成庫相比,開源的語音合成庫並不支援基於標記語言 (SSML)的語音調優,比如語調、情緒的調整,但這畢竟是免費的,要求也不能太高了。
Python指令碼語音合成
除了透過命令進行語音合成,edge-tts也支援在Python指令碼,編輯test.py檔案:
import asyncio
import edge_tts
TEXT = "你好喲,我是智慧語音助手,小伊"
VOICE = "zh-CN-XiaoyiNeural"
OUTPUT_FILE = "/Users/liuyue/Downloads/test.mp3"
async def _main() -> None:
communicate = edge_tts.Communicate(TEXT, VOICE)
await communicate.save(OUTPUT_FILE)
if __name__ == "__main__":
asyncio.run(_main())
這裡我們直接透過非同步模式呼叫communicate例項的save方法,就可以併發非同步生成語音合成的音訊檔案,非常方便。
也可以透過語音管理庫來自動尋找我們需要的語言:
import asyncio
import random
import edge_tts
from edge_tts import VoicesManager
TEXT = "中文語音測試"
OUTPUT_FILE ="china.mp3"
async def _main() -> None:
voices = await VoicesManager.create()
voice = voices.find(Gender="Female", Language="zh")
communicate = edge_tts.Communicate(TEXT, random.choice(voice)["Name"])
await communicate.save(OUTPUT_FILE)
if __name__ == "__main__":
asyncio.run(_main())
這裡透過內建的VoicesManager庫來隨機挑選中文語音角色完成語音合成操作。
除此之外,也可以透過指令碼將語音流和字母同步進行生成:
import asyncio
import edge_tts
TEXT = "這裡是語音流測試"
VOICE = "zh-CN-XiaoyiNeural"
OUTPUT_FILE = "test.mp3"
WEBVTT_FILE = "test.vtt"
async def _main() -> None:
communicate = edge_tts.Communicate(TEXT, VOICE)
submaker = edge_tts.SubMaker()
with open(OUTPUT_FILE, "wb") as file:
async for chunk in communicate.stream():
if chunk["type"] == "audio":
file.write(chunk["data"])
elif chunk["type"] == "WordBoundary":
submaker.create_sub((chunk["offset"], chunk["duration"]), chunk["text"])
with open(WEBVTT_FILE, "w", encoding="utf-8") as file:
file.write(submaker.generate_subs())
if __name__ == "__main__":
asyncio.run(_main())
這裡非同步呼叫之後,音訊會寫入test.mp3,而字幕檔案則會寫入test.vtt。後續則可以透過ffmpeg將生成的字幕檔案疊加到影片中即可,請參見:基於Python3(Autosub)以及Ffmpeg配合GoogleTranslation(谷歌翻譯)為你的影片實現雙語版字幕(逐字稿)
結語
開源語音合成edge-tts庫可以提高語音合成效率,並且極大地降低了語音合成門檻,為自動化影片剪輯鋪平了道路,未來結合基於Stable-Diffusion演演算法的AI繪圖框架,人工智慧AI一鍵式繪製、配音、上字幕的一條龍服務指日可待。