WebRTC入門

效果展示

基礎概念

• WebRTC指的是基於web的實時視訊通話，其實就相當於A->B發直播畫面，同時B->A傳送直播畫面，這樣就是影片聊天了
• WebRTC的視訊通話是A和B兩兩之間進行的
• WebRTC通話雙方透過一個公共的中心伺服器找到對方，就像聊天室一樣
• WebRTC的連線過程一般是
  1. A透過websocket連線下中心伺服器，B透過websocket連線下中心伺服器。每次有人加入或退出中心伺服器，中心伺服器就把為維護的連線廣播給A和B
  2. A接到廣播知道了B的存在，A發起提案，傳遞影片編碼器等引數,讓中心伺服器轉發給B。B收到中心伺服器轉發的A的提案，建立回答，傳遞影片編碼器等引數，讓中心伺服器轉發給A
  3. A收到回答，發起互動式連線，包括自己的地址，埠等，讓中心伺服器轉發給B。B收到連線，回答互動式連線，包括自己的地址，埠等，讓中心伺服器轉發給A。
  4. 至此A知道了B的地址，B知道了A的地址，連線建立，中心伺服器退出整個過程
  5. A給B推影片流，同時B給A推影片流。雙方同時用video元素把對方的影片流播放出來

API

• WebSokcet 和中心伺服器的連線，中心伺服器也叫信令伺服器，用來建立連線前中轉訊息,相當於相親前的媒人

• RTCPeerConnection 視訊通話連線

• rc.createOffer 發起方建立本地提案，獲得SDP描述

• rc.createAnswer 接收方建立本地回答，獲得SDP描述

• rc.setLocalDescription 設定本地建立的SDP描述

• rc.setRemoteDescription 設定對方傳遞過來的SDP描述

• rc.onicecandidate 在建立本地提案會本地回答時觸發此事件，獲得互動式連線物件，用於傳送給對方

• rc.addIceCandidate 設定中心伺服器轉發過來IceCandidate

• rc.addStream 向連線中新增媒體流

• rc.addTrack 向媒體流中新增軌道

• rc.ontrack 在此事件中接受來自對方的媒體流

其實兩個人通訊只需要一個RTCPeerConnection，A和B各持一端，不需要兩個RTCPeerConnection,這點容易被誤導

媒體流

獲取

這裡我獲取的是視窗影片流，而不是攝像頭影片流

navigator.mediaDevices.getDisplayMedia()
    .then(meStream => {
        //在本地顯示預覽
        document.getElementById("local").srcObject = meStream;
    })

傳輸

        //給對方傳送影片流
        other.stream = meStream;
        const videoTracks = meStream.getVideoTracks();
        const audioTracks = meStream.getAudioTracks();
        //log("推流")
        other.peerConnection.addStream(meStream);
        meStream.getVideoTracks().forEach(track => {
            other.peerConnection.addTrack(track, meStream);
        });

接收

other.peerConnection.addEventListener("track", event => {
    //log("拉流")
    document.getElementById("remote").srcObject = event.streams[0];
})

連線

WebSocet連線

這是最開始需要建立的和信令伺服器的連線，用於點對點連線建立前轉發訊息，這算是最重要的邏輯了

ws = new WebSocket('/sdp');
ws.addEventListener("message", event => {
    var msg = JSON.parse(event.data);
    if (msg.type == "connect") {
        //log("接到提案");
        var other = remotes.find(r => r.name != myName);
        onReciveOffer(msg.data.description, msg.data.candidate, other);
    }
    else if (msg.type == "connected") {
        //log("接到回答");
        var other = remotes.find(r => r.name != myName);
        onReciveAnwer(msg.data.description, msg.data.candidate, other);
    }
    //獲取自己在房間中的臨時名字
    else if (msg.type == "id") {
        myName = msg.data;
    }
    //有人加入或退出房間時
    else if (msg.type == "join") {
        //成員列表
        for (var i = 0; i < msg.data.length; i++) {
            var other = remotes.find(r => r.name == msg.data[i]);
            if (other == null) {
                remotes.push({
                    stream: null,
                    peerConnection: new RTCPeerConnection(null),
                    description: null,
                    candidate: null,
                    video: null,
                    name: msg.data[i]
                });
            }
        }
        //過濾已經離開的人
        remotes = remotes.filter(r => msg.data.find(x => x == r.name) != null);
        //...
    }
});

RTCPeerConnection連線

在都已經加入聊天室後就可以開始建立點對點連線了

//對某人建立提案
other.peerConnection.createOffer({ offerToReceiveVideo: 1 })
    .then(description => {
        //設定成自己的本地描述
        other.description = description;
        other.peerConnection.setLocalDescription(description);
    });

在建立提案後會觸發此事件，然後把提案和互動式連線訊息一起傳送出去

//互動式連線候選項
other.peerConnection.addEventListener("icecandidate", event => {
    other.candidate = event.candidate;
    //log("發起提案");
    //傳送提案到中心伺服器
    ws.send(JSON.stringify({
        type: "connect",
        data: {
            name: other.name,
            description: other.description,
            candidate: other.candidate
        }
    }));
})

對方收到提案後按照同樣的流程建立回答和響應

/**接收到提案 */
function onReciveOffer(description, iceCandidate,other) {
    //互動式連線候選者
    other.peerConnection.addEventListener("icecandidate", event => {
        other.candidate = event.candidate;
        //log("發起回答");
        //回答信令到中心伺服器
        ws.send(JSON.stringify({
            type: "connected",
            data: {
                name: other.name,
                description: other.description,
                candidate: other.candidate
            }
        }));
    })
    //設定來自對方的遠端描述
    other.peerConnection.setRemoteDescription(description);
    other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
    other.peerConnection.createAnswer()
        .then(answerDescription => {
            other.description = answerDescription;
            other.peerConnection.setLocalDescription(answerDescription);
        })
}

發起方收到回答後，點對點連線建立，雙方都能看到畫面了，至此已經不需要中心伺服器了

/**接收到回答 */
function onReciveAnwer(description, iceCandidate,other) {
    //收到回答後設定接收方的描述
    other.peerConnection.setRemoteDescription(description);
    other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
}

完整程式碼

SDPController.cs

[ApiController]
[Route("sdp")]
public class SDPController : Controller
{
    public static List<(string name, WebSocket ws)> clients = new List<(string, WebSocket)>();
    private List<string> names = new List<string>() { "張三", "李四", "王五","鐘鳴" };

    [HttpGet("")]
    public async Task Index()
    {
        WebSocket client = await HttpContext.WebSockets.AcceptWebSocketAsync();
        var ws = (name:names[clients.Count], client);
        clients.Add(ws);
        await client.SendAsync(UTF8Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(new {type="id",data=ws.name})), WebSocketMessageType.Text, true, CancellationToken.None);
        List<string> list = new List<string>();
        foreach (var person in clients)
        {
            list.Add(person.name);
        }
        var join = new
        {
            type = "join",
            data = list,
        };
        foreach (var item in clients)
        {
            await item.ws.SendAsync(UTF8Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(join)), WebSocketMessageType.Text, true, CancellationToken.None);
        }

        var defaultBuffer = new byte[40000];
        try
        {
            while (!client.CloseStatus.HasValue)
            {
                //接受信令
                var result = await client.ReceiveAsync(defaultBuffer, CancellationToken.None);
                JObject obj=JsonConvert.DeserializeObject<JObject>(UTF8Encoding.UTF8.GetString(defaultBuffer,0,result.Count));
                if (obj.Value<string>("type")=="connect" || obj.Value<string>("type") == "connected")
                {
                    var another = clients.FirstOrDefault(r => r.name == obj["data"].Value<string>("name"));
                    await another.ws.SendAsync(new ArraySegment<byte>(defaultBuffer, 0, result.Count), result.MessageType, result.EndOfMessage, CancellationToken.None);
                }
            }
        }
        catch (Exception e)
        {
        }
        Console.WriteLine("退出");
        clients.Remove(ws);
        list = new List<string>();
        foreach (var person in clients)
        {
            list.Add(person.name);
        }
        join = new
        {
            type = "join",
            data = list
        };
        foreach (var item in clients)
        {
            await item.ws.SendAsync(UTF8Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(join)), WebSocketMessageType.Text, true, CancellationToken.None);
        }
    }
}

home.html

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8" />
    <title></title>
    <style>
        html,body{
            height:100%;
            margin:0;
        }
        .container{
            display:grid;
            grid-template:auto 1fr 1fr/1fr 200px;
            height:100%;
            grid-gap:8px;
            justify-content:center;
            align-items:center;
        }
        .video {
            background-color: black;
            height:calc(100% - 1px);
            overflow:auto;
        }
        #local {
            grid-area:2/1/3/2;
        }
        #remote {
            grid-area: 3/1/4/2;
        }
        .list{
            grid-area:1/2/4/3;
            background-color:#eeeeee;
            height:100%;
            overflow:auto;
        }
        #persons{
            text-align:center;
        }
        .person{
            padding:5px;
        }
    </style>
</head>
<body>
    <div class="container">
        <div style="grid-area:1/1/2/2;padding:8px;">
            <button id="start">錄製本地視窗</button>
            <button id="call">發起遠端</button>
            <button id="hangup">結束通話遠端</button>
        </div>
        <video autoplay id="local" class="video"></video>
        <video autoplay id="remote" class="video"></video>
        <div class="list">
            <div style="text-align:center;background-color:white;padding:8px;">
                <button id="join">加入</button>
                <button id="exit">退出</button>
            </div>
            <div id="persons">

            </div>
            <div id="log">

            </div>
        </div>
    </div>

    <script>
        /**在螢幕頂部顯示一條訊息，3秒後消失 */
        function layerMsg(msg) {
            // 建立一個新的div元素作為訊息層
            var msgDiv = document.createElement('div');
            msgDiv.textContent = msg;

            // 設定訊息層的樣式
            msgDiv.style.position = 'fixed';
            msgDiv.style.top = '0';
            msgDiv.style.left = '50%';
            msgDiv.style.transform = 'translateX(-50%)';
            msgDiv.style.background = '#f2f2f2';
            msgDiv.style.color = '#333';
            msgDiv.style.padding = '10px';
            msgDiv.style.borderBottom = '2px solid #ccc';
            msgDiv.style.width = '100%';
            msgDiv.style.textAlign = 'center';
            msgDiv.style.zIndex = '9999'; // 確保訊息層顯示在最頂層

            // 將訊息層新增到文件的body中
            document.body.appendChild(msgDiv);

            // 使用setTimeout函式，在3秒後移除訊息層
            setTimeout(function () {
                document.body.removeChild(msgDiv);
            }, 3000);
        }
        function log(msg) {
            document.getElementById("log").innerHTML += `<div>${msg}</div>`;
        }
    </script>

    <script>
        var myName = null;
        // 伺服器配置
        const servers = null;
        var remotes = [];
        var startButton = document.getElementById("start");
        var callButton = document.getElementById("call");
        var hangupButton = document.getElementById("hangup");
        var joinButton = document.getElementById("join");
        var exitButton = document.getElementById("exit");
        startButton.disabled = false;
        callButton.disabled = false;
        hangupButton.disabled = true;
        joinButton.disabled = false;
        exitButton.disabled = true;

        /**和中心伺服器的連線，用於交換信令 */
        var ws;
        //加入房間
        document.getElementById("join").onclick = function () {
            ws = new WebSocket('/sdp');
            ws.addEventListener("message", event => {
                var msg = JSON.parse(event.data);
                if (msg.type == "offer") {
                    log("接收到offer");
                    onReciveOffer(msg);
                }
                else if (msg.type == "answer") {
                    log("接收到answer");
                    onReciveAnwer(msg);
                }
                else if (msg.candidate != undefined) {
                    layerMsg("接收到candidate");
                    onReciveIceCandidate(msg);
                }
                else if (msg.type == "connect") {
                    log("接到提案");
                    var other = remotes.find(r => r.name != myName);
                    onReciveOffer(msg.data.description, msg.data.candidate, other);
                }
                else if (msg.type == "connected") {
                    log("接到回答");
                    var other = remotes.find(r => r.name != myName);
                    onReciveAnwer(msg.data.description, msg.data.candidate, other);
                }
                else if (msg.type == "id") {
                    myName = msg.data;
                }
                else if (msg.type == "join") {
                    //新增
                    for (var i = 0; i < msg.data.length; i++) {
                        var other = remotes.find(r => r.name == msg.data[i]);
                        if (other == null) {
                            remotes.push({
                                stream: null,
                                peerConnection: new RTCPeerConnection(servers),
                                description: null,
                                candidate: null,
                                video: null,
                                name: msg.data[i]
                            });
                        }
                    }
                    //過濾已經離開的人
                    remotes = remotes.filter(r => msg.data.find(x => x == r.name) != null);
                    document.getElementById("persons").innerHTML = "";
                    for (var i = 0; i < remotes.length; i++) {
                        var div = document.createElement("div");
                        div.classList.add("person")
                        var btn = document.createElement("button");
                        btn.innerText = remotes[i].name;
                        if (remotes[i].name == myName) {
                            btn.innerText += "（我）";
                        }
                        div.appendChild(btn);
                        document.getElementById("persons").appendChild(div);
                    }
                }
            });
            startButton.disabled = false;
            joinButton.disabled = true;
            exitButton.disabled = false;

        }
        //退出房間
        document.getElementById("exit").onclick = function () {
            if (ws != null) {
                ws.close();
                ws = null;
                startButton.disabled = true;
                callButton.disabled = true;
                hangupButton.disabled = true;
                joinButton.disabled = false;
                exitButton.disabled = true;
                document.getElementById("persons").innerHTML = "";
                remotes = [];
                local.peerConnection = null;
                local.candidate = null;
                local.description = null;
                local.stream = null;
                local.video = null;
            }
        }

        //推流
        startButton.onclick = function () {
            var local = remotes.find(r => r.name == myName);
            var other = remotes.find(r => r.name != myName);
            if (other == null) {
                return;
            }
            navigator.mediaDevices.getDisplayMedia()
                .then(meStream => {
                    //在本地顯示預覽
                    document.getElementById("local").srcObject = meStream;
                    //給對方傳送影片流
                    other.stream = meStream;
                    const videoTracks = meStream.getVideoTracks();
                    const audioTracks = meStream.getAudioTracks();
                    log("推流")
                    other.peerConnection.addStream(meStream);
                    meStream.getVideoTracks().forEach(track => {
                        other.peerConnection.addTrack(track, meStream);
                    });
                })
        }
        callButton.onclick = function () {
            callButton.disabled = true;
            hangupButton.disabled = false;
            var other = remotes.find(r => r.name != myName);
            //互動式連線候選者
            other.peerConnection.addEventListener("icecandidate", event => {
                if (event.candidate == null) {
                    return;
                }
                other.candidate = event.candidate;
                log("發起提案");
                //傳送提案到中心伺服器
                ws.send(JSON.stringify({
                    type: "connect",
                    data: {
                        name: other.name,
                        description: other.description,
                        candidate: other.candidate
                    }
                }));
            })
            other.peerConnection.addEventListener("track", event => {
                log("拉流")
                document.getElementById("remote").srcObject = event.streams[0];
            })
            //對某人建立信令
            other.peerConnection.createOffer({ offerToReceiveVideo: 1 })
                .then(description => {
                    //設定成自己的本地描述
                    other.description = description;
                    other.peerConnection.setLocalDescription(description);
                })
                .catch(e => {
                    debugger
                });
        }
        //結束通話給對方的流
        hangupButton.onclick = function () {
            callButton.disabled = false;
            hangupButton.disabled = true;
            var local = remotes.find(r => r.name == myName);
            var other = remotes.find(r => r.name != myName);
            other.peerConnection = new RTCPeerConnection(servers);
            other.description = null;
            other.candidate = null;
            other.stream = null;
        }

        /**接收到回答 */
        function onReciveAnwer(description, iceCandidate,other) {
            if (other == null) {
                return;
            }
            //收到回答後設定接收方的描述
            other.peerConnection.setRemoteDescription(description)
                .catch(e => {
                    debugger
                });
            other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
        }

        /**接收到提案 */
        function onReciveOffer(description, iceCandidate,other) {
            //互動式連線候選者
            other.peerConnection.addEventListener("icecandidate", event => {
                if (event.candidate == null) {
                    return;
                }
                other.candidate = event.candidate;
                log("發起回答");
                //回答信令到中心伺服器
                ws.send(JSON.stringify({
                    type: "connected",
                    data: {
                        name: other.name,
                        description: other.description,
                        candidate: other.candidate
                    }
                }));
            })
            other.peerConnection.addEventListener("track", event => {
                log("拉流")
                document.getElementById("remote").srcObject = event.streams[0];
            })
            //設定來自對方的遠端描述
            other.peerConnection.setRemoteDescription(description)
                .catch(e => {
                    debugger
                });
            other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
            other.peerConnection.createAnswer()
                .then(answerDescription => {
                    other.description = answerDescription;
                    other.peerConnection.setLocalDescription(answerDescription);
                })
        }

        function onReciveIceCandidate(iceCandidate) {
            if (remotePeerConnection == null) {
                return;
            }
            remotePeerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
        }
    </script>
</body>
</html>