投稿
  1. 首页
  2. app

iOS WebRTC多人音视频建立的流程

image • 2022-5-21 • app • 阅读 40

iOS WebRTC多人音视频建立的流程,第1张

前言

本文主要以“代码是最好的注释”为基点，介绍在处理iOS端多人音视频的建立流程。 在看本篇前建议先了解一下多人音视频通讯现在的常用架构，参考《WebRTC多人音视频聊天架构与实战》。 本方案使用的是当中提到的第一种架构：Mesh 架构

六大事件 join : 加入房间offer : 本端群发 offer及接收对端offer的处理answer : 本端发送answer及接收对端发来answer的处理ice_candidate ： 对端的网络地址通过 socket 转发给本端new_peer: 成员进入remove_peer: 成员离开 join 首先，多人音视频的发起者，通过信令发送’__join’ 加入房间消息。信令服务器收到消息后，会根据用户所加入的房间, 向房间里的其他用户发送’_peers’ 消息 ,信令中包含聊天室中其他用户的信息,客户端根据信息来逐个构建与其他用户的点对点连接 offer 群发offer

上面说到，信令服务器在接收到’__join’ 消息后， 转而发送包含房间内其他用户信息的 ‘__peer’ 消息。 那么其他端在收到’__peer’ 消息后该如何处理呢 ？

没错，在收到’_peer'消息后， 本端需要向房间里的其他用户发送offer 。 发送offer的过程如下：

记录所有对端连接的ID 
- (void)_addToConnectionIDS:(NSArray *)connections
{
    if (connections.count == 0) return;
    
    [connections enumerateObjectsUsingBlock:^(id  _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
        NSLog(@"%@", obj);
    }];
    [_peerConnectionIDS addObjectsFromArray:connections];
}

其中数组connections 为__peer信令返回的房间内其他用户的连接ID, 本端维护一个对端各连接点ID的数组_peerConnectionIDS

建立本地流信息 
- (void)_setupForLocalStream
{  
    if (!_localStream) {
        // 设置点对点工厂
        [self _setupForFactory];
        
		//本地流 通过工厂来创建
		_localStream = [_factory mediaStreamWithStreamId:@"ARDAMS"];
       
        //音频轨对象 通过工厂 来创建
        RTCAudioTrack *audioTrack = [_factory audioTrackWithTrackId:@"ARDAMSa0"];
		 // 将音频轨迹添加到本地流媒体
        [_localStream addAudioTrack:audioTrack];

        AVAuthorizationStatus authStatus = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeVideo];
        NSArray *devices;
        if(authStatus == AVAuthorizationStatusRestricted || authStatus == AVAuthorizationStatusDenied) { // 摄像头权限
            if ([_delegate respondsToSelector:@selector(webRTCHelper:setLocalStream:)]) {
                [_delegate webRTCHelper:self setLocalStream:nil];
            }
        }
        else {
#if __IPHONE_OS_VERSION_MIN_REQUIRED < 100000
            if ([AVCaptureDeviceDiscoverySession class]) {
                AVCaptureDeviceDiscoverySession *deviceDiscoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[AVCaptureDeviceTypeBuiltInWideAngleCamera] mediaType:AVMediaTypeVideo position:AVCaptureDevicePositionFront];
                devices = [deviceDiscoverySession devices];
            }
            else {
                devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
            }
#else
            AVCaptureDeviceDiscoverySession *deviceDiscoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[AVCaptureDeviceTypeBuiltInWideAngleCamera] mediaType:AVMediaTypeVideo position:AVCaptureDevicePositionFront];
            devices = [deviceDiscoverySession devices];
#endif
            
            AVCaptureDevice *device = [devices lastObject];
            
            if (device) {
                
                RTCAVFoundationVideoSource *videoSource = [_factory avFoundationVideoSourceWithConstraints:[self _setupForLocalVideoConstraints]];
                [videoSource setUseBackCamera:NO];

				//视频轨对象 也是通过工厂来创建
                RTCVideoTrack *videoTrack = [_factory videoTrackWithSource:videoSource trackId:@"ARDAMSv0"];
                // 添加视频轨迹
                [_localStream addVideoTrack:videoTrack];
                
                
                if ([_delegate respondsToSelector:@selector(webRTCHelper:setLocalStream:)]) {
                    [_delegate webRTCHelper:self setLocalStream:_localStream];   
                }
                
            }
            else {
                if ([_delegate respondsToSelector:@selector(webRTCHelper:setLocalStream:)]) {
                    [_delegate webRTCHelper:self setLocalStream:nil];
                }
            }
        }
    }
}

本地流的创建是通过点对点工厂RTCPeerConnectionFactory 来创建。 本地流创建出来后，需要将本端的音频轨及视频轨添加到流媒体中。本地流创建成功后，可以通过回调方法，回调给使用者用来显示本端视频

建立所有连接 
- (void)_setupForConnections
{
    [_peerConnectionIDS enumerateObjectsUsingBlock:^(NSString * _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
        RTCPeerConnection *connection = [self _createConnection:obj];
        [_peerConnections setObject:connection forKey:obj];
    }];
}


- (RTCPeerConnection *)_createConnection:(NSString *)connectionID
{
    [self _setupForFactory];
    [self _setupForIceServers];
    RTCConfiguration *configuration = [[RTCConfiguration alloc] init];
    [configuration setIceServers:_iceServers];
    return [_factory peerConnectionWithConfiguration:configuration constraints:[self _setupForPeerVideoConstraints] delegate:self];
}

如果要与对端所有人进行通讯，必须要先和对端所有人建立连接。 上面维护了一个对端各连接点ID的数组_peerConnectionIDS， 通过遍历这个数组创建对端各点对点的连接。 接着，本地再维护一个字典_peerConnections ， key为连接点ID, value为点对点连接的对象RTCPeerConnection

所有连接添加本地流信息 
- (void)_addLocalStreamToConnections
{
    [_peerConnections enumerateKeysAndObjectsUsingBlock:^(NSString * _Nonnull key, RTCPeerConnection * _Nonnull obj, BOOL * _Nonnull stop) {
        [self _setupForLocalStream];
        [obj addStream:_localStream];
    }];
}

遍历本地维护的所有对端点对点连接的字典_peerConnections， 让对端所有连接都添加本地流信息。

所有连接发送 SDP offer 
- (void)_sendSDPOffersToConnections
{
    [_peerConnections enumerateKeysAndObjectsUsingBlock:^(NSString * _Nonnull key, RTCPeerConnection * _Nonnull obj, BOOL * _Nonnull stop) {
        [self _sendSDPOfferToConnection:obj];
    }];
}

- (void)_sendSDPOfferToConnection:(RTCPeerConnection *)connection
{
    _role = RoleCaller;
    /** Generate an SDP offer. */
    [connection offerForConstraints:[self _setupForOfferOrAnswerConstraint] completionHandler:^(RTCSessionDescription * _Nullable sdp, NSError * _Nullable error) {
        if (error) {
            NSLog(@"offerForConstraints error");
            return;
        }
        
        if (sdp.type == RTCSdpTypeOffer) {
            __weak __typeof(connection) wConnection = connection;
            /** Apply the supplied RTCSessionDescription as the local description. */
            // A:设置连接本端 SDP
            [connection setLocalDescription:sdp completionHandler:^(NSError * _Nullable error) {
                if (error) {
                    NSLog(@"setLocalDescription error");
                    return;
                }
                
                [self _didSetSessionDescription:wConnection];
            }];
        }
    }];
}

此处是关键点，上面所有的 *** 作都在为此做准备。发送offer的过程如下

通过 RTCPeerConnection 对象的offerForConstraints:生成 sdp offer将拿到的sdp 通过RTCPeerConnection 对象的setLocalDescription:设置为本地sdp本地sdp设置成功后需要通过信令将本端sdp发送到对端 
- (void)_didSetSessionDescription:(RTCPeerConnection *)connection
{
    NSLog(@"signalingState:%zd role:%zd", connection.signalingState, _role);
    NSString *connectionID = [self _findConnectionID:connection];

    if (connection.signalingState == RTCSignalingStateHaveRemoteOffer) { // 新人进入房间就调(远端发起 offer)
        /** Generate an SDP answer. */
        [connection answerForConstraints:[self _setupForOfferOrAnswerConstraint] completionHandler:^(RTCSessionDescription * _Nullable sdp, NSError * _Nullable error) {
            if (error) {
                NSLog(@"answerForConstraints error");
                return;
            }
            
            if (sdp.type == RTCSdpTypeAnswer) {
                __weak __typeof(connection) wConnection = connection;
                // B:设置连接本端 SDP
                [connection setLocalDescription:sdp completionHandler:^(NSError * _Nullable error) {
                    if (error) {
                        NSLog(@"setLocalDescription error");
                        return;
                    }
         
                    [self _didSetSessionDescription:wConnection];
                }];
            }
        }];
    }
    else if (connection.signalingState == RTCSignalingStateHaveLocalOffer) { // 本地发送 offer
        if (_role == RoleCaller) {
            NSDictionary *dic = @{@"eventName": @"__offer",
                                  @"data": @{@"sdp": @{@"type": @"offer",
                                                       @"sdp": connection.localDescription.sdp
                                                       },
                                             @"socketId": connectionID
                                             }
                                  };
            RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
            [_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) { }];
        }
    }
    else if (connection.signalingState == RTCSignalingStateStable) { // 本地发送 answer
        if (_role == RoleCalled) {
            NSDictionary *dic = @{@"eventName": @"__answer",
                                  @"data": @{@"sdp": @{@"type": @"answer",
                                                       @"sdp": connection.localDescription.sdp
                                                       },
                                             @"socketId": connectionID
                                             }
                                  };
           
            RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
            [_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) {}];
        }
    }
}

这个方法内汇总了各情况下处理，此处我们只需要看这里：

else if (connection.signalingState == RTCSignalingStateHaveLocalOffer) { // 本地发送 offer
        if (_role == RoleCaller) {
            NSDictionary *dic = @{@"eventName": @"__offer",
                                  @"data": @{@"sdp": @{@"type": @"offer",
                                                       @"sdp": connection.localDescription.sdp
                                                       },
                                             @"socketId": connectionID
                                             }
                                  };
            RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
            [_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) { }];
        }
    }

通过signalingState属性可以拿到当前RTCPeerConnection 对象的信令状态，当状态值为RTCSignalingStateHaveLocalOffer时表示本地发送offer。 在构造好约定的传输格式@{@"type": @"offer",@"sdp": connection.localDescription.sdp } 及信令eventName '__offer'后，通过信令服务器发出。

接收offer 
/**
 远端发来 offer
 */
- (void)_receiveOffer:(NSMutableDictionary *)resultDic
{
    // 设置当前角色状态为被呼叫,(被发offer）
    _role = RoleCalled;
    NSDictionary *dataDic = resultDic[@"data"];
    NSDictionary *sdpDic = dataDic[@"sdp"];
    // 拿到SDP
    NSString *sdp = sdpDic[@"sdp"];
    NSString *type = sdpDic[@"type"];
    NSString *connectionID = dataDic[@"socketId"];
    RTCSdpType sdpType = [self _typeForString:type];
    // 拿到这个点对点的连接
    RTCPeerConnection *connection = [_peerConnections objectForKey:connectionID];
    // 根据类型和SDP 生成SDP描述对象
    RTCSessionDescription *remoteSdp = [[RTCSessionDescription alloc] initWithType:sdpType sdp:sdp];
    
    if (sdpType == RTCSdpTypeOffer) {
        // 设置给这个点对点连接
        __weak __typeof(connection) wConnection = connection;
        // B:设置连接对端 SDP
        [connection setRemoteDescription:remoteSdp completionHandler:^(NSError * _Nullable error) {
            if (error) {
                NSLog(@"setRemoteDescription error");
            }
            
            [self _didSetSessionDescription:wConnection];
        }];
    }
}

在接收到对端发送的’__offer'消息后， 本端需要做如下处理：

解析接收到的消息结构，解析出当前的sdp类型及sdp信息，并构造RTCSessionDescription对象根据解析的connectId 获取到点对点连接RTCPeerConnection对象将接收到的对端sdp通过点对点连接对象的setRemoteDescription:设置为远端sdp设置成功后，本端通过信令服务器发送’__answer'消息来响应
answer 收到远端offer后主动发送answer

接上面：在收到对端’__offer‘消息后，先设置remote sdp, 接着主动发送’__answer'消息响应
_didSetSessionDescription:方法中的处理：

else if (connection.signalingState == RTCSignalingStateStable) { // 本地发送 answer
        if (_role == RoleCalled) {
            NSDictionary *dic = @{@"eventName": @"__answer",
                                  @"data": @{@"sdp": @{@"type": @"answer",
                                                       @"sdp": connection.localDescription.sdp
                                                       },
                                             @"socketId": connectionID
                                             }
                                  };
           
            RTMMessage *message = [RTMMessage createChannelMessageWithChannelId:self.channelId data:[dic mj_JSONString]];
            [_channel sendMessage:message completion:^(RTMSendChannelMessageStatusCode code) {}];
        }
    }

构造好约定的传输格式@{@"type": @"answer",@"sdp": connection.localDescription.sdp } 及信令eventName '__answer'后，通过信令服务器发出。

接收远端answer 
/**
 远端发来 answer
 */
- (void)_receiveAnswer:(NSMutableDictionary *)resultDic
{
    NSDictionary *dataDic = resultDic[@"data"];
    NSDictionary *sdpDic = dataDic[@"sdp"];
    NSString *sdp = sdpDic[@"sdp"];
    NSString *type = sdpDic[@"type"];
    NSString *connectionID = dataDic[@"socketId"];
    RTCSdpType sdpType = [self _typeForString:type];
    RTCPeerConnection *connection = [_peerConnections objectForKey:connectionID];
    RTCSessionDescription *remoteSdp = [[RTCSessionDescription alloc] initWithType:sdpType sdp:sdp];
    
    if (sdpType == RTCSdpTypeAnswer) {
        __weak __typeof(connection) wConnection = connection;
        /** Apply the supplied RTCSessionDescription as the remote description. */
        // A:设置连接对端 SDP
        [connection setRemoteDescription:remoteSdp completionHandler:^(NSError * _Nullable error) {
            if (error) {
                NSLog(@"setRemoteDescription error");
            }
            
            [self _didSetSessionDescription:wConnection];
        }];
    }
}

接收到对端的’__answer’消息后，解析出消息的结构。 将对端的sdp信息设置为本端的remote sdp。 这时的代码处理如下：

if (connection.signalingState == RTCSignalingStateHaveRemoteOffer) { // 新人进入房间就调(远端发起 offer)
        /** Generate an SDP answer. */
        [connection answerForConstraints:[self _setupForOfferOrAnswerConstraint] completionHandler:^(RTCSessionDescription * _Nullable sdp, NSError * _Nullable error) {
            if (error) {
                NSLog(@"answerForConstraints error");
                return;
            }
            
            if (sdp.type == RTCSdpTypeAnswer) {
                __weak __typeof(connection) wConnection = connection;
                // B:设置连接本端 SDP
                [connection setLocalDescription:sdp completionHandler:^(NSError * _Nullable error) {
                    if (error) {
                        NSLog(@"setLocalDescription error");
                        return;
                    }
         
                    [self _didSetSessionDescription:wConnection];
                }];
            }
        }];
    }

通过RTCPeerConnection对象的answerForConstraints: 生成响应端（本端）的sdp. 并通过 setLocalDescription:设置为本端的sdp

未完待续

欢迎分享，转载请注明来源：内存溢出

原文地址: http://outofmemory.cn/web/997130.html

赞 (0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
image image 一级用户组
0 0
上一篇 2022-05-21
下一篇 2022-05-21

发表评论

登录后才能评论

评论列表（0条）

保存