We make video calls in the iOS application (using the example of a child monitor and without WebRTC)

  • Tutorial
imageIn this post we will talk about how to write an application - baby monitor, when you install one device (tablet) near the baby’s bed, and the second (phone), take it with you, say to the kitchen, and from time to time look at the child through the screen .

As a newly made parent, I want to say that such an application saves a lot of nerves - you do not need to listen to every rustle or scream from the street, you can make sure with a glance that everything is in order with the child. A little about the technical part: the application uses our library of iOS video chat, including the server side (signaling and TURN server for NAT traversal), this is all in the public domain. The video stream will work both through Wi-Fi and through 2G / 3G / 4G. In appstore, until recently, there was no baby monitor application that would work through the mobile internet (apparently because of difficulties with the NAT traversal), but we prokrastinirovali preparing the post, one of the leaders of the applications released a paid version that supports this functionality. In any case, the article will be useful to you if you want to record video monitoring or two-way video call in your iOS application. We specifically point out that this is a version without WebRTC, because we are going to write about the web-compatible version (as well as about Android) separately, it has its own nuances.

TK:
In our case, the application is monitoring small children (infants) through a mobile device running iOS. At startup, the application had to find a neighboring device, synchronize with it, and then make a video call. During the connection, the parent sees the child, and can also control the device on the other side - turn on the light (flash), play the lullaby, talk into the microphone there.

Actually, the project is not difficult, the main difficulties lay in the implementation of 2 points:
  • device search and synchronization
  • video calling

Let's consider these points in more detail:

Search and sync devices


Synchronization occurs via Wi-Fi or Bluetooth. Googling, found 4 ways to do this. Here is a brief description, advantages and disadvantages:

  1. Bonjour service - Wi-Fi synchronization. It is not difficult to find such a sample on the Internet. Works on iOS 6-7
  2. Core Bluetooth - it works, no matter how unexpected it may sound, over the Bluetooth channel with iOS 5 and above. But here's the nuance - only Bluetooth 4 LE is supported.
  3. GameKit . Cool stuff. Basically, everything is just like a door. It works fine on regular bluetooth (for iPhone 4 devices and even lower). Bonjour also works - and for WiFi networks. But there is a small drawback - deprecated since iOS 7.
  4. Multipeer Connectivity is a new framework added in iOS 7. In fact, for us it looked like an analogue of GameKit, only for iOS 7. We used it in the future.

We were able to encapsulate these services under one interface and no matter which of these four services we chose to use in the end. It turned out to be very convenient.

The general interface of such a service looks like this (the prefix “BB” is from our application name, of course, you can call it something else):

#import "BBDeviceModel.h"
typedef enum CommonServiсeType {
	CommonServiceTypeBonjour = 0,
	CommonServiceTypeBluetoothLE,
	CommonServiceTypeGameKitWiFi,
	CommonServiceTypeGameKitBluetooth,
	CommonServiceTypeMultipeer,
}CommonServiсeType;
@protocol BBCommonServiceDelegate;
@interface BBCommonService : NSObject
@property (nonatomic, weak) id delegate;
-(void) setConnectionType:(CommonServiсeType)type;
-(void) startServerSide;
-(void) stopServerSide;
-(void) startSearchDevices;
-(void) stopSearchDevices;
-(void) selectDevice:(BBDeviceModel *)deviceModel;
-(void) clean;
@end
@protocol BBCommonServiceDelegate 
@optional
-(void) service:(BBCommonService *)serviсe didFindDevice:(BBDeviceModel *)device;
-(void) service:(BBCommonService *)serviсe didRemoveDevice:(BBDeviceModel *)device;
-(void) service:(BBCommonService *)serviсe serverDidFinishedSync:(BOOL)isFinished;
-(void) service:(BBCommonService *)serviсe clientDidFinishedSync:(BOOL)isFinished;
@end
@interface BBDeviceModel : NSObject
@property (nonatomic, strong)	id		device;
-(NSString *)deviceName;
-(void)setDeviceName:(NSString *)name;
-(BOOL) isDeviceEqualTo:(id)otherDevice;
@end

Next, we inherit from BBCommonService depending on the type of connection and redefine the start- and stop-, clean methods, and then call the delegate methods in the right places.

Video calling


For video communications, we used QuickBlox. First you need to register - as a result of which you will get access to the admin panel. In it you create your application. Next, download the framework itself from the official site . Connection is described in more detail here - http://quickblox.com/developers/IOS-how-to-connect-Quickblox-framework . In short, then:

1) download Quickblox.framework, add 15 libraries to the project, connect them to the tutorial
2) After that, you need to return to the admin panel, select your application and copy three parameters - Application id, Authorization key and Authorization secret in project settings:

[QBSettings setApplicationID:APP_ID];
[QBSettings setAuthorizationKey:AUTH_KEY];
[QBSettings setAuthorizationSecret:AUTH_SECRET];

Everything, now you can work.

1. Session

In order to make client-server interactions with QuickBlox, you need to create a session. This is done very simply:

[QBAuth createSessionWithDelegate:self];

Thus, a request to create a session is sent and the response comes to the delegate method:

- (void)completedWithResult:(Result *)result {
	QBAAuthResult *authResult = (QBAAuthResult *)result;
	if ([authResult isKindOfClass:[QBAAuthResult class]]) {
	// do something	
	}
} 


2. Creating a user or login.

For further work, we need a user. Nowhere without him. This is also done quite simply:

// registration
QBUUser *user	= [QBUUser new];
	user.password	= aPass;
	user.login		= aLogin;
	[QBUsers signUp:user
		   delegate:self];

or

// login
[QBUsers logInWithUserLogin:aLogin
                       password:aPass
                       delegate:self];

For these and all requests, the response from the server comes to the delegate method. completedWithResult:

Accordingly, the login / password can be taken from UITextFields if desired. In our case, in order not to force the user to enter something else additionally, we did hidden authorization, so we created a username and password based on vendorID.

3. Storage of pair information

After synchronization, we decided to create a Pair entity in which to store our id and opponent (the second device synchronized with the data). Also, it did not stop sending somewhere to the server, so as not to do synchronization in the future. The Custom Objects module helped us with this , which is essentially a database with custom fields. So, it looked like this:

QBCOCustomObject *customObject = [QBCOCustomObject customObject];
	customObject.className = @"Pair";
	// Object fields
	[customObject.fields setObject:@(userID) forKey:@“opponentID”];
	customObject.userID = self.currentUser.ID;
	// permissions
	QBCOPermissions *permissions = [QBCOPermissions permissions];
	permissions.readAccess = QBCOPermissionsAccessOpen;
	permissions.updateAccess = QBCOPermissionsAccessOpen;
	customObject.permissions = permissions;
    [QBCustomObjects createObject:customObject
                         delegate:self];
- (void)completedWithResult:(Result *)result {
	QBCOCustomObjectResult *coResult = (QBCOCustomObjectResult *)result;
	if ([authResult isKindOfClass:[QBCOCustomObjectResult class]]) {
		// do something	
		QBCOCustomObjectResult *customObjectResult = (QBCOCustomObjectResult *)result;
		BBPair *pair = [BBPair createEntityFromData:customObjectResult.object];
		self.currentPair = pair;
		// .. 
	}
} 

The only thing is that you need to go to the admin panel and create the appropriate model with fields in the Custom Objects tab. Everything is very simple and intuitive there, so I will not give an example (what you need to pay attention to are the supported data types for fields - integer, float, boolean, string, file).

If you need to get any entities from the database, this is done as follows -

NSMutableDictionary *getRequest = [NSMutableDictionary dictionary];
[getRequest setObject:@(self.currentUser.ID) forKey:@"user_id[or]"];
[getRequest setObject:@(self.currentUser.ID) forKey:@"opponentID[or]"];
[QBCustomObjects objectsWithClassName:@"Pair"
						  extendedRequest:getRequest
								 delegate:self];

This query searches for all entities where the given user is either the current user or opponent.

Removing a custom object is even simpler - you just need to know its ID.

NSString *className = @"Pair";
[QBCustomObjects deleteObjectWithID:self.currentPair.pairID
							  className:className
							   delegate:self];

For us, this is necessary when the user wants to unsynchronize his ipad / ipod / iphone then, for example, to connect it later with another device. In the application, we provided for this the “Unpair” button in the Settings interface.

4. Broadcast video

Here is a little more complicated. Firstly, in addition to creating a session and login, we must also log in to the chat, as The chat server is used for video signaling. This is done as follows -

[QBChat instance].delegate = self;
[[QBChat instance] loginWithUser:self.currentUser];   // self.currentUser - QBUUser

so we take the current user and log in to the chat, after setting the delegate. If all is well, then one of the methods will work almost immediately:

-(void)chatDidLogin {
	self.presenceTimer = [NSTimer scheduledTimerWithTimeInterval:30 target:[QBChat instance] selector:@selector(sendPresence) userInfo:nil repeats:YES];
}
-(void)chatDidNotLogin {
}
-(void)chatDidFailWithError:(NSInteger)code {
}

depending on the outcome. Also, we immediately hang a presence on the timer in case of a successful login. Without them, we will automatically go offline somewhere in a minute.

If everything went well, then you can proceed to the hardest part. To work with video communication, we are offered the QBVideoChat class.

The caller first creates an instance using

self.videoChat = [[QBChat instance] createAndRegisterVideoChatInstance];

We configure the view for ourselves and the opponent, if necessary, the sound state (on / off) and additional settings - for example, useBackCamera:

self.videoChat.viewToRenderOwnVideoStream = myView;
self.videoChat.viewToRenderOwnVideoStream = opponentView;
self.videoChat.useHeadphone = NO;
self.videoChat.useBackCamera = NO;
self.videoChat.microphoneEnabled = YES;

and make a call:

[self.videoChat callUser:currentPair.opponentID conferenceType:QBVideoChatConferenceTypeAudioAndVideo];


The next step is to implement the delegate methods according to the behavior. If everything is successful, the opponent must work out the following method:

-(void) chatDidReceiveCallRequestFromUser:(NSUInteger)userID withSessionID:(NSString *)_sessionID conferenceType:(enum QBVideoChatConferenceType)conferenceType {
	self.videoChat = [[QBChat instance] createAndRegisterVideoChatInstanceWithSessionID:_sessionID];
    // video chat setup
	self.videoChat.viewToRenderOwnVideoStream = nil;
	self.videoChat.useHeadphone = NO;
	self.videoChat.useBackCamera = NO;
	if (self.videoSide == BBVideoParentSide) {
		self.videoChat.viewToRenderOpponentVideoStream = self.renderView;
		self.videoChat.viewToRenderOwnVideoStream = nil;
		self.videoChat.microphoneEnabled = NO;
	}else if (self.videoSide == BBVideoChildSide) {
		self.videoChat.viewToRenderOpponentVideoStream = nil;
		self.videoChat.viewToRenderOwnVideoStream = self.renderView;
		self.videoChat.microphoneEnabled = NO;
	}
	BBPair *currentPair = [QBClient shared].currentPair;
	[self.videoChat acceptCallWithOpponentID:currentPair.opponentID conferenceType:QBVideoChatConferenceTypeAudioAndVideo];
}

or

-(void) chatCallUserDidNotAnswer:(NSUInteger)userID {
}

We hope for a successful outcome :) In our case, we specifically set the view and sound from our side. Everything is the same here as in the beginning, with the only difference being that in the end we send accept to the call initiator.

His method should work

-(void) chatCallDidAcceptByUser:(NSUInteger)userID {	
}

And then it will work on both sides

-(void)chatCallDidStartWithUser:(NSUInteger)userID sessionID:(NSString *)sessionID {
}

This method is useful for the UI - for example, you spin a spinner while this whole thing is happening, and then you hide it in this method. From this moment, video calling should work.

When you need to end the session and “hang up” - call

[self.videoChat finishCall]; 

after which the delegate method on the opposite side is triggered

-(void)chatCallDidStopByUser:(NSUInteger)userID status:(NSString *)status {
}

There is also a version of this method with parameters, in case you need to transfer something else.

In this case, a standard audio-video session is used. Depending on the TOR — if you need to record video and audio, for example, and then do something with it — then you better use custom audio-video sessions. SDK allows this. This article is not considered, but can be read in more detail here: http://quickblox.com/developers/SimpleSample-videochat-ios#Use_custom_capture_session

So, the video connection is established. Now the last thing to do is to implement the inclusion of a lullaby on the baby’s device, change the camera, take a screenshot, etc.
All this is done quite simply. Remember we logged in additionally in a chat? so - this is another module, it is called - Chat :)
In it you can send messages. What we will do is simply send different messages, and parse them on the opponent’s side and, depending on the message, take some actions - turn on the flash, for example, or something else.

Sending a message is simple (we put it in a separate method) -

-(void) sendServiceMessageWithText:(NSString *)text parameters:(NSMutableDictionary *)parameters{
	BBPair *currentPair = [QBClient shared].currentPair;
	QBChatMessage *message = [QBChatMessage new];
	message.text = text;
	message.senderID = currentPair.myID;
	message.recipientID = currentPair.opponentID;
	if (parameters)
		message.customParameters = parameters;
	[[QBChat instance] sendMessage:message];
}

Text is our message type in this case.

A message comes here -

-(void)chatDidReceiveMessage:(QBChatMessage *)message {
	if ([message.text isEqualToString:kRouteFlashMessage]) {
	// .. do something
	}else if ([message.text isEqualToString:kRouteCameraMessage]) {
	// .. 
	}
}

Everything else is the UI and some additional features. In general, everything turned out well. In the end, I would like to draw attention to two nuances:

1) the life of the session is 2 hours. It is automatically renewed after each completed request. But if for example the user minimized the application for half a day, then it needs to be restored somehow. This is not difficult - with the help of extended request:

QBASessionCreationRequest *extendedRequest = [QBASessionCreationRequest new];
extendedRequest.userLogin = self.currentUser.login;
extendedRequest.userPassword = self.currentUser.password;
[QBAuth createSessionWithExtendedRequest:extendedRequest
									delegate:self];

You can run, for example, in applicationWillEnterForeground.

2) The method - (void)completedWithResult:(Result *)resultgrows very quickly, which becomes quite inconvenient. Almost every method is in 2 versions - simple and with context. Alternatively, you can use blocks - pass them as a context. Here's what it looks like when creating a session:

typedef void (^qbSessionBlock)(BOOL isCreated, NSError *anError);
-(void) createSessionWithBlock:(qbSessionBlock)aBlock {
    void (^block)(Result *) = ^(Result *result) {
        if (!result.success) {
			aBlock(NO, [self errorWithQBResult:result]);
        } else {
			aBlock(YES, nil);
        }
    };
    [QBAuth createSessionWithDelegate:self
                              context:(__bridge_retained void *)(block)];
} 
- (void)completedWithResult:(Result *)result
                    context:(void *)contextInfo {
    void(^myBlock)(Result *result) = (__bridge void (^)(Result *__strong))(contextInfo);
    myBlock(result);
    Block_release(contextInfo);
}

So much easier.

That, in principle, is all. If that is not clear - write in a personal or comments. Here you can add that the application discussed in this article was recommended by Apple, came out in the US Appstore in first place and out of three paid in-app purchases, the video monitor function turned out to be the most popular. We work a lot on applications related to video calls for iOS, Android, Web - usually it is dating / social networks or security / video surveillance, so I will be happy to help with advice or code examples if you do something like that.

Also popular now: