Category: webrtc

WebRTC – Mobile Support ”; Previous Next In the mobile world, the WebRTC support is not on the same level as it is on desktops. Mobile devices have their own way, so WebRTC is also something different on the mobile platforms. When developing a WebRTC application for desktop, we consider using Chrome, Firefox or Opera. All of them support WebRTC out of the box. In general, you just need a browser and not bother about the desktop”s hardware. In the mobile world there are three possible modes for WebRTC today − The native application The browser application The native browser Android In 2013, the Firefox web browser for Android was presented with WebRTC support out of the box. Now you can make video calls on Android devices using the Firefox mobile browser. It has three main WebRTC components − PeerConnection − enables calls between browsers getUserMedia − provides access to the camera and microphone DataChannels − provides peer-to-peer data transfer Google Chrome for Android provides WebRTC support as well. As you”ve already noticed, the most interesting features usually first appear in Chrome. In the past year, the Opera mobile browser appeared with WebRTC support. So for Android you have Chrome, Firefox, and Opera. Other browsers don”t support WebRTC. iOS Unfortunately, WebRTC is not supported on iOS now. Although WebRTC works well on Mac when using Firefox, Opera, or Chrome, it is not supported on iOS. Nowadays, your WebRTC application won”t work on Apple mobile devices out of the box. But there is a browser − Bowser. It is a web browser developed by Ericsson and it supports WebRTC out of the box. You can check its homepage at http://www.openwebrtc.org/bowser/. Today, it is the only friendly way to support your WebRTC application on iOS. Another way is to develop a native application yourself. Windows Phones Microsoft doesn”t support WebRTC on mobile platforms. But they have officially confirmed that they are going to implement ORTC (Object Realtime Communications) in future versions of IE. They are not planning to support WebRTC 1.0. They labeled their ORTC as WebRTC 1.1, although it is just a community enhancement and not the official standard. So today Window Phone users can”t use WebRTC applications and there is no way to beat this situation. Blackberry WebRTC applications are not supported on Blackberry either, in any way. Using a WebRTC Native Browser The most convenient and comfortable case for users to utilize WebRTC is using the native browser of the device. In this case, the device is ready to work any additional configurations. Today only Android devices that are version 4 or higher provide this feature. Apple still doesn”t show any activity with WebRTC support. So Safari users can”t use WebRTC applications. Microsoft also did not introduce it in Windows Phone 8. Using WebRTC via Browser Applications This means using a third-party applications (non-native web browsers) in order to provide the WebRTC features. For now, there are two such third-party applications. Bowser, which is the only way to bring WebRTC features to the iOS device and Opera, which is a nice alternative for Android platform. The rest of the available mobile browsers don”t support WebRTC. Native Mobile Applications As you can see, WebRTC does not have a large support in the mobile world yet. So, one of the possible solutions is to develop a native applications that utilize the WebRTC API. But it is not the better choice because the main WebRTC feature is a cross-platform solution. Anyway, in some cases this is the only way because a native application can utilize device-specific functions or features that are not supported by HTML5 browsers. Constraining Video Stream for Mobile and Desktop Devices The first parameter of the getUserMedia API expects an object of keys and values telling the browser how to process streams. You can check the full set of constraints at https://tools.ietf.org/html/draft-alvestrand-constraints-resolution-03. You can setup video aspect ration, frameRate, and other optional parameters. Supporting mobile devices is one of the biggest pains because mobile devices have limited screen space along with limited resources. You might want the mobile device to only capture a 480×320 resolution or smaller video stream to save power and bandwidth. Using the user agent string in the browser is a good way to test whether the user is on a mobile device or not. Let”s see an example. Create the index.html file − <!DOCTYPE html> <html lang = “en”> <head> <meta charset = “utf-8” /> </head> <body> <video autoplay></video> <script src = “client.js”></script> </body> </html> Then create the following client.js file − //constraints for desktop browser var desktopConstraints = { video: { mandatory: { maxWidth:800, maxHeight:600 } }, audio: true }; //constraints for mobile browser var mobileConstraints = { video: { mandatory: { maxWidth: 480, maxHeight: 320, } }, audio: true } //if a user is using a mobile browser if(/Android|iPhone|iPad/i.test(navigator.userAgent)) { var constraints = mobileConstraints; } else { var constraints = desktopConstraints; } function hasUserMedia() { //check if the browser supports the WebRTC return !!(navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia); } if (hasUserMedia()) { navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia; //enabling video and audio channels navigator.getUserMedia(constraints, function (stream) { var video = document.querySelector(”video”); //inserting our stream to the video tag video.src = window.URL.createObjectURL(stream); }, function (err) {}); } else { alert(“WebRTC is not supported”); } Run the web server using the static command and open the page. You should see it is 800×600. Then open this page in a mobile viewport using chrome tools and check the resolution. It should be 480×320. Constraints are the easiest way to increase the performance of your WebRTC application. Summary In this chapter, we learned about the issues that can occur when developing WebRTC applications for mobile devices. We discovered different limitations of supporting the WebRTC API on mobile platforms. We also launched a demo application where we set different constraints for desktop and mobile browsers. Print Page Previous Next

WebRTC – RTCPeerConnection APIs ”; Previous Next The RTCPeerConnection API is the core of the peer-to-peer connection between each of the browsers. To create the RTCPeerConnection objects simply write var pc = RTCPeerConnection(config); where the config argument contains at least on key, iceServers. It is an array of URL objects containing information about STUN and TURN servers, used during the finding of the ICE candidates. You can find a list of available public STUN servers at code.google.com Depending upon whether you are the caller or the callee the RTCPeerConnection object is used in a slightly different way on each side of the connection. Here is an example of the user”s flow − Register the onicecandidate handler. It sends any ICE candidates to the other peer, as they are received. Register the onaddstream handler. It handles the displaying of the video stream once it is received from the remote peer. Register the message handler. Your signaling server should also have a handler for messages received from the other peer. If the message contains the RTCSessionDescription object, it should be added to the RTCPeerConnection object using the setRemoteDescription() method. If the message contains the RTCIceCandidate object, it should be added to the RTCPeerConnection object using the addIceCandidate() method. Utilize getUserMedia() to set up your local media stream and add it to the RTCPeerConnection object using the addStream() method. Start offer/answer negotiation process. This is the only step where the caller”s flow is different from the callee”s one. The caller starts negotiation using the createOffer() method and registers a callback that receives the RTCSessionDescription object. Then this callback should add this RTCSessionDescription object to your RTCPeerConnection object using setLocalDescription(). And finally, the caller should send this RTCSessionDescription to the remote peer using the signaling server. The callee, on the other, registers the same callback, but in the createAnswer() method. Notice that the callee flow is initiated only after the offer is received from the caller. RTCPeerConnection API Properties RTCPeerConnection.iceConnectionState (read only) − Returns an RTCIceConnectionState enum that describes the state of the connection. An iceconnectionstatechange event is fired when this value changes. The possible values − new − the ICE agent is waiting for remote candidates or gathering addresses checking − the ICE agent has remote candidates, but it has not found a connection yet connected − the ICE agent has found a usable connection, but is still checking more remote candidate for better connection. completed − the ICE agent has found a usable connection and stopped testing remote candidates. failed − the ICE agent has checked all the remote candidates but didn”t find a match for at least one component. disconnected − at least one component is no longer alive. closed − the ICE agent is closed. RTCPeerConnection.iceGatheringState (read only) − Returns a RTCIceGatheringState enum that describes the ICE gathering state for the connection − new − the object was just created. gathering − the ICE agent is in the process of gathering candidates complete the ICE agent has completed gathering. RTCPeerConnection.localDescription (read only) − Returns an RTCSessionDescription describing the local session. It can be null if it has not yet been set. RTCPeerConnection.peerIdentity (read only) − Returns an RTCIdentityAssertion. It consists of an idp(domain name) and a name representing the identity of the remote peer. RTCPeerConnection.remoteDescription (read only) − Return an RTCSessionDescription describing the remote session. It can be null if it has not yet been set. RTCPeerConnection.signalingState (read only) − Returns an RTCSignalingState enum that describes the signaling state of the local connection. This state describes the SDP offer. A signalingstatechange event is fired when this value changes. The possible values − stable − The initial state. There is no SDP offer/answer exchange in progress. have-local-offer − the local side of the connection has locally applied a SDP offer. have-remote-offer − the remote side of the connection has locally applied a SDP offer. have-local-pranswer − a remote SDP offer has been applied, and a SDP pranswer applied locally. have-remote-pranswer − a local SDP has been applied, and a SDP pranswer applied remotely. closed − the connection is closed. Event Handlers Given below are the commonly used Event Handlers of RTCPeerConnection. S.No. Event Handlers & Description 1 RTCPeerConnection.onaddstream This handler is called when the addstream event is fired. This event is sent when a MediaStream is added to this connection by the remote peer. 2 RTCPeerConnection.ondatachannel This handler is called when the datachannel event is fired. This event is sent when a RTCDataChannel is added to this connection. 3 RTCPeerConnection.onicecandidate This handler is called when the icecandidate event is fired. This event is sent when a RTCIceCandidate object is added to the script. 4 RTCPeerConnection.oniceconnectionstatechange This handler is called when the iceconnectionstatechange event is fired. This event is sent when the value of iceConnectionState changes. 5 RTCPeerConnection.onidentityresult This handler is called when the identityresult event is fired. This event is sent when an identity assertion is generated during the creating of an offer or an answer of via getIdentityAssertion(). 6 RTCPeerConnection.onidpassertionerror This handler is called when the idpassertionerror event is fired. This event is sent when the IdP (Identitry Provider) finds an error while generating an identity assertion. 7 RTCPeerConnection.onidpvalidation This handler is called when the idpvalidationerror event is fired. This event is sent when the IdP (Identitry Provider) finds an error while validating an identity assertion. 8 RTCPeerConnection.onnegotiationneeded This handler is called when the negotiationneeded event is fired. This event is sent by the browser to inform the negotiation will be required at some point in the future. 9 RTCPeerConnection.onpeeridentity This handler is called when the peeridentity event is fired. This event is sent when a peer identity has been set and verified on this connection. 10 RTCPeerConnection.onremovestream This handler is called when the signalingstatechange event is fired. This event is sent when the value of signalingState changes. 11 RTCPeerConnection.onsignalingstatechange This handler is called when the removestream event is fired. This event is sent when a MediaStream is removed from this connection. Methods Given below are

WebRTC – Overview ”; Previous Next The Web is no more a stranger to real-time communication as WebRTC (Web Real-Time Communication) comes into play. Although it was released in May 2011, it is still developing and its standards are changing. A set of protocols is standardized by Real-Time Communication in WEB-browsers Working group at http://tools.ietf.org/wg/rtcweb/ of the IETF (Internet Engineering Task Force) while new sets of APIs are standardized by the Web Real-Time Communications Working Groupe at http://www.w3.org/2011/04/webrtc/ of the W3C (World Wide Web Consortium). With the appearance of WebRTC, modern web applications can easily stream audio and video content to millions of people. Basic Scheme WebRTC allows you to set up peer-to-peer connections to other web browsers quickly and easily. To build such an application from scratch, you would need a wealth of frameworks and libraries dealing with typical issues like data loss, connection dropping, and NAT traversal. With WebRTC, all of this comes built-in into the browser out-of-the-box. This technology doesn”t need any plugins or third-party software. It is open-sourced and its source code is freely available at http://www.webrtc.org/. The WebRTC API includes media capture, encoding and decoding audio and video, transportation layer, and session management. Media Capture The first step is to get access to the camera and microphone of the user”s device. We detect the type of devices available, get user permission to access these devices and manage the stream. Encoding and Decoding Audio and Video It is not an easy task to send a stream of audio and video data over the Internet. This is where encoding and decoding are used. This is the process of splitting up video frames and audio waves into smaller chunks and compressing them. This algorithm is called codec. There is an enormous amount of different codecs, which are maintained by different companies with different business goals. There are also many codecs inside WebRTC like H.264, iSAC, Opus and VP8. When two browsers connect together, they choose the most optimal supported codec between two users. Fortunately, WebRTC does most of the encoding behind the scenes. Transportation Layer The transportation layer manages the order of packets, deal with packet loss and connecting to other users. Again the WebRTC API gives us an easy access to events that tell us when there are issues with the connection. Session Management The session management deals with managing, opening and organizing connections. This is commonly called signaling. If you transfer audio and video streams to the user it also makes sense to transfer collateral data. This is done by the RTCDataChannel API. Engineers from companies like Google, Mozilla, Opera and others have done a great job to bring this real-time experience to the Web. Browser Compatibility The WebRTC standards are one of the fastest evolving on the web, so it doesn”t mean that every browser supports all the same features at the same time. To check whether your browser supports WebRTC or not, you may visit http://caniuse.com/#feat=rtcpeerconnection. Throughout all the tutorials, I recommend you to use Chrome for all the examples. Trying out WebRTC Let”s get started using WebRTC right now. Click the “JOIN” button. You should see a drop-down notification. Click the “Allow” button to start streaming your video and audio to the web page. You should see a video stream of yourself. Now open the URL you are currently on in a new browser tab and click on “JOIN”. You should see two video streams − one from your first client and another from the second one. Now you should understand why WebRTC is a powerful tool. Use Cases The real-time web opens the door to a whole new range of applications, including text-based chat, screen and file sharing, gaming, video chat, and more. Besides communication you can use WebRTC for other purposes like − real-time marketing real-time advertising back office communications (CRM, ERP, SCM, FFM) HR management social networking dating services online medical consultations financial services surveillance multiplayer games live broadcasting e-learning Summary Now you should have a clear understanding of the term WebRTC. You should also have an idea of what types of applications can be built with WebRTC, as you have already tried it in your browser. To sum up, WebRTC is quite a useful technology. Print Page Previous Next Advertisements ”;

WebRTC – Signaling ”; Previous Next Most WebRTC applications are not just being able to communicate through video and audio. They need many other features. In this chapter, we are going to build a basic signaling server. Signaling and Negotiation To connect to another user you should know where he is located on the Web. The IP address of your device allows Internet-enabled devices to send data directly between each other. The RTCPeerConnection object is responsible for this. As soon as devices know how to find each other over the Internet, they start exchanging data about which protocols and codecs each device supports. To communicate with another user you simply need to exchange contact information and the rest will be done by WebRTC. The process of connecting to the other user is also known as signaling and negotiation. It consists of a few steps − Create a list of potential candidates for a peer connection. The user or an application selects a user to make a connection with. The signaling layer notifies another user that someone want to connect to him. He can accept or decline. The first user is notified of the acceptance of the offer. The first user initiates RTCPeerConnection with another user. Both users exchange software and hardware information through the signaling server. Both users exchange location information. The connection succeeds or fails. The WebRTC specification does not contain any standards about exchanging information. So keep in mind that the above is just an example of how signaling may happen. You can use any protocol or technology you like. Building the Server The server we are going to build will be able to connect two users together who are not located on the same computer. We will create our own signaling mechanism. Our signaling server will allow one user to call another. Once a user has called another, the server passes the offer, answer, ICE candidates between them and setup a WebRTC connection. The above diagram is the messaging flow between users when using the signaling server. First of all, each user registers with the server. In our case, this will be a simple string username. Once users have registered, they are able to call each other. User 1 makes an offer with the user identifier he wishes to call. The other user should answers. Finally, ICE candidates are sent between users until they can make a connection. To create a WebRTC connection clients have to be able to transfer messages without using a WebRTC peer connection. This is where we will use HTML5 WebSockets – a bidirectional socket connection between two endpoints – a web server and a web browser. Now let”s start using the WebSocket library. Create the server.js file and insert the following code − //require our websocket library var WebSocketServer = require(”ws”).Server; //creating a websocket server at port 9090 var wss = new WebSocketServer({port: 9090}); //when a user connects to our sever wss.on(”connection”, function(connection) { console.log(“user connected”); //when server gets a message from a connected user connection.on(”message”, function(message){ console.log(“Got message from a user:”, message); }); connection.send(“Hello from server”); }); The first line requires the WebSocket library which we have already installed. Then we create a socket server on the port 9090. Next, we listen to the connection event. This code will be executed when a user makes a WebSocket connection to the server. We then listen to any messages sent by the user. Finally, we send a response to the connected user saying “Hello from server”. Now run node server and the server should start listening for socket connections. To test our server, we”ll use the wscat utility which we also have already installed. This tool helps in connecting directly to the WebSocket server and test out commands. Run our server in one terminal window, then open another and run the wscat -c ws://localhost:9090 command. You should see the following on the client side − The server should also log the connected user − User Registration In our signaling server, we will use a string-based username for each connection so we know where to send messages. Let”s change our connection handler a bit − connection.on(”message”, function(message) { var data; //accepting only JSON messages try { data = JSON.parse(message); } catch (e) { console.log(“Invalid JSON”); data = {}; } }); This way we accept only JSON messages. Next, we need to store all connected users somewhere. We will use a simple Javascript object for it. Change the top of our file − //require our websocket library var WebSocketServer = require(”ws”).Server; //creating a websocket server at port 9090 var wss = new WebSocketServer({port: 9090}); //all connected to the server users var users = {}; We are going to add a type field for every message coming from the client. For example if a user wants to login, he sends the login type message. Let”s define it − connection.on(”message”, function(message){ var data; //accepting only JSON messages try { data = JSON.parse(message); } catch (e) { console.log(“Invalid JSON”); data = {}; } //switching type of the user message switch (data.type) { //when a user tries to login case “login”: console.log(“User logged:”, data.name); //if anyone is logged in with this username then refuse if(users[data.name]) { sendTo(connection, { type: “login”, success: false }); } else { //save user connection on the server users[data.name] = connection; connection.name = data.name; sendTo(connection, { type: “login”, success: true }); } break; default: sendTo(connection, { type: “error”, message: “Command no found: ” + data.type }); break; } }); If the user sends a message with the login type, we − Check if anyone has already logged in with this username If so, then tell the user that he hasn”t successfully logged in If no one is using this username, we add username as a key to the connection object. If a command is not recognized we send an error. The following code is a helper function for sending messages to a connection. Add it to the server.js file − function

WebRTC – Browser Support ”; Previous Next The Web is moving so fast and it is always improving. New standards are created every day. Browsers allow updates to be installed without the user ever knowing, so you should keep up with what is going on in the world of the Web and WebRTC. Here is an overview of what this is up to today. Browser Support Every browser doesn”t have all the same WebRTC features at the same time. Different browsers may be ahead of the curve, which makes some WebRTC features work in one browser and not another. The current support for WebRTC in the browser is shown in the following picture. You can check an up-to-date WebRTC support status at http://caniuse.com/#feat=rtcpeerconnection. Chrome, Firefox, and Opera The latest versions of Chrome, Firefox, and Opera on mainstream PC operating systems such as Mac OS X, Windows, and Linux, all support WebRTC out-of-the-box. And most importantly, the engineers from Chrome and Firefox developer teams have been working together to fix issues so these two browsers could communicate with each other easily. Android OS On Android operating systems, WebRTC applications for Chrome and Firefox should work outof-the-box. They are able to work with other browsers after Android Ice Cream Sandwich version (4.0). This is due to the code sharing between desktop and mobile versions. Apple Apple has not yet made any announcement about their plans to support WebRTC in Safari on OS X. One of the possible workarounds for hybrid native iOS applications os to embed the WebRTC code directly into the application and load this app into a WebView. Internet Explorer Microsoft doesn”t support WebRTC on desktops. But they have officially confirmed that they are going to implement ORTC (Object Realtime Communications) in future versions of IE(Edge). They are not planning to support WebRTC 1.0. They labeled their ORTC as WebRTC 1.1, although it is just a community enhancement and not the official standard. Recently they”ve added the ORTC support to the latest Microsoft Edge version. You may learn more at https://blogs.windows.com/msedgedev/2015/09/18/ortc-api-is-now-available-in-microsoftedge/. Summary Notice that WebRTC is a collection of APIs and protocols, not a single API. The support for each of these is developing on different browsers and operating systems at a different level. A great way to check the latest level of support is through http://canisue.com. It tracks adoption of modern APIs across multiple browsers. You can also find the latest information on browser supports as well as WebRTC demos at http://www.webrtc.org, which is supported by Mozilla, Google, and Opera. Print Page Previous Next Advertisements ”;

WebRTC – Architecture ”; Previous Next The overall WebRTC architecture has a great level of complexity. Here you can find three different layers − API for web developers − this layer contains all the APIs web developer needed, including RTCPeerConnection, RTCDataChannel, and MediaStrean objects. API for browser makers Overridable API, which browser makers can hook. Transport components allow establishing connections across various types of networks while voice and video engines are frameworks responsible for transferring audio and video streams from a sound card and camera to the network. For Web developers, the most important part is WebRTC API. If we look at the WebRTC architecture from the client-server side we can see that one of the most commonly used models is inspired by the SIP(Session Initiation Protocol) Trapezoid. In this model, both devices are running a web application from different servers. The RTCPeerConnection object configures streams so they could connect to each other, peer-to-peer. This signaling is done via HTTP or WebSockets. But the most commonly used model is Triangle − In this model both devices use the same web application. It gives web developer more flexibility when managing user connections. The WebRTC API It consists of a few main javascript objects − RTCPeerConnection MediaStream RTCDataChannel The RTCPeerConnection object This object is the main entry point to the WebRTC API. It helps us connect to peers, initialize connections and attach media streams. It also manages a UDP connection with another user. The main task of the RTCPeerConnection object is to setup and create a peer connection. We can easily hook keys points of the connection because this object fires a set of events when they appear. These events give you access to the configuration of our connection − The RTCPeerConnection is a simple javascript object, which you can simply create this way − [code] var conn = new RTCPeerConnection(conf); conn.onaddstream = function(stream) { // use stream here }; [/code] The RTCPeerConnection object accepts a conf parameter, which we will cover later in these tutorials. The onaddstream event is fired when the remote user adds a video or audio stream to their peer connection. MediaStream API Modern browsers give a developer access to the getUserMedia API, also known as the MediaStream API. There are three key points of functionality − It gives a developer access to a stream object that represent video and audio streams It manages the selection of input user devices in case a user has multiple cameras or microphones on his device It provides a security level asking user all the time he wants to fetch s stream To test this API let”s create a simple HTML page. It will show a single <video> element, ask the user”s permission to use the camera and show a live stream from the camera on the page. Create an index.html file and add − [code] <html> <head> <meta charset = “utf-8”> </head> <body> <video autoplay></video> <script src = “client.js”></script> </body> </html> [/code] Then add a client.js file − [code] //checks if the browser supports WebRTC function hasUserMedia() { navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia; return !!navigator.getUserMedia; } if (hasUserMedia()) { navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia; //get both video and audio streams from user”s camera navigator.getUserMedia({ video: true, audio: true }, function (stream) { var video = document.querySelector(”video”); //insert stream into the video tag video.src = window.URL.createObjectURL(stream); }, function (err) {}); }else { alert(“Error. WebRTC is not supported!”); } [/code] Now open the index.html and you should see the video stream displaying your face. But be careful, because WebRTC works only on the server side. If you simply open this page with the browser it won”t work. You need to host these files on the Apache or Node servers, or which one you prefer. The RTCDataChannel object As well as sending media streams between peers, you may also send additional data using DataChannel API. This API is as simple as MediaStream API. The main job is to create a channel coming from an existing RTCPeerConnection object − [code] var peerConn = new RTCPeerConnection(); //establishing peer connection //… //end of establishing peer connection var dataChannel = peerConnection.createDataChannel(“myChannel”, dataChannelOptions); // here we can start sending direct messages to another peer [/code] This is all you needed, just two lines of code. Everything else is done on the browser”s internal layer. You can create a channel at any peer connection until the RTCPeerConnectionobject is closed. Summary You should now have a firm grasp of the WebRTC architecture. We also covered MediaStream, RTCPeerConnection, and RTCDataChannel APIs. The WebRTC API is a moving target, so always keep up with the latest specifications. Print Page Previous Next Advertisements ”;