WebRTC in video surveillance; How it works with Viinex

This piece offers insight into specific aspects of realization, advantages and area of implementation of WebRTC as a part of Vinex middleware.

To begin with let’s go back in time a little. WebRTC stands for Web Real Time Communication. Google company has invested extensive resources into this standard, which was officially announced in May 2011. WebRTC specification has been changed and updated since, the project has an open code and is supported by Apple, Google, Microsoft, Mozilla and other developers. As a result, we all have a fairly complete open web standard, also fully applicable for video surveillance and available as JavaScript API interfaces in all main browsers (according to webrtc.org)

Why is this so important? In video surveillance, thick client applications were widely used, they helped to settle all the questions regarding approaches to network organization and development or using standards of data exchange between a server and a client. Nevertheless, using a thick client on users’ equipment may be impossible, restricted or even be unwelcomed by users. It became especially clear due to the global trend of remote work  in post-COVID era.

Web technologies are applied in almost all software that we use daily. More often than not they are used for integration with other systems when building a complex solution, while building a distributed system inherently requires data transfer over the Internet.

There is a certain problem of using web technologies in video surveillance. A case in point is an IP camera itself, a basic element of video surveillance systems. IP cameras stream video using RTSP standard, and this video stream should be somehow played back in a browser to set up the cameras - watch video, zoom in, setup onboard video detectors etc. For these tasks video cameras manufacturers often use additional browser plugins that need to be installed (particularly quite often it’s ActiveX with all its restrictions).

But thanks to WebRTC capabilities in terms of video streaming to mobile clients and browsers there finally is a solution that settles the question for all platforms and browsers and helps us get rid of additional plugins and thick clients.

WebRTC was created to enable video calls over the Internet, while providing efficient use of network infrastructure and traffic economy. That’s why it co-opted many VoIP technology solutions. Some of its advantages are:

  • providing communication with consideration to modern networking, namely establishing connection via NAT when none of the communicators is in the Internet with a direct IP address;
  • low latency, which is crucial for video conferencing;
  • comfortable and easy API using.

Let’s first inquire into the first main point that is interesting from the point of view of video surveillance. Here’s the basic WebRTC scheme with signaling server:

If Browser 1 is replaced with a video server, it’s plain to see that the whole structure looks exactly like an example of video surveillance system deployment. To make it more illustrative let’s add the cameras and network environment:

It has to be explained that STUN (Session Traversal Utilities for NAT) server is a special application in the Internet with an IP address. STUN server provides tuning of NAT routing tables in both routers and allows them by proxy exchange information about each other and what data they need. All further video streaming is direct between the server and the client. As a result, the cost of maintaining such a central service in the Internet will not be high, since there is no need for great processing power, fail safe data storages or wide bandwidth for video traffic.

In general STUN server can be both independent service in the Internet or deployed with video management system service, depending on the architecture of selected video solution. Viinex includes a ready to use STUN server, which gives certain advantages:

  • we support a complete set of software necessary for deployment of a video surveillance system with the use of WebRTC;
  • the communication becomes easier, since data pass through the single application with the same IP address and port, which makes connection faster, and in some cases allows to establish it in networks where independent STUN and signal servers installed in Internet couldn’t be used.

Real-time streaming

Let’s proceed to the second point - low latency streaming video. Latency in delivering information to the screen may appear due to a wide variety of reasons. Data format, information processing, receiving and decoding data, and in case with WebRTC its deciphering as well. The smaller and easier these steps are on all stages, the sooner we’ll see the image on the screen.

We have already described the first step - providing the direct communication between the clients makes routers in fact directly exchange video streams. This, no doubt, helps to reduce latency.

The second step is using UDP transport protocol as a base for online video streaming, since it requires  no additional packet exchange control. It should be noted that real time tuning of routing tables and further streaming are possible when using UDP protocol and proceed from the basics of building communication networks for broadcasting. It’s a standard technology that is supported by virtually all routers in the market, from SOHO hardware to professional solutions. QoS technology is used to ensure high quality packet delivery.

The third step to reduce latency would be the way that information is transferred and received, which might make a contribution too. To demonstrate this, let’s go back to our previous piece on HLS support in Viinex. As can be seen from a diagram of sequences of data streaming in HLS protocol, to start video playback a chunk of video should be formed and sent to the client. The time that data transfer will take mainly consists of three components: time of preparing the chunk (a), time of streaming (b) and time for its full decoding and preparations for playback (c). Assuming that our agreement is 10 second video chunks. What we get is: 10 seconds to gather data at the sender side, then, say, 1 second to stream it and 1 second to decode it. In all, a + b + c = 12 seconds. We can surely change the agreement to 5 seconds, but it’s important to bear in mind that there are also some pitfalls: preparing video files, transferring them through network, as well as preparing them for buffering to playback it without any delay on the client's side.

There is none of mentioned above in WebRTC, as there is no chunks playlist. Video streams in real time without stopping to prepare every single chunk in a certain format. Consequently, we may almost totally eliminate (a) from the formula, while (b) and (c) are always minimal, since the volume of data streamed is not big. So total latency when streaming video from an ordinary USB camera is b + c = 230 ms and 300...400 ms with IP-cameras (they need time for onboard video coding). This are the numbers that we got with our tests.

This is important in a video surveillance system. Quick delivery of alarm incident with online video verification to a remote security officer makes real time surveillance possible. Obviously, manual PTZ operation allows for quick and convenient turning of cameras towards the objects in the control area.

WebRTC connection optimization with Viinex

However, it is not only the speed of delivering video image that is important, but system responsiveness in general when transmitting various commands. Let’s see the diagram below:

This diagram is here not so much for explaining the details as for showing that quite a lot of communication happens on the stage of so-called handshaking, i.e. establishing the connection. Only the blue arrow means that finally secure video streaming has started, everything that happened before was just a preparation. What we have optimized in Viinex for using in video surveillance systems is reducing the number of these handshakes, or, in other words, using the connections that have already been established, which may be pretty expensive in terms of time. After we have once established connection, the user might at some point need to change the video stream on the screen. Our API contains a simple command for changing the stream, that in case with WebRTC simply replaces existing video stream in a current session with a new one, so the user doesn’t have to wait for new connection to be established and the video from the requested camera appears on the screen in some time. This way we achieve a significant increase of responsiveness when sending a command to change the video stream because there is no need for new connection initialization.

We also have to remind that Viinex is not only an online streamer, but a video recorder too. When we stream a video archive, time positioning, pause, playback and other commands are being run within a current session.

Secure Video

Protection against MitM (Man-in-the-Middle) attacks is significant. Data interception may be possible, but its decoding won’t be a trivial task. Put another way, it is virtually impossible to understand what is in the bytes that you stream. WebRTC uses DTLS standard for encryption keys and SRTP for media streaming. Encryption in WebRTC not only ensures information security but also improves communication quality, since it brings along the checksums that can be used to detect damaged data chunks and not decode it. More specifically, it is impossible, since data cannot be deciphered using a wrong checksum or a damaged data chunk. This data is tossed aside.

Another type of attacks is data spoofing. Viinex supports usage of custom certificates. All that is needed is to check them on the client’s side. This way the client will always know that incoming data is not fake.

Standardization and widespread pervasiveness

Are browsers as video surveillance client applications ready for WebRTC today? We can check with caniuse.com, where information is usually up to date. At the moment the situation looks pretty decent (WebRTC caniuse.com).

Do browsers support hardware acceleration? For any video surveillance system, it is crucial to have a capability to show several video streams on the screen simultaneously. That’s why the browser should support hardware decoding with the graphic processing unit (GPU) to increase the number of simultaneously displayed video streams. WebRTC specification at the moment mentions only two codecs: H.264 and VP9. Historically the latter is not used in video surveillance. With H.264 everything seems to be fine (H.264 caniuse.com). As for H.265, at the date of this writing it is not supported neither by WebRTC standard, nor by absolute majority of browsers. You can compare (H.265 caniuse.com). At the moment H.265 codec is supported mostly by thick clients applications, not by the most browsers.

It is very easy to check whether your browsers support hardware decoding. For instance, in the most popular browser, Chrome, just type chrome://gpu.

WebRTC online and archive video streaming

Let’s switch to the topic of video streaming. One can guess that it makes no difference for WebRTC standard if the video being streamed is archived or live. All commands for playback of needed parts of archived records are sent via Viinex API. Viinex allows working not only with cameras and its own video archive, but also with other video sources, like third-party video surveillance systems, actually realizing a gateway software to manage VMS video streams but in HTML5 compatible format.

Let’s add these solutions to the scheme to make it more illustrative:

In Viinex middleware external command signature is the same when working with Viinex's archive video records or records from other video surveillance systems. The advantage is not only that it is possible to work with online or archive video using a single API, but also that Viinex supports working with several video surveillance systems at the same time. This allows uniting video solutions that are in use on the installation and having a universal tool set for video management. It makes work of our partners developing and dispatch systems and PSIM significantly easier, for it gives them a possibility of working with several video surveillance systems simultaneously using our API.

Via its API Viinex enables video archive streaming both in RTSP and WebRTC, including external VMS, which sometimes have no HTML5 compatible streaming feature, and that allows our partners that want to provide their customers web access to video from these VMS, stream video in a HTML5 compatible format.

To compare WebRTC and HLS, it is quite difficult to establish interaction with external video archives using HLS, since VMS usually give out archive as a continuous video stream, not as video data chunks, so it’s impossible to know in advance how to break the stream into chunks and detect the i-frames that chunks should start with. In this case HLS no longer offers a possibility of streaming the video right away from any chosen point in the archive, but Viinex does, since it supports WebRTC and makes video archive streaming available for web applications.

Managing video on the Internet with Viinex middleware

This piece summarizes the above mentioned answers to the complex question of how to quickly organize this type of “cloud” functionality when working with video in your application, if you need to provide your users a possibility of watching online video or a video archive records from their servers installed on the edge.

We have to clarify that in spite of using the word “cloud”, most likely we’ll be far from the real sense of it, since we want to talk about a particular quite common scenario of accessing the video surveillance system resources online from any point on the globe. Our goal is to quickly create a "cloud service" that provides users from the Internet access to their video. If we assume that you already have a video surveillance system, it is clear that by adding to it WebRTC standard based on Viinex middleware we actually turn it into a such “cloud” service.

One can see that our video surveillance system is situated behind the router, there is no direct access to it, not a single port is open. Servers are still where they were, in a safe location. Everything opens up and is available only when someone behind the router wants it to. It can’t be managed from outside. This is secure and simple at the same time. And it’s well known that simple things work best. There is no need to put or move any part of the video surveillance system in the Internet. The traffic goes as usual - from the camera to the video archives in local network. Additional services in the Internet due to this upgrade consume as little traffic as possible and do not put through any video streams. If any video stream is needed, it goes directly from user’s router to video surveillance system router without going through data center.

WebRTC allows working with a video surveillance system even when there is no dedicated IP address, ignoring the network infrastructure that we use for Internet access. In streaming it's the information source (in our case it’s a video surveillance system) that sends the data. Thanks to WebRTC standard, no matter where you are, the stream from the video surveillance system will find you and you’ll see the image on a screen of your laptop, tablet or smartphone.

Now let’s see in more details how Viinex solves the task of video surveillance via the Internet. Our middleware includes two main services, each of them deals with a part of this task.

  • First thing that should be noted - Viinex contains a STUN server, in the terminology of WebRTC. STUN server may be located in the Internet and it is responsible for initial and all further steps of establishing communication between client and video surveillance system.  As soon as STUN part of the service receives a request from the client’s side, its signal part already has this information. In the Internet there are external services or their free analogues with an open-source code that can be downloaded and installed, but all these solutions not always can be a part of a commercial proposal. It is important for us to emphasize that STUN server is a part of our Viinex middleware and we’ll keep on improving it.
  • Second is the Viinex video server. For your solution development it performs all necessary tasks relating to video surveillance and video streaming using WebRTC standard, as well as other popular standards like RTSP and HLS.
  • Third is important to remind that Viinex can work as a proxy for other video surveillance systems that have already been installed by customers and re-stream video from them with WebRTC protocol.

One way or another, your application will be ready to perform this cloud task.

Let’s sum up the advantages of this solution:

  • Video surveillance may work in almost any browser. WebRTC at the moment is supported by an absolute majority of browsers, including popular Safari, Firefox and Chrome.
  • Low latency.
  • Provides security for video streaming.
  • Does not require wide bandwidth in the data center.
  • Fully preserves existing video surveillance system infrastructure. As a result, it helps to build an actual cloud video surveillance system on the base of existing one.

Leave a Reply