Storage in video surveillance

Storage is critical in a video surveillance system. What are the latest trends in storage and how should the user decide which solution to get? This series will address those points.

Source: William Pao

Storage is an important element in video surveillance. It makes sure that the video is properly stored and can be accessed anytime. Indeed, with video datasets constantly growing due to higher resolution and longer retention periods, storage is needed more than ever. According to MarketsandMarkets’ forecast, the global video surveillance storage market size will grow from US$7.5 billion in 2020 to $10.2 billion by 2025, at a CAGR of 6.1 percent.

So what are some of the current solutions and future trends in storage? We take a closer look below.

Storage device

From a device perspective, network video recorders (NVRs) are still the most popular solution. “NVRs that use hard disk drives (HDDs) remain the most popular hardware globally for video storage because of their cost effectiveness, simplicity of installation, and ease of use. Today HDD NVRs are almost like little computers with a raft of internal error correction technologies that make them incredibly robust and reliable. However, quality varies, and users should look for reputable manufacturers that approve HDDs for video recording,” said Peter Kim, Global Technical Consultant at IDIS.

With the increasing popularity of cloud-based video surveillance, more and more NVRs now come with cloud capabilities. “NVR complete with cloud solutions is enabling people to view live or recorded footage remotely over the network, simply by logging into their computer, phone, or tablet. Should an alarm be triggered, an email notification can be sent to a device, and the user can remotely verify, save a snapshot or immediately email the footage to authorities,” said Stefan Lundberg, Senior Expert Engineer at Axis Communications.

Storage medium

From a storage medium perspective, as aforementioned, the hard disk drive is still the mainstream. “Hard disk drives are still the mainstay of almost all video surveillance storage – even in the cloud. There have been some shortages of high-capacity hard disks due to component shortages and a technology shift in cryptocurrency which used massive amounts of storage, although the collapse in cryptocurrency values has significantly decreased the demand for high-capacity drives for those purposes. Solid-state storage has not progressed as expected some years ago, as it has remained too expensive for mass-storage applications such as video. It has also suffered from worldwide component shortages. Therefore, hard disks are still the king of video surveillance storage applications,” said Alastair McLeod, CEO of Veracity.

Software-defined storage

Software-defined storage (SDS) relies on a computer program for managing data storage resources and functionality and has no dependencies on the underlying physical storage hardware. It’s becoming a more popular storage infrastructure in the IT world due to several factors, including the growth of unstructured data, the availability of high-performance servers, and the popularity of the cloud. Increasingly, it’s caught on in the video surveillance industry as well.

“SDS is slowly starting to show some good footing in the video surveillance space. With an agnostic approach to hardware and flexibility of workloads, this option provides great benefits to those well-versed in managing this type of storage infrastructure. Although this approach introduces another layer of complexity, the benefits of SDS can exceed traditional storage environments,” said Eugene Kozlovitser, CTO at BCD. “With the ability to distribute workloads between low-performing and higher-performing systems accordingly, SDS does offer enough elasticity to cater to a broad spectrum of customer service needs. And with every software vendor offering a different approach to this concept, there is a large variety of choices out there for any scale of deployment.”

Hyperconverged infrastructure

Meanwhile, hyper-converged infrastructure (HCI), a software-defined unified system combining compute, networking, and storage elements in a single platform, is also gaining traction in video surveillance. “HCI is the foreseeable next step in the video surveillance market. Although slower inception of the technology, the growth trend has been promising based on some recent market analysis, and more customers are looking to adapt to this model much more quickly and align with the concept of video surveillance as a service (VSaaS),” Kozlovitser said.

Lower energy consumption

Since a storage system consumes a lot of energy, lowering energy consumption has become a priority. “A key trend that will finally emerge as being critically important in surveillance storage systems is energy consumption. Veracity has been arguing this case for many years, but with the recent dramatic increases in electricity costs, many more customers are going to see this as a critical factor in their choice of solution,” McLeod said. “Lower power consumption means lower heat dissipation, and therefore lower-cost A/C systems can be used with lower running costs. Further, lower power consumption means that lower capacity UPS (uninterruptible power supplies) can be employed, again lowering capital and running costs whilst increasing efficiency.”

Increased datasets

It’s important to point out that storage for video surveillance is hugely impacted by the amount of video data, which has become bigger than ever due to higher resolution and longer retention times.

“Across a wide range of vertical market sectors, we are seeing organizations required to store video for longer durations, sometimes significantly longer. For example, in the U.S., regulations state that cannabis operators must store surveillance footage for 90 days across cultivation and logistics sites as well as dispensaries. This is also the standard retention time for a variety of sectors in the Middle East, most notably banking and finance,” Kim said.

He added: “In addition to longer retention periods, the volume of data being generated is also increasing significantly. A few years ago, many users were upgrading to Full-HD systems; now we are seeing 5MP cameras becoming the norm as customers look for the additional benefit of AI-powered video analytics. For wide-area surveillance such as public spaces, parking lots, large perimeters, and stadia, 4K PTZ cameras are also becoming the go-to solution.”

Needless to say, bigger data takes up more space in storage, incurring higher storage costs in the process. Luckily, advances in technology, especially in the area of compression, have been able to address this issue.

“Video compression algorithms are used to encode the original information by reducing and removing redundant information. These algorithms locate regions in the video that have already been transferred so that redundant sending in the next image frame can be avoided. The algorithms also identify places in the video where details can be removed without reducing the visual quality,” Lundberg said.

“Advances in technology have meant that the overall cost of data storage has decreased. The switch from H.264 to H.265 over the last decade brought about a 50 percent reduction in costs, and with the addition of intelligent codecs an additional 30 percent reduction has been achieved, with storage requirements brought down significantly,” Kim said. “In addition, users can choose to use motion adaptive transmission (MAT) technology on specific cameras to gain savings of up to 90 percent on storage and bandwidth compared to older H.264 systems. When switched on – and many users neglect to do this – MAT technology can be a particularly effective tool, restricting transmission during live surveillance to periods without movement in the scene.

How to choose the best solution for you

Storage ensures that the video captured is properly stored and can be accessed whenever needed. Given its important role, purchasing a solution should be done with care. Below we look at how the user should select a storage system that best fits them.

Camera count

One way to select a storage solution is by considering the user’s camera count, which can be divided into low, medium, and high. We take a look at each.

Low camera count (up to 32 cameras)

There are two approaches here if the entity has a low camera count. “One is the conventional NVR system with onboard storage, which may be an integrated appliance, or which may use a separate client application to manage the system. The second is to go for a cloud-based solution, with no on-site equipment other than the cameras. However, the total cost of ownership for the latter can be quite high due to the constant subscription costs, especially compared to one of the more reliable NVR-based VMS systems, which should last for years,” said Alastair McLeod, CEO of Veracity.

“For the smallest applications, compact 4- and 8-channel NVRs will be the most cost-effective. We see these, for example, as a popular choice for individual retail outlets and locations such as coffee shops, restaurants, and pharmacies,” said Peter Kim, Global Technical Consultant at IDIS. “But most SMEs will find that a 16- or 32-channel NVR provides them with the storage capacity they need for full site coverage, internal and external. And these devices give them the flexibility to scale and adapt.”

Medium camera count (32 to 100 cameras)

As the camera count gets higher, a cloud may be less ideal. “Paying for 100 recording subscriptions plus, say, 400Mbits/sec of internet uplink bandwidth (on top of any additional, corporate requirement) can be very expensive,” McLeod said. “The optimum solution here is either one or two NVR units with a client PC, or, depending on camera actual count and retention times required, an integrated high-capacity storage/recording server – again with client PC or web-based management.”

High camera count (100 to 5,000 cameras)

According to McLeod, larger systems require some design effort to ensure the customer gets the solution they require for their application. “We would advise against systems that can store very large numbers of cameras in one server, as this makes the system far less resilient. We would recommend no more than 100 cameras per server. Thus a 1,000-camera system would have a minimum of 10 servers plus at least one failover unit,” he said.

On-prem vs. cloud

As mentioned above, cloud and on-prem solutions each have their own merits and disadvantages. Users should choose based on their scenarios, including camera count and bandwidth availability.

“While cloud storage is gaining ground in the market, the increased bandwidth requirements it brings are an issue. Users may resort to storing lower-resolution video as backup, but this is not ideal if the footage is needed for evidential purposes,” Kim said. “In most cases, on-prem storage can still deliver the best value and most benefits for users. When robust and reliable hardware is used, maintenance issues will be reduced, and choosing a single source provider rather than a mix-and-match solution can also help with this.”

Hybrid, which combines the best of both, may be a better option for some. “A hybrid deployment option is also a great solution for those who have the expertise in managing such an architecture. With the utilization of on-prem servers, a bridge extension to a public or private cloud might just be the better option,” said Eugene Kozlovitser, CTO at BCD.

Storage calculator

To calculate how much storage to get, the user can benefit from a storage calculator. “Over time, business needs are likely to change and so too may the surveillance requirements. A user may add additional locations or add space to an existing location and therefore need to add more cameras. In turn, they’ll need more storage,” Stefan Lundberg, Senior Expert Engineer at Axis Communications. “A storage estimation method is very important, usually this is built into the system design tools. For very large systems it is important to also make a trial to improve the estimate.”

But the user should careful when using a storage calculator. “Most storage calculators are provided by camera manufacturers who will try to emphasize the efficiency of their image compression,” McLeod said. However, the best and most accurate way to estimate the storage required is to use a calculator which uses the average bitrate per camera as an input, rather than the more common frame rate, compression type, scene activity, and resolution settings used by many storage calculators. For example, a full HD camera at 30 FPS and using H.264 compression will produce about 4Mbits/sec for a reliable high-quality video stream.

Additional points

Given storage is as vulnerable to security breaches as other networked devices, cybersecurity measures should be taken. “Things like drive encryption, role-based access control, strong endpoint security, and redundancy are a must for any cyber protection plan. With data in the cloud or data on-prem, securing stored data is, and should be, a top priority for any customer. Regardless of the amount of video surveillance data or where you choose to store it, it’s always best practice to have ‘zero trust’ awareness of any stored private information,” Kozlovitser said.

It’s also important to point out that when the user recycles or disposes of their NVRs, all contents should be erased. “Usually when an NVR or a VMS system is replaced or taken offline, it is kept for the retention period to keep older recordings until they are all automatically deleted. The system is then ready for recycling/disposal; in that process, it is very important to securely erase any video storage medium or any medium containing sensitive configurations like certificates and passwords,” Lundberg said.

“It is best practice to either degauss any storage media or crush the drives completely that have had recorded video data. Overwriting drives do not always guarantee all the data will be fully erased. Recycling old NVR units should always be done by an R2-certified facility and one that can issue a certificate upon completion,” Kozlovitser said.