You may well have heard the term ‘metadata’ over the past couple of years. In simple terms, metadata is data that describes the objects and what is taking place in a scene covered by video surveillance.
Improvements in image quality from surveillance cameras and the application of deep learning and AI provide the foundation for improved scene intelligence, and detailed analysis and categorization then creates the metadata.
This post doesn’t aim to provide a deep dive into metadata itself (if you’re interested in doing that, you can check out previous blog posts like this one, and also our whitepaper linked to at the end of this post). Rather we’re keen to look at how metadata will be used, both in itself and in combination with data from other sensors, to create new applications, services, and analytics.
In the technology industry, it can be easy to overstate the potential of new innovations. But we genuinely believe that the increase in metadata and its quality provides an opportunity with almost infinite possibilities in security, safety and – perhaps most of all – operational efficiency.
Here are five ways in which metadata is being used today and will be employed in the future:
1. Finding what you’re looking for, fast
Since video surveillance was first created, the need to search through hours of footage has been a requirement, particularly in post-incident investigation. When a site or city might use dozens or perhaps hundreds of surveillance cameras – all operating 24/7 – this can quickly become an onerous manual task, significantly slowing down investigations.
The detailed descriptions of a scene provided by metadata transforms the speed and accuracy of search. Rather than manually reviewing hours of video footage, operators can use simple search parameters to find, for example, footage which includes, “a person with red upper clothing”. Additional filters can be applied to aid the search, such as areas of interest, times and dates, and even the speed and direction of travel of certain objects.
The speed of post-incident investigation is a key factor in the ability of security and law enforcement personnel to find and contact those involved. Metadata is the foundation for a revolution in search.
2. A more accurate view in real time
Another traditional role for security operators being dramatically enhanced by metadata is the monitoring of real-time video. With resource limitations affecting the ability to effectively monitor the feeds from multiple surveillance cameras, we need to find ways to augment and support operators in their roles, particularly when incidents require an immediate response.
Metadata brings the ability to more accurately analyze a scene automatically, creating alerts for operators or triggering actions (e.g. automated audio announcements) based on observed activity. The attention of operators is directed where it is most needed and beneficial, with automated actions supporting the response.
In addition, dashboards delivered through video management software (VMS) will give a real-time view of the metadata from surveillance solutions. Sitting alongside the video feed itself, this adds greater levels of operator insight on a scene.
The same metadata can also be integrated into Business Intelligence (BI) platforms, cloud-based services, access control systems, or parking solutions, providing the basis for more efficient operations and customer experience.
In the retail environment, as another example, managers could be alerted to an increase in customer numbers and ensure that staff levels matched the need. Or on industrial or construction sites, access might only be granted to individuals wearing appropriate personal protective equipment (PPE). The potential use cases are endless.
3. Combining data from multiple sensors
While the physical form of surveillance cameras may not have changed greatly over recent years, their capabilities have been transformed: today, they have become powerful ‘sensors with a lens’. AI has accelerated this transformation, allowing the camera to recognize objects and produce reliable, high-quality images and metadata.
In itself this data has great value, but when combined with data from other sensors – for instance, those measuring the quality of the environment, sound, or vibrations in machinery – it becomes even more powerful. The insights that this combined data provides – whether in real-time or over extended periods – is opening numerous opportunities for new applications and use cases.
Alerts can be created by any sensor when specific thresholds are met, with video surveillance cameras allowing for rapid visual verification of an incident or evolving situation, and appropriate remedial action to be taken. In essence, the combination of data means that an operator’s ‘senses’ around a scene are enhanced beyond just the ‘sight’ that video alone brings.
For instance, audio analytics can be a valuable complement to visual information. In detecting sounds that could indicate and give early warning of potential physical aggression, such as screaming or shouting, or criminal activity, like breaking glass or even gunshots, these bring another level of awareness to operators.
As another example, sensors detecting excessive levels of heat or vibration in machinery can give early warning of a potential issue. Where this machinery is located remotely, or in difficult to reach or dangerous areas, visual verification can help assess whether preventative maintenance is required, safeguarding ongoing operations.
The metadata from other sensors is also valuable in rapid post-incident investigation. Take metadata for audio analytics. Being able to search for the sound of screaming or shouting allows operators to pinpoint incidents accurately and almost instantly. Adding ‘ears’ to the ‘eyes’ of your surveillance solution delivers a far more comprehensive understanding of what is happening – or has happened – in a scene.
4. Spotting trends and patterns over time
The capabilities of today’s surveillance cameras means that analytics and the creation of metadata takes place at the edge of the network, within the camera itself. This brings significant benefits in the quality of analytics, being able to react to events in real-time, and in reducing the amount of data that needs to be shared across the network.
But with the huge volumes of metadata that are already being generated from the increasing amount of captured video, even at these relatively early stages, another opportunity emerges. Organizations can aggregate and analyze metadata over time, spotting trends and patterns and leading to enhanced operations in all aspects of business and life. Again, this is even more powerful when combined with data from other sensors.
Take, for instance, the metadata captured from cameras monitoring the flow of traffic across a city. Combining with the data of air and noise quality sensors, city authorities will be able to plan for changes that will have a positive impact of the lives and health of citizens, for example banning construction work during certain hours, or heavy vehicles entering the city limits. Vehicle counting analytics can help city planners locate bottlenecks to optimize roads, intersections, and infrastructure and minimize traffic jams and congestion.
Similarly, data collected in so-called smart buildings can be analyzed to optimize the management of heating, air conditioning, and lighting systems to support efficiency and environmental goals.
It is in the area of trend analysis over time where we are likely to see some of the most impactful benefits in operational efficiency arrive.
5. Powering the next generation of applications and services
We’re merely scratching the surface of the opportunity that exists through the use of metadata.
The next generation of applications and services is on the way - some based at the edge, in the camera itself, others in the cloud (or, most likely, in a hybrid system to take advantage of the benefits of each environment). These will make use of combined data from multiple sensors and sources, and will bringing benefits to traditional video surveillance use cases as well as opening up new possibilities.
The creativity of thousands of application developers around the world will be the foundation for these applications and services. The Axis Camera Application Platform (ACAP) is one place for developers to explore the possibilities that enhanced metadata provides.
We’re excited to see how we and our partners take advantage of the opportunity. Particularly as the biggest transformations may well come ‘beyond security’, where insights from metadata can help organizations, cities, even countries, operate more efficiently and to the positive benefit of their citizens’ wellbeing.
