Despite being more than 100 years old, radar technology has only been used in commercial surveillance for the last 10 years. Having now won 14 design and innovation awards for their work, we spoke with the team at Axis responsible for bringing radar to the masses.
(In the image above from left; Carl-Axel Alm, Andres Vigren, Elin Sällberg, Aras Papadelis, Niklas Lindman and Nicklas Olofsson.)
It was 2015. Surveillance cameras were becoming more powerful, boasting ever higher resolutions and breakthrough advances in compression. Yet, despite this progress, the industry faced a pervasive challenge. One that plagued customers indiscriminately: false alarms.
“At that time, cameras didn’t have any reliable means of detecting humans and vehicles in the scene,” explains Aras Papadelis, Expert Engineer at Axis. “Birds, insects on lenses, or even heavy rainfall and wind was triggering systems sometimes hundreds of times a day. It was a huge issue”. At night, it was the opposite problem. Poorly lit scenes would mean objects were simply missed or different challenging light effects could cause false alarms.
Innovating to overcome false alarms
A team of engineers at Axis were tasked with solving this. Providing information on speed, distance, and direction regardless of lighting conditions, radar posed a strong but untested solution. “The idea was that radar, as a technology designed to detect and track movement, would trigger more reliably,” adds Nicklas Olofsson, R&D Director Fixed Cameras & Radar. “Operators would have far fewer false alarms to sift through and the amount of storage needed for unreviewed footage would be significantly lower.”
The automotive industry paved the way
“But nothing suitable currently existed in the security industry,” Olofsson continues. “Up to this point, radar had only been used for surveillance purposes by the military or very high-end and thus very expensive solutions.” These were specialist high-power systems, covering areas by the square-kilometer and manufactured in such low volumes as to make production extremely costly. What’s more, “they didn’t integrate with video management software (VMS) systems easily and so calibrating them to work with cameras was extremely technical.”
It was, in fact, the automotive industry that made radar a commercially viable option. Around the same time, cars were starting to be fitted with cruise control and driver assistance features. These used a different form of radar, built using arrays of antennas instead of the iconic mechanical scanners that many people think of when they hear the word radar.
A team of six engineers on a mission
“We were a very small team,” Papadelis outlines. “There were only three firmware developers, me, developing our tracking algorithm, one electronics engineer, and one mechanical engineer. And none of us had ever worked with radar before. But we knew that members of the Products, Concepts & New Ideas (PCNI) team, and Senior Expert Engineer Carl-Axel 'Cacke’ Alm being one of them, had been experimenting with these new automotive radars.” They had created a functioning protype that worked with Axis Camera Application Platform (ACAP) to flag moving objects that could be humans or vehicles to clients.
The “Klinger” project took form
As these smaller radars were being produced en masse, the procurement cost for parts would be significantly lower, in theory. The challenge was finding a supplier. “We tried our luck with one of the automotive giants, but they just did not want to sell to us. But we never gave up! After some serious research and negotiating we found a German radar manufacturer who was willing and able to supply us with modules.”
It was the perfect springboard for the team. “The PCNI team’s initial proof of concept and sourcing the radar module really set things in motion,” Olofsson adds. “So, to make things official we decided to start a product project called ‘Klinger’, after Corporal Klinger who was friends with Corporal Radar in the TV series M*A*S*H. We knew it would be a lot of hard work and improvisation, but we held a pioneering spirit and pushed forward.”
The first pilot: an industrial lot
“Our initial installation reference was an industrial lot for Byggmax, a Swedish construction materials supplier,” Olofsson continues. “The Product Manager, Andres Vigren, had identified it as a really promising candidate and reference for future installations. He’d taken a short drive around the city and nearby industrial estate and noticed countless, almost identical lots, each using multiple floodlights and cameras to deter trespassing – all of which would have been facing the same issues.”
“It was a nice and accessible starting point for us,” Papadelis chimes in. “It let us focus on area surveillance, monitoring the distance between a building and its surrounding fence.” However, it brought with it a steep learning curve as the team quickly found radar was not without its own set of challenges. First was coverage. “Radars of this type interfere with one another, limiting the usable number of devices to just two,” he continues. “Elin Sällberg, Engineering Manager Core Technologies Radar, started up a team focusing on signal processing and device communication side of things a few years later.” Her team’s work has since proved instrumental to advancing the number of radars able to be deployed.
Borrowing dogs and visiting zoos for building datasets
“I remember one of our first tests running over night. When studying the recording we cheered “Wow, we can see rabbits”. A couple of minutes later we said “Nooo! We see the rabbits,” laughs Cacke. This highlighted the second and equally pressing challenge of classification data. Without data that tells a radar what it is identifying, radars would trigger almost as many false alarms as the cameras they hoped to help. This data did not exist.
There was only one thing for it. The team set to building large and diverse datasets of their own from scratch (which are to this day still being expanded on), with a concerted effort on animals. They would source data from any place they could, from borrowing dogs and visiting zoos, to asking colleagues abroad to capture data on racoons.
“I recall going to a field outside Malmö where I was told it was full of rabbits during night,” details Papadelis. “I set up my recording equipment at 8p.m. and returned early next morning to collect the equipment and harvest it for data. On another occasion, a Finnish power company helped by collecting data on rabbits that would frequent their power stations.” Classification sets like these allow radars to effectively remove and ignore nuisance detections and accurately determine whether an object is a person or a vehicle with very high accuracy.
Tracking a human moving for the first time
“We had some fantastic, eureka moments,” he continues. “For me it was when we detected and tracked a human moving in real time for the first time. I recall holding the radar prototype out of my office window and seeing a person visualized by the radar on my screen. It was so rewarding!”
Thanks to these great efforts the exercise was a success. It proved the concept viable and gave clear development focus areas.
Axis radar development takes shape
From early product development, Axis radar development focused on five cornerstones:
- The team would leverage in-house radar development of module production in-house rather than use standard automotive radars.
- The firmware platform would be based on the same platform as in all Axis video products to simplify development processes.
- Integration with existing VMS systems would be prioritized, specifically Axis Camera Station (ACS), Genetec, and Milestone systems.
- Radar information would be sent to the VMS as a video stream.
- Devices would be powered by PoE for easy installation.
With several lessons in hand, the team set to work on producing their first generation of commercially available products. “It was clear from these initial projects that radar would often be paired with a visual camera,” explains Olofsson.
“It sparked ambitions to create a software application which allowed the radar to control a pan-tilt-zoom (PTZ) camera that would follow radar detected objects as they move across a wide area,” Olofsson continues, “and then move toward ultimately creating some kind of fusion, single unit device with both video and radar capabilities.”
Launching the very first radar detector
Just one year from when the idea of using radar was pitched, Axis launched the D2050-VE Network Radar Detector, its first commercially available standalone radar solution. These radars were anywhere between five to thirty times more affordable than any alternatives in the market and readily integrated with existing surveillance systems.
“Launching the AXIS D2050-VE Network Radar Detector was a huge moment for the team,” Andres Vigren adds. “I was Product Manager at the time, and I recall how well customers and the market as a whole responded to its availability. Not only did it garner plenty of positive attention and remarks on how innovative it was, but it also received accolades such as the SIA New Product Showcase Award at ISC West in 2018, highlighting its impact and innovation in the security industry.”
In-house production for enhanced control
“We also made a really big decision to bring all radar module production in-house,” adds Elin Sällberg. “My team needed full control over the parts to unlock vital avenues for R&D into signal processing. And it really paid off. With in-house production we were able to focus on wide detection coverage, ensuring that we could cover 180 degrees in one product without any gaps. We had full control over the design and could build antennas and signal processing that were optimized for security use-cases. What’s more, and arguably most importantly, we figured out how to deploy up to six radars in the same zone without running into interference issues.”
A world first fusion radar-video camera
Radar quickly proliferated with several new generations of devices launching over the years. Thanks to the work of Elin and her team, up to eight radars could now be used at once and advances in deep learning and software meant customers were able to leverage radar for all kinds of use cases. Radar-controlled PTZ tracking, developed by Leif Persson at the PTZ department and the Podracer team, proved a huge success, and yet despite all this, the idea of a two-in-one fusion camera remained an elusive goal. That was until 2021.
“It had been a real challenge, presenting a seemingly endless string of problems,” explains Papadelis. “But we knew it was too good a concept to give up on. So, after a lot of hard work and dedication from the team we finally had a working prototype that could run fusion live on the device. Once it was up and running it was immediately clear this was a winning concept.”
Axis had officially launched the world’s first fusion radar-video camera, the AXIS Q1656-DLE. Not only does the device afford the synergies of these two technologies working in perfect synchronization, but it also comes with a dedicated traffic mode for gathering, visualizing, and using vehicle speed for traffic statistics.
Reflections on the evolution of radar technology
Niklas Lindman, Product Manager for radar at Axis today, reflects on the journey to this point. “Looking back through the history of why we at Axis started to develop radar products, and later incorporate them into the camera system, I can draw two conclusions. Firstly, a lot of the assumptions we made quite a few years ago are considered facts today. We went with our gut instincts because we could see the potential these ideas held. Combining these technologies and bringing them together in a single solution seems obvious in hindsight but the team really used out of the box thinking to make it a reality. And it absolutely continues to provide value to this day. Secondly, is that innovation didn’t and does not peak at the initial product launch. Combining different technologies can give rise to countless new ideas and provides the foundations for some of the most innovative solutions today. The future continues to look extremely bright.”
As the surveillance and security industry continues to move toward proactive and preventive solutions, the detail and capabilities that technologies like video-radar fusion afford will become ever more important. But it was thanks to a skilled and innovative team at Axis with the grit, determination, and drive to make their vision a reality that we have radar technology found in locations across the world.
Nicklas Olofsson, Aras Papadelis, Niklas Lindman, Elin Sällberg, Andres Vigren, and Carl-Axel “Cacke” Alm all contributed to this story.
Special thanks go to the entire “Klinger” project team, to PCNI, to the “Podracers” team, and to Leif Persson.