Radar Applications: The Potential of 4D Imaging – Vayyar Interview

This article was originally published on i-Micronews. To read the original article, click here.

Vayyar, an Israeli based startup company, has been developing a 4D radar imaging sensor over many years. Initially applied for medical purposes, the sensor has been adapted for automotive radar applications including in-cabin and ADAS features, from vital sign monitoring to blind spot detection. This technology has been deeply analyzed by System Plus Consulting, sister company of Yole in a new report: Vayyar VYYR2401 4D UWB Radar Imaging SoC.

For better insight, Cédric Malaquin, Technology & market analyst at Yole, and Stéphane Elisabeth, Cost analyst at System Plus Consulting, interviewed Raviv Melamed, Chief Executive Officer and Co-founder of Vayyar Imaging, to discover the full potential of the 4D imaging Radar Sensor. Indeed, the domain of applications is not restricted to only medical or automotive but could be extended to the consumer world.

Stéphane Elisabeth (SE): Please describe your position and mission at Vayyar to our readers?

Raviv Melamed (RM): Hi, I’m Raviv, the CEO and co-founder of Vayyar Imaging. Vayyar started with a vision of developing a more effective way of detecting early-stage breast cancer using radio frequency technology. As our technology matured, we created additional opportunities including solutions for senior care, automotive and smart buildings. Our 4D radar imaging sensors can see through walls & objects and track & map everything happening in an environment in real-time. Unlike other products that rely on cameras and optics, Vayyar’s sensors do not collect any optical data, ensuring users’ privacy at all times.

Vayyar’s mission is to continue expanding the range of consumer radar applications, so we can deliver the next generation of sensing technology that is miniature, affordable and versatile enough to impact everyone’s lives, enabling a safer world.

SE: Could you summarize your company’s activities since its creation and the product offerings and roadmap you are pursuing now?

RM: Since Vayyar was founded, we’ve successfully completed four funding rounds, raising a total of $189million. Vayyar has launched two generations of RFIC – our first was a UWB imaging radar and our second is a UWB + mmWave imaging radar.

In 2016, we released our first consumer product: Walabot DIY, designed to make home improvement safer and easier.

Our second product, Vayyar Home is the world’s first touchless personal emergency response system (PERS) based on 4D radar imaging. Vayyar Home protects seniors at home or within senior care facilities, providing 24/7 monitoring and instant emergency alerts without the need for cameras, wearables, buttons or cords. Vayyar Home not only prevents falls but also monitors ongoing senior activity patterns, alerting for changes in behavior, indicating deteriorating health patterns.

In the automotive domain, Vayyar’s intelligent sensors create holistic safety opportunities for in-cabin and ADAS, using automotive-grade 4D imaging radar technology. These sensors provide comprehensive detection in and around the vehicle, while simultaneously tracking multiple targets and objects. In the cabin, this technology is used for Child Presence Detection, enhanced seat belt reminders and other life-saving radar applications. Outside of the car, just 4 Vayyar sensors can replace up to 14 ADAS sensors, and even facilitate vehicle autonomy.

During Q4, we’re launching our Vayyar Element module, the first-ever commercialized radar sensor, giving developers and engineers the option to create their own safety, security, healthcare and IoT radar applications using easily integrated mmWave technology.

As the global leader in 4D imaging radar sensors, we’re also active in the retail, robotics, smart buildings, medical and homeland security industries. We develop all of our imaging algorithms for complex signal processing, as well as the SDK and APIs to complement our versatile software-hardware radar platform. Vayyar intends to expand to more verticals and additional radar applications, grow our number of partnerships and continue to make 4D imaging radar commercially available and affordable. 

Cédric Malaquin (CM): At Yole, we have forecast significant growth for an already large market: +5% CAGR between 2019 and 2025. How do you see the radar industry evolve, from your side? In your opinion, what will be the killer applications for the coming year(s)?

RM: Due to its versatile nature, radar is rapidly expanding into dozens of industries. The fact that the next generation of radar integrates 4D imaging and a large MIMO array helps increase its potential substantially. The resolution of our imaging radars today, combined with traditional radar benefits such as privacy protection and robustness, is already causing high demand in automotive, senior care, smart home, medical, robotics and smart buildings—and that’s just the tip of the iceberg. Overall, this technology is ideal for use in many settings because it’s non-ionizing, low-power and energy efficient, allowing for use in the home, workplace, vehicle and public space.

CM: Did the COVID-19 outbreak impact Vayyar’s activities? Can you say few words on the effects to our readers?

RM: Despite the challenges every company is facing during COVID-19, Vayyar has been able to leverage the versatility of the technology in two ways:

• Adapting to the new work-from-home reality. Traditionally, we offer a 4D imaging radar evaluation kits (EVKs) for our partners to test and develop applications. Because our EVKs are easy to set-up, integrate and evaluate, our partners are able to continue developing and validating radar applications from inside their homes, their vehicles and their personal computers. Our sensor solution provides a full-stack hardware-software platform, so both our own team members and our clients are able to seamlessly transition to a home workspace.  Our technical customer support team continues to accompany our partners during this time and releases OTA (over-the-air) software updates that allowed partners to continue with their innovation.
• Realizing the opportunities 4D imaging radar offers in the face of the pandemic. Vayyar partnered with the Israeli Ministry of Defense, in a pilot to prevent contamination in hospitals at the height of the pandemic. We also partnered with Medisana, the developers of the MediTemi robot, to allow the robot to detect vital signs and track people (and COVID-19 symptoms) in public places. Our latest opportunity is in the smart building domain, where our sensors monitor congestion and enable touchless building operation to reduce contamination. By automating doors, dispensers, elevators, lights and HVAC systems, employees don’t have to touch buttons, switches, door handles, etc.

CM: Vayyar demonstrated the potential of radar sensing for many market verticals. On which of those markets will Vayyar focus first?

RM: Vayyar’s primary focuses are senior care and automotive industries. In senior care, we market the world’s first contact-free PERS solution for care facilities, providing 24/7 monitoring and instant emergency alerts without the need for cameras, wearables, buttons or cords. Vayyar Home is mounted on a wall and automatically detects falls and summons help. The system can also prevent falls through advanced analytics that monitor ongoing senior activity patterns, alerting for changes in behavior, indicating deteriorating health patterns and enabling an early intervention, minimizing hospital care and expensive medical bills.

In the automotive industry, our priorities are in-cabin safety sensors and life-saving ADAS (uSRR/SRR/MRR) radar applications. Vayyar’s in-cabin sensors help prevent the tragedy of hot-car deaths, in which children are killed because they are forgotten in vehicles by parents or caregivers. In 2019, there were 53 hot car deaths in the US alone, with thousands of children worldwide being rescued from locked vehicles in distressed conditions. These tragedies are all preventable, by means of our reliable, accurate and affordable technology. Vayyar’s  intelligent sensor is designed to meet Euro NCAP 2022/23 requirements and enables Child Presence Detection (to prevent hot car incidents) even if the infant is covered by a blanket or hidden in a car seat or in the foot well. The mmWave radar-based CPD solution detects passengers in each seat in the car, differentiating between adults and children. Once it identifies that a child is the sole occupant of a vehicle, the driver can be immediately alerted. Because our sensors are multifunctional, the same CPD solution can also provide Seat Belt Reminders, airbag optimization, intruder alerts and more.

Our sensor solutions go beyond the vehicle interior. As more cars occupy the roads, so do more scooters and pedestrians increasing the need for exterior detection and monitoring capabilities to prevent injuries or deaths. Vayyar’s ADAS sensors provide collision avoidance by incorporating cutting edge 4D point cloud imaging with an ultra-wide field of view both in azimuth and elevation. This enables holistic detection of multiple static and dynamic targets such as cars and pedestrians. The sensors support the following functions: Collision Warnings (FCW/RCW), Valet Parking (parking assistance), Blind Spot Detection (BSD), Lane Change Assist (LCA), Automatic Emergency Braking (r/f AEB) and 360^o Around Vehicle Monitoring (AVM).

We also focus on smart building sensors. Vayyar’s smart building platform includes intelligent sensors that map and track people in real time anywhere in the building. With more jobs migrating to the home and paid-for office space increasingly vacant, buildings must work harder to maintain a positive ROI.  Vayyar’s new platform delivers a full suite of smart building features that enhance productivity, cost savings, security and Covid-19 safety and hygiene requirements. Since Vayyar sensors do not use cameras, employee and visitor privacy is maintained at all times.

In the retail industry, Vayyar’s intelligent IoT sensors tell retailers what’s really going on in their store, on shelves and in checkout lines. They’ll turn any store into a smart store by providing insights that increase sales and improve profits. Our sensors monitor how many shoppers are in the store in real time and how long checkout lines are. This data is used to analyze sales conversion funnels and shopper behavior. Vayyar’s solution also monitors inventory to improve restock-times and eliminate lost sales.  All these insights are delivered through a customizable dashboard, that can monitor KPI’s and create benchmarks. This enables managers to analyze performance on the store level or compare across the organization.

CM: Could you describe Vayyar’s 4D radar imaging technology and highlight what makes it unique?

RM: Vayyar’s intelligent sensors can see through surfaces, classify objects and people, and track targets within an environment in real time. We’re the only imaging radar manufacturer with both a range of 3GHz-81GHz and a substantial MIMO array of up to 72 transceivers.

Part of our mission to make 4D imaging radar an integral part of day-to-day life, involves creating mature solutions that allow our partners to develop radar applications seamlessly over our platform. Our Radar-on-Chip features an integrated DSP, MCU and all other analog and digital RF components on one chip, eliminating the need for partners to develop the radar side or to invest extensive resources on all of the software and hardware behind a fully integrated radar solution.

Beyond all hardware needs, Vayyar’s solutions include cutting edge 4D image processing algorithms, full reference designs, generic APIs for application development & evaluation as well as mature applications (in relevant cases). In addition, Vayyar offers partners over-the-air software updates even after sensor deployment, to ensure ongoing innovation and optimization.

On the performance level, our proprietary 4D imaging point cloud produces rich images with unprecedented levels of accuracy, and ultra-high resolution, due to the large MIMO array. We can also reach LIDAR-like resolutions by with multi-chip solutions that are still more cost-efficient than LIDAR.

Vayyar’s sensors operate effectively in all lighting and weather conditions, while maintaining user privacy. They’re miniature, affordable and versatile enough to impact everyone’s lives, enabling a safer world.

CM: Does it apply to any specific market segment?

RM: These advantages apply to all markets. With better resolution, richer data is made available. This enables a better understanding of the environments the sensor is monitoring, allowing the vehicle, home or device connected to the radar to react accordingly. 4D imaging radar can see through materials and is robust in all conditions, so its data output doesn’t get compromised as the environmental conditions around it change. The environment we live in is complex, and requires versatile, reliable technology.

CM: Does it meet cost expectations of system makers?

RM: Yes, on multiple levels such as product, maintenance, deployment and development costs. Vayyar’s multifunctional radar reduces the number of sensors needed for any application and offers more features than any other single tracking sensor. Just one unit can be purposed for multiple different radar applications in stores, vehicles, homes and buildings. Inside a vehicle, a Vayyar sensor can enable half a dozen safety applications (Child Presence Detection, Seat Belt Reminders, Intruder Alerts, airbag deployment, etc), replacing up to seven different safety sensors. Outside the vehicle, four Vayyar sensors can replace fourteen different ADAS sensors, providing monitoring that is far more reliable and accurate, with more savings. In a retail environment, each Vayyar sensor can be used for shopper analytics, inventory management and storefront traffic monitoring. For homes and smart buildings, a single Vayyar sensor can detect presence, automate appliances, monitor health and secure a perimeter.

We offer an advanced 4D imaging radar-on-chip with the best price-performance balance – high resolution and high performance at the price point of traditional 2D radar sensors. Vayyar provides our partners with an end-to-end imaging radar solution, complete with analog and digital RF components, MIMO array, a compact PCB with a full reference design, advanced algorithms and an SDK with integratable API. This eliminates the need to develop software, additional hardware and algorithms, giving partners a complete application-level system, ready for seamless integration. By receiving a full end-to-end solution, our partners save substantial effort, resources and time, while drastically reducing direct and indirect costs. In addition, our innovative engagement model enables partners to access our massive testing database for use in evaluation and development, ensuring fast development of more functions and an even faster time to market.   

CM: How about power consumption?

RM: Vayyar sensors consume very little power and transmit even lower power – 1000x weaker than WiFi.

SE: System Plus consulting has evaluated the cost of the Walabot system. Vayyar’s RF SoC, which features in this system, represents almost 25% of the overall cost. Could you elaborate on the added value of the RF SoC with respect to this end system?

RM: The RF SoC is the heart of the device. It includes all the RF components for MiMo radar which, in a discrete implementation, would increase the cost dramatically and hurt the ability to cater high end technology for a low cost. Because it’s a full end-to-end solution that includes a powerful DSP and MCU, all complex processing is done internally, and the technology is cost effective.

Both in the case of the Walabot, and in all our other tracking use-cases, the RF SoC acts as an IoT system that makes any device “aware” of its surroundings. An appliance can suddenly act as a health monitor or security system. A room can now monitor vital signs.  

By adding the RF component to other products, they instantly gain people-tracking capabilities, while becoming more efficient, automated and multifunctional. People-tracking is becoming increasingly popular in wellness, safety and security, and our RoC enables this functionality without invading privacy and without being affected by line-of-sight.

SE: Can we expect the same results for 60 or 80 GHz radar systems?

RM: Partners can expect an exceptional RoC-to-system-cost ratio as our chip includes all the needed components to act as a full Radar system. From a development-cost perspective, our mmWave solutions are also entirely end-to-end (including all RF components, supporting hardware, reference designs, software, algorithms and APIs). This enables partners to develop applications over radar with minimal resource, time and money because they don’t have to focus on anything beyond the application development. The above is relevant both for 60GHz and 80 GHz designs.

After extensive testing and evaluation of both our 60 and 80GHz imaging radar solutions, we’ve achieved industry-leading accuracy rates for tracking, presence detection and classification across a number of industries, including automotive, senior care, retail and more. For senior care, we’ve tested thousands of fall scenarios to ensure maximum reliability and minimum false alarms. In the automotive domain, we’ve extracted millions of test samples with adults, children, pets and objects in cars. We’ve validated our technology under several different conditions such as moving vehicles, large spaces, darkness, smoke, steam, inclement weather and poor visibility. In both frequencies, the sensor has reliably monitored its surroundings and provided accurate results, even when it did not have line of sight with its targets.

SE: It seems that Vayyar is also investigating the consumer market. Is this as a result of customer demand ? What is the target application ?

RM: Relating radar back to smartphones is a very fair assumption to make. It is clear that in the future we will see high resolution 4D radar imaging embedded into different consumer products such as smartphones, home appliances etc., enabling much more than gesture recognition.

Vayyar has garnered significant experience with consumer markets through the home improvement and senior care markets. Our primary consumer product is in the home improvement industry. It’s a plug-and-play visual wall scanner called Walabot DIY that connects to cellphones and uses UWB imaging radar to detect studs, pipes, wires and movement behind the wall. This helps prevent DIY disasters such as hitting a wire, bursting a pipe or missing a stud.  Customer demand has grown significantly since the launch, leading to the development of 3 different generations of the device. With each generation, we’ve expanded Walabot DIY’s range of wall compatibility and operating systems it can be used with.

Our senior care solution, Walabot Home, was initially marketed to consumers in an effort to better understand the market and to meet our business partners’ needs.  Our current strategy is to mitigate channel conflict and to ensure that Vayyar Home is available through multiple B2B relationships serving the residential and assisted care home facilities.

SE: Other devices, such as home assistants, are very popular. Do you track this, and would you suggest 4G imaging radar as a new generation HMI ?

RM: You’re correct about the potential for 4D imaging radar applications in the home assistance domain. Vayyar Home provides rich data about resident or occupant presence and health, enhancing the functionality of existing smart home or office systems. It is continuously aware of how many people there are, their location, and whether they’re moving, standing, sitting, or lying on the floor. The sensor integrates seamlessly with smart home systems to turn any home—small or large, old or new—into an intelligent home with all the latest features, including home automation, senior care, security, healthcare, and energy efficiency capabilities. Vayyar Home can also sense and display each person’s vital signs. It monitors activity patterns, breathing and even health problems. For security applications, the system protects people and property from intruders. It can interface with security systems and emergency responders to monitor any home or office. Given these rich abilities, our sensor can be embedded in different home appliances and smart home systems.

Vayyar sensors can also be found in a number of robotic personal assistants, both for senior care, medical and convenience applications. They’re ideal for consumer robotics and smart home systems because of their ability to accurately track people while maintaining privacy. If a home assistant or smart appliance needs to know where its human companion is, 4D imaging radar can help the device track its owner within a room or even on a street. Because it’s not optic-based, this technology doesn’t collect any identifiable data, and works in all lighting and weather conditions. All of this has the potential to make it an instrumental component in HMI systems.

SE: Will you develop a machine learning engine for such advanced features ? Or cooperate with other companies?

RM: Vayyar already has an active machine learning engine. Our senior care solution engages in a self-learning process upon being installed in a home or facility, in order to map the room and accurately monitor it. Our Child Presence Detection functionality is constantly trained and enhanced by the database of samples we’ve collected, to ensure outstanding accuracy. We combine classic signal processing algorithms with a neural network to facilitate this.

SE: Would you like to add some final words for our readers?

RM: We’re excited about the potential that 4D imaging radar has on everyday life – in homes, offices, stores, public spaces, mobility settings and the medical sector. Our mission is to continue expanding the potential of consumer radar applications, so we can deliver the next generation of sensing technology that is miniature, affordable and versatile enough to impact everyone’s lives.

Interviewee

Interviewers