Seeing With Sound
Up until now bats have been the advocates of being able to navigate with sound. They rely on echolocation to detect obstacles in flight, forage for food and to “see” in dark caves. Now the Munich-based startup Toposens managed to extract the bat’s trick to navigate and use it to detect people and objects accurately. The company thereby developed the first 3D ultrasound sensor worldwide.
Toposens provides echolocation, i.e. the ability to “see with sound” for the automotive industry and robotics. Up until today, ultrasound has only been used for one-dimensional applications, e.g. precise distance measurement for industrial applications. Toposens invented the first 3D ultrasound sensor system. It can detect multiple objects and people in real-time and generate a 3D point cloud of its near-field environment.
By using the principles of bionics i.e. engineering a reliable product based on biological methods found in nature, many of the sensor's features are closely related to the echolocation system used by bats. Toposens sensors send out signals, evaluate the echoes that come back and precisely detect where in a given space both static as well as dynamic objects are located in real time.
Even though ultrasound has been studied by scientists for over 200 years, its true capabilities have not yet been unleashed. Ultrasound technology provides many interesting characteristics that have not been combined into any other 3D sensor yet. Toposens' current sensor TS ALPHA is very robust and can see in the dark, under varying lighting conditions, through dust and dirt and does not have any problems with transparent or reflecting surfaces. It currently operates at the lower end of ultrasound with approximately 40kHz.
To give potential customers the possibility to test the sensor in various applications and under multiple conditions, Toposens launched a Development Kit for testing and prototyping purposes. The latest Development Kit TS ALPHA covers applications in both the robotic as well as the automotive field. To simulate a real-world use case where the sensor evaluates the near field environment during relatively slow, Toposens recently tested the Development Kit in a specific robotics parcours.
Toposens engineers challenged the sensor unit TS ALPHA by mounting it
onto a robot and sending the robot through a parcours consisting of
various obstacles located in the close range. Under dimmed light and bad
visual conditions, TS ALPHA stepped onto the battlefield. Navigating
through the parcours, the sensor - while still in movement - had to
detect a transparent bottle and a bowl consisting of very fine grates
and fight its way through dynamic obstacles in the form of coworkers
reaching out for the sensor. A pipe of convex shape, fluffy fabric
surfaces and a door mat rounded off the parcours for the ultrasound
sensor. While TS ALPHA felt at home in his natural habitat and
calculated the 3D coordinates of all nearby objects in real time, those
same objects would not have been detectable under the same conditions by
conventional cameras, radars or LiDAR systems. Due to limited
visibility and lighting but also due to the shape and characteristics of
the objects like the transparency of the glass bottle or the fine mash
of the bowl, the objects would be difficult to detect in the raw data of
existing sensor technology. These issues of reflecting and transparent
surfaces or bad lighting conditions appear frequently when sensors are
being used in the automotive field. Darkness, fog, wind and sun do not
interrupt ultrasound, whereas the weather heavily affects cameras and
As outlined in the parcours experiment, 3D ultrasound technology has strong benefits when it comes to near-field sensing. It covers objects in the opening angle of up to 180° (horizontally and vertically), has a range of up to 5m and generates real time data in the form of clustered data points.
Toposens is using a minimalistic approach with the TS ALPHA Sensor by limiting its size while keeping all of the functionalities at a peak level. Standard hardware components are put together in a special layout and combined with proprietary algorithms, machine vision and AI-software. The hardware components are off-the-shelf. They are very small, use little energy and weigh below 20g. Unique is the ability of the sensor to immediately evaluate the received data on a small built-in microcontroller, making an external control unit obsolete and keeping the generated data flow low. Only for further clustering and processing of the data, additional hardware is needed. The company's main knowhow lies in the sophisticated algorithms, which turn the ultrasound signals into 3D coordinates.
Besides covering the long distance, a fundamental requirement for autonomous vehicles is to perceive their near-field surroundings as precisely as possible. Toposens sensors provide reliable, rich and three-dimensional data (point cloud) for the close-range environment around the vehicle. The sensors are therefore well suited for applications in the automotive field and are able to cover areas, that conventional radar and camera technologies are struggling with.
For improved accuracy and safety in everyday parking situations, sensors have become an essential in every car. Up until today, they helped the driver steer more safely and guide him into a parking spot via sound and light signals. Although the currently installed sensors provide a good guideline to avoid collisions, while navigating the car into a tight parking spot, they still leave a lot of room for improvement. Conventional ultrasound sensors used for parking assistance only record one-dimensional data, namely the distance to the closest object. Azimuth and especially elevation angles of objects can hardly be calculated, and vertical opening angles must be limited strictly, so many objects - such as the curb - cannot be detected at all.
As every drive starts from a parking position and ends with a parking maneuver, close-range sensing is one of the crucial challenges in going from L3 to L4 automation. Thinking about autonomous parking and precise manoeuvring in tight parking spaces, distance measurement alone quickly becomes insufficient. In order to take things to the next level, more sophisticated sensors are needed around the car. By ultimately using 3D ultrasound sensors, the car's surroundings can be evaluated accurately. Furthermore, blind spot monitors are an option that may include more than monitoring the sides of the vehicle. What mainly differentiates the technology from existing parking sensors are both the algorithms and the special layout which allow 3D sensing with commoditized, readily available hardware.
The sensor data is further useful for additional comfort features, e.g. gesture control to open doors and trunks, positioning the vehicle for automated charging (for EVs), collision avoidance for automatically opening doors and many more.
With further improvements in autonomous driving space, the behavior of the driver is also likely to change. While drivers today have to be completely focused on the road, ready to react every millisecond, this could change once cars can steer and break fully automatically. The drivers can lean back and relax, work on their computers, turn to the children in the back seat, or temporarily enjoy an expanded infotainment program.
In this vision of autonomous driving, drivers can use their travel time for activities besides driving. This puts new demands on assistance systems. Just like the numerous sensors available for analyzing a car's external environment, similar knowledge is needed for the interieur in order to realize a more secure and intuitive interaction experience. In this scope of application, the usage of 3D ultrasound provides interesting advantages. Data gained from the 3D ultrasound sensor is able to identify the number of people sitting in the car, their size and their posture. Based on the information regarding where people are sitting and how big they are, airbags could also be adjusted to individual body sizes to further improve safety. Toposens technology does not collect any personal data since ultrasound cannot evaluate visual material. Instead, anonymous point cloud data is recorded. This is an especially important consideration in terms of privacy and data protection. Last but not least, gesture simulation in the interior of the car can be used for information and entertainment purposes, e.g. controlling your car's infotainment systems with simple gestures.
There is no doubt that future autonomous vehicles need more assistance to operate safely in our everyday lives. Whether you are living in a big city where people detection and accurate parking plays a major role or on the countryside where automated charging can assist you, sensors will have a big impact on your everyday life. Toposens contributes to the high quality of sensor output and keeps developing new products to accelerate the advancements in sensor technology.