Sensor fusion is the ultimate form of sensor integration. Moore's Law enables the combination of diverse sensor types at the chip level within a single sensor module. While sensor manufacturers focus on perfecting their sensor technologies, Sentech independently works on the integrated sensors of the future. Read why sensor fusion enables next-generation applications.

Business development manager Marco Leeggangers responds enthusiastically to the latest sensor technologies. “Old and new techniques at the chip level are emerging. With new sensor techniques, the sensor manufacturer focuses on the further development of one technology. We see many opportunities for the integration of different sensors into one compact sensor application.”

What is sensor fusion?

When you google a Explanation of sensor fusion, the impression arises that it concerns sensor data. The term is also often equated with ‘multisensory data fusion’. Or the combining of data from different types of sensors in one system.

Leeggangers believes that is too limited a definition. “It's not just about data. True sensor fusion is combining sensor technologies in one integrated sensor module or application.” According to him, this offers many advantages. It also makes new applications possible because “more difficult detections” are feasible. In the following, you will read how fusion elevates autonomous movement to a higher level.

Diverse types of sensors examined

According to Leeggangers, Sentech regularly receives requests from startups and research centers to bring promising high-tech sensors to market. “We see various types of sensors and promising sensor technologies come through. Sentech focuses on innovation in sensor integration, not on mass production of sensors.”

Ultrasonic sensors

An ultrasonic sensor works with sound that is imperceptible to the human ear. This type of sensor is used in all sorts of detection applications. For example, for person detection, quality control, and for medical purposes.

A major advantage of ultrasonic sensor technology is the simplicity of processing detection signals. This technology is also relatively inexpensive. However, sound detection also has limitations, for example, the need for a controlled environment. The speed of sound is influenced by all sorts of factors.

Lidar and radar sensors

Lidar and radar sensors measure according to the same principle: ‘time of flight (TOF)’. The reflection of an emitted signal is received and processed by a receiver. By measuring the time between transmission and reception, the position, size, and speed of an object can be measured. Lidar works with light pulses (laser or infrared) and radar with radio waves.

Since both signals travel at the speed of light, detection is lightning fast. According to Leeggangers, sensor manufacturers are currently investing heavily in the further development of these sensor technologies, particularly to enable autonomous driving. Think of UAVs (unmanned aerial vehicles and drones) and AGVs (Automated Guided Vehicles).

Which method is preferred is a continuous discussion among users, manufacturers, and independent experts.

Leveraging the advantages of lidar and radar

“Innosent, a manufacturer of radar sensor technology, will particularly emphasize the advantages of radar. And Lidar expert Leddartech will underscore the benefits of Solid State Lidar,” explains the business developer.

Lidar scanning has more limitations in extreme weather conditions (such as snow, fog, and rain) than radar. On the other hand, radar is less capable of accurately determining the size and shape of objects. Furthermore, the resolution becomes less accurate as distances increase. Radar also requires more software filtering to remove interference.

“At Sentech, we integrate Solid State Lidar technology. The latest generation is much smaller, more robust, and more reliable due to the absence of moving parts. And radar has become significantly cheaper because it's now possible at the chip level,” says Leeggangers.

Suitable for autonomous movement

In the automotive industry, ultrasound, lidar, and radar are used separately for various autonomous driving functions. Such as lane assistance, parking assistance, cruise control, anti-collision systems, and so on.

The Netherlands is at the forefront of AGVs and UAVs in the Agriculture and horticulture. With drones, farmers keep an eye on their land. Robots clean stables, milk cows, and feed livestock for farmers.

Sensor fusion for next-generation applications

Sentech uses sensor fusion as the ultimate integration tool to enable next-generation applications. According to Leeggangers, there are no bad sensors. “However, a sensor is sometimes used incorrectly, which leads the user to see it as a bad sensor,” he says.

“We look at the customer's application, think about what they want to achieve with their application. Based on that, we select the best sensor technology and integrate it. That increasingly leads us into the field of sensor fusion. The combination of two sensor techniques yields new information. That information makes the customer's application smarter and better,” explains the product developer.

Chip-level development

Transmitters, receivers, and printed circuit boards are getting smaller. “That is also necessary to enable innovative integrations,” says Leeggangers. Weight, installation space, and power are limiting factors that require small-scale sensor development.

This is where Moore's Law also applies. The number of transistors in an integrated circuit doubles every two years. And according to Leeggangers, this offers opportunities for sensor fusion. Radar and lidar sensors with chip-sized transmitters and receivers are already available.

More complex detections possible with fusion sensor

As a sensor integrator, Sentech operates independently of sensor manufacturers. “There isn't one all-encompassing technology that can accurately detect everything yet. By combining sensor techniques, we want to enable more complex detections,” says Leeggangers.

“To allow a vehicle to move fully autonomously on the road or in a business environment, you must be able to detect and process all variables in the environment. Our primary focus is now on Agrotechnology.”

For example, Sentech works closely with Lely to enable advanced barn automation. “With sensor fusion, we are driving efficiency on farms, but also animal welfare and reducing environmental impact,” he concludes.

Fully automated herd management is still a long way off. However, feeding and manure robots are already bustling around many livestock farms, determining their position with sensors. According to Leeggangers, the next step is communication between fusion sensors in machines, vehicles, on the livestock, in the barn, and in the pasture.

Combination of high-tech sensors ultimate for integration

Sensor fusion therefore appears to be the ultimate integration technology. If you also (frequently) experience detection limitations and sensor challenges, then this technique is promising.

The question is not if, but when fully autonomous driving will arrive on public roads. The latest Teslas can already do it, and Automated Guided Vehicles (AGVs) are commonplace. The vehicles of the future will combine advanced technologies. Here, you can read about which sensor technologies these are and what their advantages and disadvantages are.

A pilotless airplane or a driverless bus is possible in the foreseeable future. Only legal and psychological objections stand in our way; just as the steam locomotive caused controversy and challenges in the 19th century.

“Cameras and various types of sensors in fused sensor applications are the eyes and ears of the future drivers of our cars,” predicts business development manager Marco Leeggangers.

The evolution of autonomous movement

Autonomous driving was one of the main themes at the IAA Frankfurt this year. The automotive industry is working on technologies that enable completely autonomous movement in public spaces.

The automotive world uses a Scale level from 0 to 5. Level 5 for a fully automated car ride, while you read a book or watch a movie.

According to Leeggangers, all new car models must be automated at level 2 from 2018 onwards to receive a 4- or 5-star safety rating. “The car will then be equipped with Advanced Driver Assistance Systems (ADAS). Such as Automatic Emergency Breaking, Lane Assistance, and Road Edge Detection.”

Tesla has made the leap from ADAS to autonomous in its latest models. The latest version of Tesla's Autopilot is already balancing on the border of level 4 and 5.

Business applications: AGVs

Businesses have long been using autonomously guided vehicles (AGVs) for distribution applications in particular. In many distribution centers, automatic forklifts operate, and order picking is done by robots.

The Netherlands leads in innovation in Agricultural and horticultural automation met UAV's (drones) and AGV's (robots for cleaning stables, feeding livestock, and performing logistical tasks in greenhouses).

Why do we want self-driving vehicles?

Idlers: “In my eyes, this is a logical consequence of technological evolution. Actually, autonomous driving fits with the digital revolution because large amounts of sensor data need to be processed to react independently to the environment. Moreover, the self-driving car is part of the Internet of Things (IoT).”

The benefits of autonomous vehicles are numerous:

• Positive impact on traffic safety. Advanced computers can perform human tasks more efficiently, better, and safer.
• Better utilization of road capacity. Self-driving vehicles drive at shorter distances from each other. This allows them to utilize road capacity more efficiently, reducing and even preventing traffic jams.
• Improved car-sharing opportunities. The use of the self-driving car can be planned so that we can share it. The car for commuting can be available for someone else during the day. Autonomous driving will boost the predicted sharing economy.
• Sustainability: AGVs perform their tasks more efficiently than humans and save raw materials and energy in various industries.
• Productivity: An AGV never gets tired, can handle heavier tasks, and operates flawlessly.
• Cost savings: AGVs enable the full automation of distribution processes. Mobile robots also help reduce costs in agriculture and horticulture.

Detection challenges for distance measurement and positioning

To enable a vehicle to drive autonomously, it needs a comprehensive view of its surroundings. There are four detection challenges for dynamically generating an environmental model.

1. 1. Determining the clear passing space on the road surface.
2. Determining the geographical route via the navigable space.
3. 3. Detecting moving objects (other road users and moving obstacles).
4. 4. Recognizing and interpreting road signage, such as traffic signs, traffic lights, road markings, and other visual cues.

Sensor technology has advanced so much nowadays that there are solutions for all detection challenges.

Detection tools for autonomous vehicles

For autonomous driving and advanced driver-assistance systems, primarily radar, lidar, and sonar sensors applied. Combined with cameras and GPS, a vehicle thus dynamically scans its environment. Smart software processes the large amount of data, allowing it to always know its position relative to objects.

These techniques are possible because processors have become increasingly powerful and smaller.

Sensor technology development

Leeggangers indicates that Sentech plays a role in the development and R&D of sensor technology for AGVs. “For example, we already use radar, lidar, and ultrasonics in distance sensors and orientation sensors. As an independent sensor integrator, we are now working on integrating radar and lidar into compact ‘fused’ sensor applications.”

According to the Business Development Manager, sensor fusion leads to smarter and better customer applications, specifically in the area of autonomous movement.

Pros and cons of sensor techniques

The most promising sensor technologies for self-driving vehicles are lidar and radar. Lidar scans the environment with light (laser or infrared), while radar does so with radio waves. “The development of lidar and radar is progressing very rapidly. This is because processor chips are getting smaller and the technology has become more affordable,” according to Leeggangers.

Lidar has significant advantages in remote sensing. One of these is its high resolution, which is necessary for accurately detecting stationary and moving objects. On the other hand, weather conditions like fog and rain have a greater negative impact on accuracy. “Lidar is suitable for observing moving objects in the immediate vicinity of a vehicle,” explains Leeggangers.

Radar can see further, but as the distance increases, accuracy decreases. Therefore, according to him, radar is more suitable for remotely detecting moving objects in front of the vehicle.

The future of self-driving vehicles

“What's special is that the technological visions of car manufacturers differ. One prefers lidar, another prefers radar. The car manufacturers have a sensor-based system as a common starting point. We see a future with advanced fusion sensors in integrated sensor applications,” says Leeggangers.

He also sees new players on the autonomous driving market with a different technological approach, such as Google and Intel. Google has developed its own 3D technology, based on route information and 3D maps.

Intel, the processor manufacturer, has entered the autonomous driving market with the acquisition of Mobileye. The technology concern expects its first self-driving car on public roads in 2021. Intel uses the most advanced visual technology (cameras and software) in vehicles for environmental perception.

However, Leeggangers expects sensors to remain critical links in autonomous driving technology. “You will always need redundant sensor systems to supplement camera or GPS systems. No matter how advanced, anything can break. Redundancy will therefore become increasingly important as the fleet evolves toward full autonomy and driverless traffic.”

More about the development of lidar and radar

Sentech is focusing heavily on the further development of lidar and radar sensors, with an emphasis on sensor fusion. These are the most suitable sensor solutions for autonomous movement in public spaces and business environments.

Sensor fusion is the ultimate form of integration and enables next-generation automotive applications.

Read more about it and let yourself in good direction send.

Do you think the ISO 9001 quality standard is sufficient for developing a sustainable sensor solution? This is not the case for industries such as Automotive. They go a step further and work with IATF 16949. In your market as well, this high quality standard ensures a reliable and sustainable end product. You can fully adapt such a process to your quality needs. Read in this article what IATF entails and how to apply it to your project.

IATF 16949 is a step up from ISO 9001. But how do they relate to each other? According to Marco Leeggangers, Operations Director at Sentech, you can compare it like this: “ISO 9001 is equivalent to the Eredivisie (top Dutch football league), and IATF is like the Champions League.”.

Quality monitoring at the highest level

With the 9001 standard, companies accurately record how they operate. By following processes, the result is established and quality is monitored. For the automotive industry, that is not enough. The bar is set considerably higher there. “Significant requirements have been added to ensure quality even better. Such as the way of developing and producing, work processes, the development of your people, and continuous improvement,” according to Leeggangers.

If you want to supply parts to the automotive industry, you must be IATF 16949 certified. The collaboration with DAF was the reason for Sentech to obtain such a certificate. The latest certificate is valid until July 9, 2021. The Operations Director explains how it went: “It took about a year and a half to meet the strict standards of IATF. Working according to IATF means documenting your development process through Advanced Product Quality Planning (APQP). In this way, the standard ensures that you develop a product and production process through a well-thought-out step-by-step plan.”

Ultimately, it's all about the customer's request. “At the start of a collaboration, you discuss the delivery performance, logistical setup, warranty, and lifespan, among other things. By following the processes within the strict Automotive standard, we meet those customer wishes,” says Leeggangers.

How does IATF 16949 work?

IATF 16949 requires companies to use so-called ‘core tools’. These are prescribed methods, tools, and documents that are used throughout all development phases. Leeggangers explains what such a process looks like: “The first phase of a sensor integration project consists of a feasibility study. Failure Mode and Effects Analysis (FMEA) is part of this, through which we accurately map out risks.”

Analyzing and eliminating risks is also captured by the process. “The process forces you to document how you mitigate a risk. If a risk is truly too great, you investigate whether the design is even feasible. This clarifies early on whether the requirements are realistic within the agreed-upon frameworks. Additionally, this allows you to implement checks at the right moments in your production process,” explains the Operations Director.

Documenting for quality assurance

Continuous monitoring is central to the automotive standard. This also includes monitoring the expected lifespan. Leeggangers provides an example: “If a sensor solution needs to last one million kilometers, we develop a test setup and process for it. With those tests and checks, we gather important input for the production process.”

Monitoring also includes documentation according to IATF conditions. Although rules, documents, and procedures are often not an engineer's favorite activity, according to Leeggangers, they are very necessary: “In addition to devising a technical solution, you must also demonstrate that your idea corresponds to what you have agreed upon with the customer. Quality plays an important role in this. By checking your own work, you ensure that you ultimately deliver the quality the customer expects.”

Semicon also helped with IATF

The path to becoming IATF certified was a considerable investment, especially considering Sentech only had one automotive client at the time, DAF. “However, we quickly noticed that the quality system also adds value in other industries. Increasingly, customers from other markets are requesting documents and procedures. This is fully embedded within IATF,” explains the Operations Director.

A good example is the semiconductor industry, where quality and reliability are paramount. These companies in the Semicon sector often outsource projects. For their partners, it is therefore even more important to demonstrate that they deliver on their promises.

Efficient checks

Many processes in the semiconductor world take place in a vacuum. Johan van den Biggelaar, Senior Project Engineer at Sentech, knows that materials in these applications must not outgas: “Together with the client, we make agreements on how often we share an outgassing report. Sometimes they specify that we have to perform a residual gas analysis for each instance. Of course, that's possible, but it does increase the costs.”

However, a periodic check is sufficient if you ensure that the process remains the same. “Our sensor solution is often part of a larger module, which in turn is subjected to a residual gas analysis. Because our processes are thought out in detail, many of our customers opt for a semi-annual or annual check,” says Van den Biggelaar.

Accurate follow-up of work instructions

If anything in the process changes, you need to register it. John van Schaik, Production Engineer at Sentech, explains that a single detail concerning Grade 2 cleanroom components can have consequences: “For example, when cleaning an adhesive surface. If you replace the prescribed alcohol with acetone, that can cause problems later in the process.”

It is extremely important that production employees follow the work instructions accurately. “They should also always ask questions if they are unsure. No matter how insignificant the detail may seem. In such situations, we act quickly and consult with the customer if necessary. This way, we make the right decision and prevent mistakes. This is especially important in industries like Automotive and Semiconductor,” says van Schaik.

You don't just change a process. The Production Engineer explains how it's done: “Every adjustment to the process is tested again with an FMEA. If new risks arise from that adjustment, we trace them at that moment. Thanks to the IATF core tools, vigilance lives throughout our entire organization.”

Adjust quality level to your needs

The very high standards of IATF are not essential for all projects. For many clients, the guidelines of ISO 9001 are sufficient. Leeggangers admits that he and his colleagues found it challenging to strike the right balance: “We want to deliver quality. But if we follow the enormously extensive IATF procedure for every project, we are not accessible enough for some clients.”

Yet, the solution lies with IATF. “We start every project with the first phase, namely the feasibility study. At that point, we determine which steps of the process we will and will not carry out. This is how we adapt the quality level to the customer's needs,” according to Leeggangers.

Which 5 phases do you go through in a sensor integration project?

Every sensor integration requires a certain quality. You document this at the start of a project. Whichever market you are active in, the adaptable roadmap of the quality standard IATF 16949 ensures a reliable and sustainable solution for every sensor integration project.

The well-thought-out action plan is divided into 5 phases. In addition to mapping risks, these steps compel you to find the balance between quality, costs, and lead time. How do you develop a reliable sensor solution that seamlessly integrates with your application?

Door loop The 5 phases of a sensor integration project plan.

This article appeared in Mechatronics & Mechanical Engineering no. 4 2020 and was written by Alexander Pil